I am working on a WL Pro 5.1 day-trading script that operates on a portfolio of symbols.

I would like to be able to read and write variables that operate across the entire portfolio, and are evaluated sequentially in time. Example: after I get two losing short trades in the same day, set a dont_trade_short flag, always checked before opening a new position, and cleared when the date changes.

This needs to work in both backtesting and live trading.

C# programming examples for the solution or solutions would be helpful.

Thanks.

You could simply use SetGlobal and GetGlobal for that, or read and write to any place .NET allows you to, like files or database (which will guarantee the data is saved and retrieved even if you had to restart WL).

Since trading is dependent on trading for other symbols in the Portfolio, I'd just write the script that way. Give me a few minutes...

I just tried SetGlobal and GetGlobal, and although it compiles and runs, it doesn't do what I want, I assume because WL runs the script per bar per symbol rather than per symbol per bar.

Here's your symbol-per-bar example.

CODE:
Please log in to see this code.

Thanks.

Is this sort of script to be run on only one symbol in the data set?

Will this work in both backtesting and live trading on 1 minute bars? I vaguely recall that this type of script didn't work in live trading in WL4.

Is there an adverse impact on CPU or memory usage compared to standard type of script?

In the example, it looks like loserCount will be reset to 0 at each bar. I think int loserCount = 0 should go before the bar loop, and the following code or something functionally equivalent should go inside the bar loop:

CODE:
Please log in to see this code.

QUOTE:
Is this sort of script to be run on only one symbol in the data set?

Right. Click the symbol with the most complete history.

QUOTE:
Will this work in both backtesting and live trading on 1 minute bars?

You'll probably have to give this more thought for live trading and for intraday. That's because the architecture somewhat assumes that the DataSet is "synched", i.e., each symbol has a bar for every bar in the primary (clicked) symbol. Unless you're trading symbols that are sure to produce a 1-minute bar for every minute of the trading day, the secondary symbol synch will "mess with" the indicators.

The problem with Version 4 was that PortfolioSynch was limited to only the primary symbol, that's all. You could still trade it manually.

CODE:
Please log in to see this code.

Of course. It's going to take a lot longer because it has to process all the symbols in the DataSet. You'll have to be reasonable about the number of symbols for live trading. Of course, limiting the history when trading live will go a long way too.

How many 1-minute symbols are we talking about here?


Re: loserCount - You can do anything you like. This was just an example!

I am interested in trading hundreds of symbols on 1 minute bars intraday, which does not work so far in WL due to slowness of data update. I tested it again on the S&P 500 components in 5.1 a day or two ago; oddly enough, a quote window with all 500 stocks was fine, but running a strategy on 1 minute bars for the portfolio was not feasible because the update often took more than a minute. I understand that this is due to be rectified in 5.3, based on my past discussion here with you and on recent discussions by phone with Fidelity.

My question regarding resource usage was a comparison of two different ways of processing the same number of symbols: the standard type of script run on the entire portfolio, versus a script with a symbol loop that runs on one symbol, but loops through all of them in the portfolio. My question is whether the latter is more resource-intensive than the former; both will be processing hundreds of symbols.

I plan to trade high-volume stocks, but there is no gurantee that every stock will have a bar every minute, although I can select a symbol to run on, such as QQQQ, that will always have a bar. If a stock looped through does not have trades within a particular 1 minute period, what happens to indicators? To be more specific, what will WL provide at that particular bar interval for OHLCV? I can calculate my own indicators if need be, and am already doing so in this particular script (with C# arrays rather than WL series), so if WL does something silly like give zeroes, I can fill in with the last valid value.

Do I need to use a synch keyword somewhere?

Are you sure this sort of script can be run on live data? Isn't it likely that some symbols looped through will not yet have been updated at the time the script accesses them?

I seem to recall from previous discussions that the solution was something like keeping in global memory a count of symbols updated and a list of candidate trades, and after all the updates were complete (which might not be all stocks), in another section of code doing portfolio-level calculations and actual trades. This would involve a script that does not loop through symbols, and is run on the whole portfolio.

QUOTE:
I understand that this is due to be rectified in 5.3

There isn't anything extraordinary changing with streaming in 5.3, nor in the Strategy Monitor update (which is static data). There were plans to enhance getting data to the Strategy Monitor that I thought would have made 5.3, but it did not, and as far as I know right now, that project isn't even in work.

QUOTE:
what happens to indicators? .. Do I need to use a synch keyword somewhere?

See how Secondary Symbols are synched in the WS Programming Guide. (Same idea here: How Secondary Data Series are Synchronized) The basic idea is that synching to primary symbol will add (or delete) bars from the secondary symbols, which affect the indicators. The way to work with it is to disable synch when accessing the secondary symbols the first time, create the indicators, and then synch them. Just more programming required.

QUOTE:
Are you sure this sort of script can be run on live data?

I just remembered why this type of strategy wouldn't work in V4 - you couldn't loop through a WatchList because the concept of a WatchList for a streaming window did not exist. You'll notice that static, streaming, and simulator windows are all the same thing now.

QUOTE:
not yet have been updated at the time the script accesses them?

I can't say for sure for Fidelity data. My understanding of Fidelity streaming is that an interval is "closed" when a tick occurs within the next interval or 5 seconds after the interval mark determined by a internal clock. If a script is accessing external sybmols, I believe (not sure) there's logic that waits for their updates. In any case, I don't think anything like what you're trying to do with a DataSetSymbol script has been tried before, and I can't predict the results of the first attempt.

QUOTE:
recall from previous discussions that the solution

There are almost always other solutions. Try any that you're comfortable with. This is your task, not ours - Our Support Policy

The big issue has been the slowness of data updates on a large portfolio using the strategy monitor.

I didn't realize that in WL 5.1 I can run a strategy against streaming data. In looking at WL during non-market hours, it appears that streaming data can be activated in a strategy window, I hope on a portfolio.

I will try this during market hours. If it works well it may resolve the timely data update issue that has been the major roadblock I've run into.

Portfolio-wide statistics are a secondary issue I think I can resolve one way or another.

QUOTE:
appears that streaming data can be activated in a strategy window

Of course this has always been possible - even in Version 2 (perhaps Version 1, but that was before my time).

QUOTE:
timely data update issue that has been the major roadblock I've run into.

Maybe. It's not possible to do a Multi-Symbol Backtest while streaming. To run a whole DataSet in a single streaming window, you'd have to do it as demonstrated above - and as I mentioned, I don't even think it's been attempted before on intraday scales since strategies like this have traditionally focused on EOD data.

I'll try this during market hours.

Thanks for your help.

It doesn't work.

Too bad Fidelity doesn't support strategy execution against a portfolio of streaming data. All the elements seem to be there: support for portfolio-oriented stratagies, support for quote windows containing multiple symbols of streaming data. I don't know why the developers didn't connect these pieces together.

QUOTE:
It doesn't work.

I didn't think to try it myself, but what exactly did you see?

QUOTE:
I don't know why the developers didn't connect these pieces together.

There are lots of pieces to connect, and not all of them will ever be connected. With the exception of pairs trading, I don't think anyone has ever requested something like this for a streaming strategy. It's a 0.01% request and we're shooting just to get the features that 90% of everyone will use.

QUOTE:
I didn't think to try it myself, but what exactly did you see?

I tried your script above, with the incrementation of loser_count commented out. It worked properly on a portfolio in backtesting (running on one symbol in the portfolio and looping through the others). On a portfolio of 1 minute bars of the Dow 30, it executed but behaved in a way that did not seem useful. An hourglass appeared and never went away. The chart screen became black, other screens such as the list of trades turned grey. It gave an error message about not being able to get data on demand for one or two specific stocks, and repeated those messages every 60 seconds, so the data was apparently not actually streaming, at least for the 29 stocks not selected explicitly. The task manager showed network usage on a 60 second cycle. Sometimes it generated alerts, sometimes not; don't know whether alert behavior was proper. Basically, it was a mess.

QUOTE:
It's a 0.01% request and we're shooting just to get the features that 90% of everyone will use.

To clarify, my request is not necessarily for a script like yours above to work with streaming data, although that might be nice. My request is primarily for a standard script, without an explicit symbol loop, to be able to execute on a portfolio of streaming data, perhaps with a streaming option in the strategy monitor. Or with fast snapshot data.

I understand the constraints and prioritizations needed for a product, and if short-term intraday portfolio trading is not in high demand among WL customers or potential customers, it indeed doesn't make sense to put time and money into providing it.

Hi jsmith,

I was wondering if you have continued to pursue this and whether you have had any more success. I am very interested in the sort of ideas that you discussed above. Maybe that would make it a 0.02% request.

Thanks,

Dave

Hi Dave,

In a subsequent thread, I recounted my conversation with Fidelity that revealed that they plan to allow more symbols to be processed in short time frames, but not earlier than 2010. I do not know how well this will work, how many symbols it will allow to be handled concurrently, and when it will actually happen. Other important things WL needs to do are supply the missing data feed API documentation, and provide true 64-bit mode; lack of the latter hinders backtesting many symbols in short time-frames in WL 5. WL 4.5 is much better for this type of backtesting, using C# and the API.

I haven't found any other platforms that provide the capability out-of-the-box to execute on hundreds or thousands of concurrent symbols, although RightEdge can possibly be made to do this with some possibly extensive programming.

I may wind up writing my own execution framework, and using Wealth-Lab for trading algorithm development and backtesting.

Or I may wind up figuring out how to profitably trade on larger timeframes and/or with smaller portfolios.