Hi all,
First I want to say "THANK YOU" to Eugene, LenMoz, and Cone for all their help, patience, wisdom and tips they've provided to this community. I've learned a lot reading posts on this forum!

I'm new to NNs and also to Neuro-Lab (and have only bee nusing Wealth-Lab for a month or so) so my questions are hopefully simple. I've also read through the excellent NL section in the WL user guide.

I'm having fun training different types of data series in input scripts of NNs, different training, etc. and am getting familiar with what effect they have on the output. But I am still rather confused about the output script. Most of the output scripts have code similar to this:

CODE:
Please log in to see this code.

1. Can someone explain in stupid user terms what this code is actually doing to the Bars data series? Why do we want to subtract earlier prices from later ones, and then multiply the result by 100 and divide by the later price?

2. Also, if this is supposed to make the NN predict 5 bars ahead, then I would expect the NNindicator output values to be offset back 5 bars from the actual prices when I see the indicator in a graph, but it isn't. In the graphs, the indicator moves in perfect step with the closing prices (if the input price goes up on a bar, then the indicator goes up at the exact same time (although sometimes not as much as the input), same thing if the closing price goes down. In other words, the NN indicator basically mirrors the closing price of each bar, which means when creating a strategy I don't know how the indicator can be any more useful than just using the current price? Could someone please help me understand what edge the NN gives, because so far it seems like it just mimics the price of the instrument! Maybe an example strategy that leverages whatever the NN has learned. (I've looked at the 3 provided example NNs, but i don't really know how to best use them in a strategy.)

3. Also, I've noticed that sometimes when viewing the NNindicator in a graph, it will suddenly flip/invert so instead of going up when the price goes up, it will go down instead. This doesn't always happen, but I have no idea what is causing it, and it freaks me out!

Thanks a bunch, and keep up the great work!

-Tim

Re 1.: It may be easier to see if the code is revised. The code you cite is equivalent to this one-liner...

CODE:
Please log in to see this code.

As the comment says, it gives "Gain after 5 bars", where a 10% gain is expressed as the whole number 10.

Personally, I like to express it like this

CODE:
Please log in to see this code.

The formula finds the gain in percentage terms.

Example:

CODE:
Please log in to see this code.

The % gain from 1/12 to 1/18 is:

CODE:
Please log in to see this code.

What does the code in the output script do? It aligns the actual [future] gain (truth data) with the base price series so that the NN's estimated gain can be compared to it for training. It just another way of stating what is said here: User Guide: Neuro-Lab® > Defining a Neural Network > Output Script

As for your observations, are they statistical? or are you just eyeballing it? Anyway, if your NN doesn't help predict future gain (or whatever measure you're trying to predict), then don't use it or try something else. That said, I would tend to expect a more gain with prices moving in the same direction, right?

Thanks Cone & LenMoz,

OK, thanks to that great explanation I think I understand what the code is doing now and how it is used to train the NN. So using the example output series above, am I correct to say that the NN would train itself to guess the price of the current bar using the actual percentage gain/loss over previous 5 bars?

In the graph, is the NN indicator displaying the value of the current bar based on the percentage gain of the previous 5 bars? If so, then why does the indicator appear inverted from the true prices (see attached screenshot)?

And I am still confused how to use this in a strategy, since it appears the best a perfectly trained NN can do is reflect the closing value of the current bar (rather than predict the next bar). In other words, in looking at the NN graph for each bar, I'm just seeing a reflection of the actual closing price. How could the indicator help predict a future price any more than just looking at the current actual price? Is there an example strategy that uses this indicator in any useful way?

Thanks,
Tim

Tim,

Well, you're right, a useful NN doesn't simply mirror the closing value. Attached is a picture of one of my NNs as used by a strategy. The top pane is the NN value.

Notice:
1. The peak (54.7) in September, 2016, is a bar or two after the lowest nearby price, and the buy signal was generated based on that peak.
2. The (much lower, 52.1) peak in November also generated a buy.

I typically only use NNIndicator(NN) for buy side rules...

The basic usage is:
Buy when NN > paramBuyScore (where paramBuyScore is a parameter tuned by optimization, usually above 50) and the NN is lower than the previous bar (buy on the downturn of NN). I sometimes also condition on NN < paramBuyLimit to screen out outliers. A very high NN often indicates bad data.

Though I've tried, I've not been able to use NN effectively in my sell rules. Other sell rules work better.

(Of course I only show you an example where the NN worked well) ;-)

Edit: You got a free stock tip! Notice that the last value of NN is above 52. This strategy is poised to generate a buy signal for AMD at the next downturn of NN. Edit2: Signal generated. Just bought AMD @ $9.92 (20Jan2017@9:35)


Edit: You got a free stock tip! Notice that the last value of NN is above 52. This strategy is poised to generate a buy signal for AMD at the next downturn of NN.

Len

Thanks Len,

OK, then if I'm understanding you right, it seems that we don't want to use an NN that tries to mirror the closing value, like the above example code does. If that's true, then I assume you are doing something different to create your output series. Could you perhaps shed any light on what a useful output series might look like?

Your strategy sounds reasonable, but I don't know what your NN values are supposed to be predicting or what the paramBuyScore represents (is it the current bar's closing price or some type of calculated value?).

Sorry I'm so dense about this. My gut sense is that NNs should be useful, but I'm just missing something obvious.
So far I'm just not seeing how to take advantage of them in a strategy.

Eugene or Cone, is there an example strategy that uses an NN in a practical way (that could potentially have better results than trivial RSI-based one)?

Thanks,
Tim
PS. Cone, in your one-liner [double x = 100 * (Bars.Close[i + predictBars] / Bars.Close[ i ] - 1)], why did you subtract 1 from the (i+predictBars)/i division result? This results in a different value for "x" than the example code.

My output script is (almost) standard...

CODE:
Please log in to see this code.

Modification? To not use outliers, I limit the range of the output Dataseries to 3 standard deviations.

There are 11 inputs to my neural network. Those I'm not willing to freely share.

QUOTE:
My gut sense is that NNs should be useful

They are the most powerful tools in my toolbox. I've used them for two decades.

Thanks Len,

I'm grateful for the help and completely understand your reluctance to share all of your code (if I had something that actually worked I wouldn't share it either!). Of course I realize that carefully choosing the input data (beyond simply feeding it the closing value) could potentially help the NN guess the closing values.

But it appears that your output script (even when enhanced to removes outliers) can't do anything except "simply mirror the closing value". And your graph seems to also show the indicator is more or less mirroring the actual closing value for each bar.

And we both agreed that a NN that does this isn't very useful (and as far as I can tell, provides nothing above just using the current price).

Yet you say that NNs are useful for prediction. So, unfortunately I'm still not understanding how an NN that mirrors the closing value can provide any benefit in a buy/sell strategy.

What am I missing?
Anyone else have any insight into how to use a NN in a strategy? I don't want anybody's actual strategy, just an example that shows how to use the indicator.

Thanks,
Tim

One more try.

Training:
The output script is only for training. It peeks ahead to see what actually happened 5 bars ahead. During training, a black box is built to calculate corresponding values of output based on the Dataseries in the input script. Training tunes the black box to minimize the difference between the actual (peeked) output from the output script and the calculated output, by adjusting the NN weights.

NNIndicator in a Strategy script:
Only runs the input script and the black box, producing a NNIndicator DataSeries. At each bar, the NN DataSeries value is the best guess of the gain 5 bars ahead, based only on the input DataSeries. When the NN value is high, the probability of a gain in the next 5 bars is high; a low value indicates a probable loss.

Hope this helps. If not, I would suggest external reading on neural networks.

Thanks Len!

That explanation helps a lot!

I sort of assumed this was how it worked, but what confuses me is that the NNindicator, when graphed, appears to display, at each bar, the value that is the best guess of the gain/loss on this bar (not 5 bars ahead).

You can see this in the graph that both of us attached. The indicator is moving up & down on the same bar that the actual value goes up & down (not offset 5 bars like I expected).

Why does the indicator not appear to be offset 5 bars? If it was, I would intuitively know how to use this indicator in a strategy. But since all I can see in the graph is that the indicator appears to predict the current bar (rather than a gain/loss 5 bars ahead), I don't know how to use it.
Do you understand what I'm trying to say?

Again, thanks for all the help,
Tim

QUOTE:
You can see this in the graph that both of us attached. The indicator is moving up & down on the same bar that the actual value goes up & down (not offset 5 bars like I expected).

I can assure you that this mirroring is not always the case. Part of it is the phenomenon that as the price drops, the probability that it will subsequently rise increases, since when the market rises and falls, all stocks tend to rise and fall too, regardless of their quality. This is the beta component of price action.

My goal in strategy development is to predict the alpha component, the price action attributable to a specific stock.

Hi Len,

Unfortunately it seems we are talking at cross-purposes.

My main complaint is that I still don't understand how to use the NNindicator as a useful stock price/direction prediction tool.
It seems that the main benefit of a neural net is the ability to predict a future value, rather than provide the current (already known) value.

Maybe my Neuro-Lab module is broken, but it seems that however the output script is used, it doesn't seem to make much difference in training, or at least in the values returned by NNIndicator.

As an extreme example of this, I decided to feed the output data series the bar volume, rather than the bar close price. Obviously training the NN to predict the var volume based on the close price is silly and pointless, but I wanted to see what the resulting NN would predict according to the NNIndicator values.

I'll let the attached pictures explain what I did, and the result I got.
Can you explain this result?

Thanks,
Tim

Can I explain it? Yes. Having only one independent variable, it doesn't matter what weights training comes up with (except zero), the output will be a multiple of the single input. It's not a meaningful test. Try using Bars.Volume in the input script and the NNIndicator will mirror the volume.

Hi Len,
You're right! I didn't realize that the NNIndicator will always reflect some multiple of the input values. That explains why the resulting NNIndicator didn't reflect the volume, but it also apparently shows that training with bogus data doesn't seem to affect the NNIndicator very much (it still mirrored the close values somewhat accurately).

I was confused and thought that the output script was where we provided the "goal values" for the training, and the NNIndicator would try to reflect these values based on values given in the input script. Part of my confusion was due to the wording of the the help page for the "Output Script" in the Neuro-Lab, (the highlighted text made me think that the NNIndicator would try to predict the values in the output script):

QUOTE:

Output Script

The Output Script Editor contains the network's Output Script, which determines the output value that the network is intended to predict. These values are used during the network's training process.
...
The Output Script is used only to train the network. It provides the truth data with which the estimates that are created by the final output of the network are compared. The training process uses the difference between the Output Series and the network prediction as a basis to adjust the weights of the neural synapses (the lines connecting the neurons).

The 2nd paragraph does say they are used to generate weights, but I didn't realize the output of the NN will only be output weighted versions of the input values.

If I understand you correctly, the values provided in the output script are only used to let the NN calculate weights to the input script values during training. After training the NN keeps the weights but totally forgets the output script values. The resulting NNIndicator will just take the same input variables you provided in the input script (but use the values for the selected stock/instrument), apply the trained weights to them, and return them.

Now I also understand why the "5 bar offset" (or any offset at all) in the output script won't (and can't) cause the NNIndicator values to be offset (and why it can't be used for predicting future bars). So now that I understand this, I tried testing 3 different NNs:

1= only bar.close for input, and bar.close for output.
2= close & volume for input, and close for output.
3= close & volume for input, and volume for output.

The graphs of the NNIndicators (attached) for 1 & 3 appear to reasonably track the cost and volume, but I'm not sure what NNIndicator 2 (in the middle) is doing (seems to be some combination of 1 & 3)? FYI, I also tested an NN with volume for input, and volume for output and it looked just like graph 3 (bottom).

I have a better understanding of how this NN works now. But unfortunately I still don't know how to get a benefit from it in a WL strategy.
(And I don't see any free strategies that use it at https://www.wealth-lab.com/Extensions/Strategy-Finder/ .)

Thanks,
Tim

QUOTE:
And I don't see any free strategies that use it...

Here's one you might use as a starting point. Use an optimizer to determine parameter values. (I use Particle Swarm Optimizer because of fine granularity parameters) In the code are comments to "Set to NN usable range". You can find the usable range during NN training on the "Evaluate Performance" tab. Rationale of the strategy...
1. Buy when the NN value reaches a threshold (parmNN)
2. Hold for at least parmMinHold bars, to reduce thrashing
3. Hold for at most parmMaxHoldBars, because in some circumstances an NN value can remain high even while losses mount
4. For the period between min and max hold bars, sell if NN value drops below parmNNHold
I always percent equity PosSizer, with the NN value as the priority, because on a down day, almost all NN values will increase, and more instruments will rise above the buy threshold than available buying power. NN priority should buy the best. Note also that I use AnnotateBar to show on a chart even those signals that were bypassed by the PosSizer.

CODE:
Please log in to see this code.

Thanks Len for providing a non-trivial NNIndicator example. It took me a little bit of playing around with that script to understand how it works.

The strategy will basically buy on any bar that has a NNIndicator value above the "(B)NN to buy" param, and as we know the NNIndicator is more or less a mirror of close price.

Which means that I think we could remove the NNIndicator altogether and replace the "if (dsNN[bar] > parmNN.Value)" in the strategy with "if (Close[bar] < parmNN.Value)", adjust the "(B)NN to buy" value to something appropriate (and do the same for the sell code) and we would have essentially the same strategy. Which is to say, a very poor one that only buys when the price is below some arbitrary value.

Thank you for all the help with Neuro-Lab. But since I still haven't figured out how to take advantage of it I guess I'll go back to the old-school formulas...

Thanks,
Tim

QUOTE:
we know the NNIndicator is more or less a mirror of close price.

That's the part you've gotten 100% wrong. It's only a mirror for your trivial input scripts. You need to put other indicators in your Input script, that you feel are indicative of future gain. I would never expect Bars.Close to be a predictor of future gain.Training the NN will be the first proof that you have a predictive batch of indicators. The simple strategy I provided above can provide a second level of proof.

You can use the WL-provided "RSI Periods" NN sample. It works fairly well on the Dow 30. Train it on Dow 30, last 60%, every 7 bars (~15K observations), for 1000 epochs. Then optimize the strategy I provided above over a 10 year period using 24.9% Percent Equity (because there are only 30 symbols). Remember to set the Start and Stop of parmNN and parmNNHold to the working range of your trained NN (from the Evaluate Performance tab) I just did that. I get a strategy having 10-year APR% of 19.37%, more than double BH% of 9.15%. For a very simple NN and rather simple strategy, that's not bad.

Hi Len,

Hmm, maybe my understanding of NNs is wrong. If the NN can't predict future bars using Bars.Close then how could could it predict future bars using a simple formula based on those values? I thought the point of an NN is to recognize a pattern/signal in the noise, so I assume they would work best on raw data rather than on formulas that modify that data. I also assumed we would only want to use a single instrument for each NN so it could concentrate on the patterns of just one entity, rather than trying to make sense of the movements of multiple instruments.

Anyway, I tried your suggestion of using the RSI Periods sample on the entire Dow 30 folder (I also added your trick of clipping back outliers in the output script). I trained it like you suggested. Then I used your above strategy and selected the last 10 years. Then I optimized for net using the swarm method profit with a portfolio simulation setting of 24.9% equity. After an hour of optimizing, the highest net profit was achieved with the following settings: (B)NN to buy: 45, (S)Min Bars:19, (S)Max Bars:82, (S)NN to hold:49.02.
These settings resulted in 474 trades. I assume it only made 477 trades because it quickly ran out of money. When I clicked on "Backtest on all Symbols in Dow 30" button I got a warning that 3044 trades were not included in the backtest results due to insufficient simulated capital. So I switched to "Raw Profit" mode and clicked it again. This time it made 3,518 trades. That is way more trades than I think anyone would be willing to make. But even with that many trades it still only had a win rate of 58% and an average profit of 5.11%. Did I do something wrong?

But I'm not sure I am understanding how this NN and the above strategy are supposed to work. It seems that for the NN, we feed it some random RSI values, and the NN assigns weights to them to try to get them to match the close price 5 bars in the future. So apparently the NNIndicator values will be those same RSI values, but modified/multiplied by those weights. Correct?
After training on the entire Dow 30, what does the the NNIndicator at each bar of each stock supposedly represent? It definitely isn't the bar close price! I'm assuming, based on the above strategy, that you think the indicator somehow has prediction power of close price, but I'm just not seeing that when I look at an example stock (See attached). But even if it did, why would we want to buy/sell when that value is above/below some arbitrary level? Wouldn't we want to compare the NNIndicator value to something related to current price movement?

Thanks for the help,
Tim

QUOTE:
I assume it only made 477 trades because it quickly ran out of money. When I clicked on "Backtest on all Symbols in Dow 30" button I got a warning that 3044 trades were not included in the backtest results due to insufficient simulated capital.

That's right, a 24.9% equity on a portfolio of 30 stocks lets one take only 3 positions (practically not even 4; here's some background: FAQ > I'm using 100% Equity position sizing and strategy doesn't seem to use all capital and/or there are trades not included due to insufficient capital.)

Couple of things...

QUOTE:
I assume it only made 477 trades because it quickly ran out of money.

I actually use that phenomenon as an integral part of the strategy. When the market drops, generally dsNN goes high for all stocks, resulting in insufficient cash. Which are best? Those with the highest dsNN. That is why it is used as Priority. The % Equity PosSizer is the final part of the strategy, only creating Alerts for the highest dsNN.

QUOTE:
... feed it some random RSI values, and the NN assigns weights to them to try to get them to match the close price 5 bars in the future. So apparently the NNIndicator values will be those same RSI values, but modified/multiplied by those weights. Correct?

They're not really random, are they? At each backtest bar those RSI values preceded a specific outcome 5 bars later. The weights are calculated to minimize the error. If RSIs are indeed predictive of gain 5 bars out, training should be able to determine those weights that result in the best correlation between RSIs and future gain.

QUOTE:
After training on the entire Dow 30, what does the the NNIndicator at each bar of each stock supposedly represent?

For that you have to look back at the NeuroLab "Evaluate Performance" tab when training the NN, specifically the "Predicted Output" and "Actual Output" columns. I've attached an image of "Evaluate Performnce" tab for an "improved" input script(shown below). I used a network topology having a single hidden layer with only 4 nodes. Simple networks result in more robust results, in my experience. Sidebar: From the Evaluate Performance tab, I learn that the working range of this specific NN is from 57.5 to 58.5. At 57.5, a no gain result is predicted. Above 58.5, the number of observations is becoming too small to be useful. I plug these values into the parameter definitions in the strategy script.

A backtest from 1/1/2008 to 2/2/2017 on the Dow 30, using 24.9% Equity, resulted in APR% of 24.36% v. BH % of 9.61%. Maximum Drawdown was only 21.29% v. BH Drawdown of 44%. I'll take it.

Finally, here's the "improved" Input Script...

CODE:
Please log in to see this code.

Hi Len,

Thanks for explaining the trick of using a high equity position sizer. Setting the position priority to the value of the NNIndicator is great idea to insure only the best trades are executed.

I suppose it is possible that RSI values could potentially project future movement for a while, and I guess I was expecting too much from a NN to figure this out by itself. I also like your idea of using a couple "difference RSIs" as input; I guess they could help detect a very recent meaningful trend.

I also like your idea of selecting only using the "useful" output range from the NN training. I'm not sure if it makes the strategy more accurate, but limiting the range of the strategy parameters surly helps speed up optimization.

So after all this, I'm starting to understand how the NNIndicator values could return predictive values, and that the values might not reflect the close price at all and still be useful.

Finally, your results definitely sounded promising, and I would be happy with them (although I would want to see how well the strategy would do on out-of-optimized data). Unfortunately I could not duplicate your results- I came up with a 17.21% APR versus a 8.12% buy and hold. That's still not too bad, but the number of trades was still very high (1546). When I then ran it on some earlier out-of-optimzed years it fared quite a bit worse (matching or slightly worse than a buy and hold strategy), which made me think that the optimized strategy params were very sensitive to the specifics of the optimized data.

I'm including some screenshots of what I did in case I missed something.

In spite of not getting great results, I can see that this NN strategy does indeed sometimes pick great spots to buy & sell. The only problem is that it also picks horrible spots too!

Thanks,
Tim