Different backtest results on each run
Author: rmandel00
Creation Date: 4/8/2020 9:22 PM
profile picture

rmandel00

#1
Guys,
I am using the position options on possizers. I have it configured it for "Use Tag" "Percent Equity". The code for the buy signals are:
CODE:
Please log in to see this code.


When I run the code, I get a different set of output parameters including APR, MaxDD and Sharpe Ratio each time. A colleague of mine is running the same code on his machine and his results are invariant from run to run.

I went to extensions on the wealth-lab.com site and tried reinstalling the possizers extension. I still get the same results.

Any suggestions to correct this error?
Thanks,
RichMan

profile picture

Cone

#2
Make sure:
1. all the Preferences > Backtest Settings are the same
2. using same data (refresh to be sure)
3. using same Data Range
4. any other options enabled in the Pos Sizer?
5. using the same margin settings
6. Tools > Symbol Info Manager: unless you're testing futures, make sure the button at the top says, "Enable Futures Mode" (which means it is disabled).

There are probably 4 or 5 more things to check, but this a good start. Something's different, you have to find it!
profile picture

Eugene

#3
QUOTE:
I get a different set of output parameters ... each time

Apparently the Position Options PosSizer has nothing to do with this. About a decade ago we created this FAQ entry:

Every time I run a Strategy I get a different result. What am I missing?

Hope this helps.

P.S. BuyAtMarket(bar) in your code is wrong, this is a serious error (peeking into the future).
profile picture

rmandel00

#4
Cone,
I checked your 5 suggestions and they were all constant from run to run. In face if I just hit the Run The Strategy button each time without Compiling the results change. Please let me know what other 4 or 5 things I should check.

Eugene
Your link suggests:
You're running a Strategy in Multi-Symbol Backtest mode (MSB) using Portfolio Simulation mode (in other words, not Raw Profit) and every time it runs you get a different result. In addition, your entry has no position priority assigned.

I am only running strategy clicking on a single symbol (SPY) in the Data Set Pane. The code uses two strings to define the actual symbols that the strategy switches between namely string riskOn = "SPY"; and string riskOff = "TLT";
The buy and sell signals use the strings riskOn and riskOff. The trades in the Trades tab show both symbols being bought and sold.

Any other thoughts?
RichMan
profile picture

Eugene

#5
No other thoughts are needed. Considering that the strategy uses Percent Equity and indeed has no Position.Priority assigned as suggested in the FAQ, what said there still applies. Your position sizing settings are unknown as well as the script, so in their absence my only suggestion will be the FAQ again: I'm using 100% Equity position sizing and strategy doesn't seem to use all capital and/or there are trades not included due to insufficient capital, first paragraph. Given that your script already peeks into the future, it may contain other errors.
profile picture

Cone

#6
Eugene's right.. this is simply a Position.Priority issue. As often is the case, I read it too quickly and only picked up that you were getting different results then your colleague, but that you 'still get the same results' - which I took out of context. :/
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).