What is the difference between running a backtest in Strategy Ranking and a Strategy Window?
Author: Carova
Creation Date: 2/25/2016 10:28 AM
profile picture

Carova

#1
I am seeing an issue where running a series of of strategies using the Strategy Ranging tool results in two separate problems:
1) Some strategies produce a "x" and do not run, but run perfectly when I click on the strategy and it runs in a separate window
2) Results that are achieved in the Strategy Ranking do not match results when run in a separate window, often by a major amount.

NB: All strategies use Priority assignments for all trades, so it is not an issue of Trade Selection.

If this matters, these strategies use Weekly scale, long time frames (1990-2016), and large WatchLists (~500 symbols).

Vince
profile picture

Eugene

#2
There shouldn't be difference (except maybe for output of cosmetic methods like PrintDebug being suppressed and certain auxiliary functions reporting that they run in the Strategy Monitor context). If you double checked the correct Priority assignment, make sure that Strategy Parameters and Preferred Values match and that other settings are identical (SSB vs MSB, DataSet etc.)

Re: red "x" - you can see what's caused this on the Errors tab.
profile picture

Carova

#3
Yes, I realize that there should be no difference, but I am seeing it.

Here is an example


but the ONLY difference between strategies with the same number but differing letters are the default parameters resulting from the optimization. Here is an example of the Error Message from the Strategy Ranking:
"MT for Eq - V.36h (Weekly): [A]WealthLab.Strategies.AMA cannot be cast to [B]WealthLab.Strategies.AMA. Type A originates from 'w10cme5z, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' in the context 'LoadNeither' in a byte array. Type B originates from 'ruehwth0, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' in the context 'LoadNeither' in a byte array."

Here is another bit of "strangeness":
If I run a Strategy Ranking with a number of Strategies and compare that to running a single strategy in the same window I get different results. Compare the results for "MT for Eq - V.33e" from above with a group of other Strategies to the image below when I remove the other Strategies and leave everything the same:


All I did was Delete the other Strategies.

Vince
profile picture

Eugene

#4
Hi,

You're onto something weird with that "cannot be cast to" error but seems like it might have to do with your Strategy code. Let's see if you can replicate the same condition with canned ones like "Moving average crossover".
profile picture

Carova

#5
Hi Eugene!

Also, just to be clear, my Priority assignment is very standard

CODE:
Please log in to see this code.


Vince
profile picture

Carova

#6
QUOTE:
You're onto something weird but seems like it has to do with your Strategy code.


Are you saying that my strategy code is influencing my other strategies?

Vince
profile picture

Eugene

#7
QUOTE:
Are you saying that my strategy code is influencing my other strategies?

Vince, at the moment only you know what's up there in your code. ;) We can't troubleshoot what we can't reproduce. So it's more practical to focus on what we have in the palm of our hand.
profile picture

Carova

#8
I agree, if it is internal to my code it is my issue. However, if there is some type of memory leak that allows one of my strategies to impact another wholly separate strategy that would be a whole different issue.

Vince
profile picture

Eugene

#9
Of course. But usually there's a simpler explanation than a memory leak. I'd like to have your primary suspect (your XML files) on the plate but if it's out of question, consider giving us a simplified test case.
profile picture

Carova

#10
Let me construct a test case that seems to display the problem. It might take a day or two.

Vince
profile picture

Carova

#11
Hi Eugene!

I think that I have narrowed down the source of my problem to the way I have implemented the Subroutine in the code below (perhaps with the cache?). What am I doing wrong?

Thanks!
Vince

CODE:
Please log in to see this code.
profile picture

Eugene

#12
Hi Vince,

Your indicator code looks (and runs) OK on the surface. Not seeing this as a problem yet.
profile picture

Carova

#13
Thanks Eugene!

I have constructed two strategies - one that uses the subroutine and one that takes that code and puts it in the main program. The one with the subroutine gives the strange results that I reported and the one with the code in the main program does not. The big difference appears to be that the subroutine code caches the data (which is why I used it)

Any ideas?

Vince
profile picture

Eugene

#14
The devil's in the details. Can you show the entire implementation?
profile picture

Carova

#15
I would prefer not to... ;) That I why I have been doing so much sleuthing myself.

Vince
profile picture

Eugene

#16
I'm afraid that the interprocess communication you're obscuring may be the culprit. Unfortunately I have no slightest idea of how it's implemented.
profile picture

Carova

#17
It is obviously incumbent on me to provide you with a strategy that demonstrates the problem. I will attempt to do so.

Vince
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).