Hi,
I have a simple strategy that takes 3 parameters; say p1 p2 and p3.
I wrote a code piece that basically loops over parameter 1, modifies the xml file appropriately, and then, let RunDonor to run it and yield some performance results. The code follows basically as follows (unnecessary parts are excluded):
CODE:
Please log in to see this code.
**
This loop gave me the NetProfit values for every p1_value alternative, programmatically, perfect! (Huge saving of time and big jump in the reliability of performance test, great!)
Then, I realized that the numbers that this loop yielded varied every time it is called.
So, i devised a very simple test to see whether I am or the runDonor is wrong.
The test procedure has been as follows:
First of all, i kept every variable constant. .i.e.:
1- I left the date range unchanged for each of the runs;
2- i kept the ticker unchained at each run;
3- I did not change the scale, nor position sizing setup as well,
4- i did not change the code of the donor strategy as well (only the p1_value changed programmatically with the help of this loop)
So, given this setup, I run this very same loop (for 5 alternative p1 figures), for 7 times.
That way, I retrieved 7 Net Profit figures for 5 alternative p1_values.
**
If everything was fine, each of the 7 Net profit figures, (for, say, the first value of p1) should be exactly the same.
In other words, this loop should return the same results for a given p1 value whenever it is called.
**
The outcome is really horrifying; of this 7 calls of the same loop for 5 different p1 values, the Net Profit results (that runDonor(DonorStrategy) returned) were completely different!!, not even close!!
To give a hint, the following is the list of ` 7 Net Profit results of a donor strategy, for the first value of the parameter p1. Theoretically, these 7 values should be exactly the same given that the parameter, data range, code, scale...everything kept unchained at each of these 7 runs):
List of p1 results for repetitive runs of the same code:
p1 = 1, NET PROFIT = '1482526.16000003'
p1 = 1, NET PROFIT = '1497802.42000003'
p1 = 1, NET PROFIT = '2679251.32000005'
p1 = 1, NET PROFIT = '2491140.16000005'
p1 = 1, NET PROFIT = '2465272.34000004'
p1 = 1, NET PROFIT = '2059570.21000003'
p1 = 1, NET PROFIT = '2364506.35000004'
In theory, these values should be the same!! Calling one method and getting different results every time is very scary and reduces to a great extent the reliability of the rundonor approach and programmatic calls of the strategies.
(And I hope it is me doing something wrong)
**
Here is my question:
- Is there any way (or need) to flush the previous calculations and global variables from this PerformanceResults - before it is recalled by the code?
- Or, what might be happening in the code and the very same code returns different values each time it is run?
Additional notes:
- i checked the state of the xml file, and it seems that it is updated appropriately before the runDonor method; So, that narrows the problematic area down to the runDonor method, i think.
- I checked another Result.property (Position.Count) and again, the code returns different results every time it is executed.
- I also checked the `correct` values (through manual run of the same strategy from Open>Strategy). None of the values I presented above were even close to the value I got through manual run.
- I switched off the commissions for backtesting to keep things simpler.
** Thanks,
Please help; being able to run alternative setups of the strategies programmatically is a huge efficiency and accuracy jump for me and I am this close to achieve that.
Best regards,
Aykut Saribiyik