Load optimization results and parameters wishes

JDardon

#1

7/6/2011 5:47 PM

I use a lot of parameters in my strategies. I know (too many degrees of freedom issue) but that is not my case, as you'll see:
- Some of my parameters are intended for optimization (e.g. EMA period), I put an "O-" as a prefix to such parameters.
- Most of the others are used to control strategy behavior that has no influence in performance, such as controlling which indicators are displayed to avoid excesive screen clutter when trading. I put a "Dsp-" as a prefix in the names of such paramters.
- Other parameters I use control strategy behavior, but are seldom subject to optimization (e.g. Sync with Real trades, or "Keep pos overnight"). I put a "Ctrl-" as a prefix for such paramaters.

When I run optimizations I delete from the control window all the "Dsp-" and "Ctrl-" parameters, leaving only a few "O-" for the optimization to use. Once the optimization is done, I save the results to an xml file for future use, and then proceed to analyze the results in Excel or other tools.

When I come back to select the one I will like to use as Preferred Values for each symbol, I open the strategy and reload the xml optimization results, and here is where the problem comes. The strategy populates the optimization control with all the straps. The results loader sees a difference in the number of straps in the optimization control window and the saved results file and therefore refuses to load the results.

I have to go and delete each parameter that was not used in the optimization (hopefully remembering which ones were the ones I deleted, because if I forget them there is no way to load the results file with the corresponding waste of huge amounts of compute time) so that WL can successfully re-load my results. After reloading the results (when using Genetic optimization) the Fitness Graph and chromosomes are no longer available.

A) It would really be helpful if WL was intelligent enough to read the results file, figure out which parameters are going to be loaded and delete the others from the control (ideally after giving a warning to the user).

B) It would be great if the fitness graph could be saved and restored with the results file.

C) Even better, it would be great to be able to declare the straps with an intention for the strap (to discriminate which ones are intended for optimization and which ones for other controls). This way when going into optimization only the straps intended for optimization would be loaded by default in the control window (or perhaps a button to delete the ones that are not intended for optimization).

D) While on the subject of straps, the straps window that lists the sliders does not account for longer strap names than 13 chars. Longer names are not displayed completely. It would be nice if the sliders would adjust to longer strap names.

E) Being able to delete multiple parameters at once from the Optimization Control window (by pressing Ctrl key and selecting the ones you want to remove) would also be helpful.

Thanks.

Size:

Color:

I use a lot of parameters in my strategies.   I know (too many degrees of freedom issue) but that is not my case, as you'll see:
- Some of my parameters are intended for optimization (e.g. EMA period), I put an "O-"  as a prefix to such parameters.  
- Most of the others are used to control strategy behavior that has no influence in performance, such as controlling which indicators are displayed to avoid excesive screen clutter when trading.  I put a "Dsp-" as a prefix in the names of such paramters.   
- Other parameters I use control strategy behavior, but are seldom subject to optimization (e.g. Sync with Real trades, or "Keep pos overnight").  I put a "Ctrl-" as a prefix for such paramaters.

When I run optimizations I delete from the control window all the "Dsp-" and "Ctrl-"  parameters, leaving only a few "O-" for the optimization to use.  Once the optimization is done, I save the results to an xml file for future use, and then proceed to analyze the results in Excel or other tools.

When I come back to select the one I will like to use as Preferred Values for each symbol, I open the strategy and reload the xml optimization results, and here is where the problem comes.  The strategy populates the optimization control with all the straps.   The results loader sees a difference in the number of straps in the optimization control window and the [i]saved results file[/i] and therefore refuses to load the results.

I have to go and delete each parameter that was not used in the optimization (hopefully remembering which ones were the ones I deleted, because [b]if I forget them there is no way to load the results file with the corresponding waste of huge amounts of compute time[/b]) so that WL can successfully re-load my results.  After reloading the results (when using Genetic optimization) the Fitness Graph and chromosomes are no longer available.

A) It would really be helpful if WL was intelligent enough to read the results file, figure out which parameters are going to be loaded and delete the others from the control (ideally after giving a warning to the user).

B) It would be great if the fitness graph could be saved and restored with the results file.

C) Even better, it would be great to be able to declare the straps with an intention for the strap (to discriminate which ones are intended for optimization and which ones for other controls).  This way when going into optimization only the straps intended for optimization would be loaded by default in the control window (or perhaps a button to delete the ones that are not intended for optimization).

D) While on the subject of straps, the straps window that lists the sliders does not account for longer strap names than 13 chars.  Longer names are not displayed completely. It would be nice if the sliders would adjust to longer strap names.

E) Being able to delete multiple parameters at once from the Optimization Control window (by pressing Ctrl key and selecting the ones you want to remove) would also be helpful.

Thanks.

JDardon

#2

7/14/2011 1:06 PM

Hi.

Is somebody reading this?
I opened the xml files saved by WL and noticed that the results file doesn't have any metadata tags to describe the information being saved. Therefore if you restore a saved results file to a optimization control with the same number of parameters (even if they are different) it would probably succeed.

Is it too much work to add to the xml file the tags to indicate the labels of the parameters that were saved and the label for the performance measures that are being saved?

Cone

#3

7/14/2011 2:21 PM

I think your workflow is clever in the way you use parameters. However the "90% use case" assumption is that straps are of the "O-" type.

The change we have been trying to get high in the priority list of enhancements will help you get about 70% of what you asked for: item "C)". The idea is to simply provide check boxes in the Optimization Control (like we had in 4x) to include/remove straps from being used in optimizations. If we had that, A), E), and the last point aren't required.

JDardon

#4

7/15/2011 12:18 AM

That sounds good. It will be a great improvement. However, will it take care of the loading of the optimization results file knowing which straps were originally saved into it?

While on the subject, another question. When you run an exhaustive optimization the graph allows you to see a plotted surface of one performance metric and two parameters. If your optmization had three parameters, version 4.x used to allow you to select for what value of the 3rd parameter was kept constant while the other 2 were graphed. In the case of WL5 there is no option to chooose the value of the 3rd parameter, so what exactly is being displayed in the surface (perhaps an average of all the values for the 3rd parameter)?

Cone

#5

7/15/2011 1:00 AM

QUOTE:
knowing which straps were originally saved into it?

I'd hope so. Just to be clear, I still don't see this "in the pipeline", but we've done our part suggesting it to be considered.

QUOTE:
In the case of WL5 there is no option

Use the slider(s).

JDardon

#6

7/15/2011 4:28 PM

QUOTE:
Use the slider(s).

Right on...Thanks.... I guess I had forgotten about them. That does the job.

JDardon

#7

3/11/2014 2:34 AM

Hi. Any news on the future of the topics in this conversation?

Cone

#8

3/11/2014 3:23 AM

The only change that I see really having a chance to be implemented is the ability to choose which parameters to use in an optimization. We tried to piggy-back that on the Walk-Forward Optimizer enhancement, but it didn't go though. The enhancement list is long, but it could happen one day.