Is there any difference between:

a) Run an optimization on a Dataset in Raw Profit mode and then assign Preferred Values based on the the highest average value per symbol.

b) Run an optimization on the same Dataset in Portfolio mode with the same fixed position size, select the single row the highest metric, and then assign Preferred Values to all symbols from this row?

These would seem to be the same, but they give me different results. I am using genetic optimization.

Thanks
Ron

Optimizing in RP mode and Port.Sim mode usually can not render same results because the whole is greater than the sum of its parts. In Portfolio Simulation mode, real world rules of position sizing apply to the backtest. In particular, "the same fixed position size" may render different results when capital constraints come into play. For more details please see "How Trades Are Chosen" and "100% of Equity Sizing" in the Wealth-Lab User Guide.

Yes, I know about the constraints but I am using conditions where the constraints do not come into play in order to eliminate those effects. In Portfolio mode I am using Fixed Dollar position size, with Position Size the same as that used in Raw Profit mode, and I am using a huge value for starting capital so that all trades are executed (no shortage of cash). So in this case should I not be getting the same results?

Of course not. Backtest/optimization in RP mode is not a direct equivalent of a Port.Sim run. Only Port. Sim. delivers portfolio simulation optimizations, whereas RP mode simply lists each symbol's runs plus an Average result on a by symbol basis. On top of that, one set uses the highest average value whereas another uses the highest value.

Can you please define "highest average value" in this context?

Which method uses highest average value and which uses highest value?

My guess is that Portfolio mode chooses a parameter set, tests that param set on all symbols, then chooses another set and repeats many times to get the maximum metric value for the total dataset, while RP mode optimizes the parameters on each symbol independently. Is that correct?

If so then I am guessing that Port mode uses highest value optimization. I have read Chap 7 in the User Guide (many times). The text says it "calculates the arithmetic mean of each Scorecard metric for each parameter set." Does this mean the average metric across all symbols for a given parameter set? If so, how is that possible if the optimizations run separately on each symbol? I.e. the selection of params is randomized, so how do can you get the same parameter values appearing across all symbols?

Thanks,
Ron

QUOTE:
Which method uses highest average value and which uses highest value?

It's detailed in the User Guide under the Help menu > Strategy Window > Optimization > Preferred Values. Highest/Lowest "Average Per Symbol" does not make any sense in Portfolio Simulation mode because there are no individual symbols but the whole portfolio.

QUOTE:
My guess is that Portfolio mode chooses a parameter set, tests that param set on all symbols,

Portfolio Simulation tests on portfolio as a whole, if you mean that by "all symbols". It's a real life simulation with money management rules, starting capital, ending capital, equity - none of the above applies to RP mode.

QUOTE:
while RP mode optimizes the parameters on each symbol independently.

That's right. RP mode executes N runs on each symbol of the DataSet, individually, where N = number of runs required. Then provides the option to average these results on a per symbol basis.

QUOTE:
If so then I am guessing that Port mode uses highest value optimization.

Both RP and PS modes can utilize "Highest Value" method of assigning PVs. Note: we're not dealing about optimization methods here, rather methods of PV assignment. However, "Assign Preferred Values based on the highest (lowest) metric value per symbol" obviously has a different effect in Port.Sim. mode when you choose a single symbol (SSB) vs. MSB (multi-symbol backtest). In SSB Port.Sim mode, it has the "per symbol" effect whereas in MSB Port.Sim mode the assignment is based on "the highest/lowest metric value" for the entire DataSet.

QUOTE:
Does this mean the average metric across all symbols for a given parameter set?

When reading about the arithmetic mean, did you notice this in the User Guide?

QUOTE:
View the Average Result for all Symbols Applies to: Raw Profit Optimizations
This option, which appears after completing an optimization on a DataSet that uses Raw Profit sizing, calculates the arithmetic mean of each Scorecard metric for each parameter set. The result is an <Average> row

QUOTE:
I.e. the selection of params is randomized, so how do can you get the same parameter values appearing across all symbols?

You're confusing RP mode and PS mode. "All symbols" don't exist in PS mode.

Thanks for the feedback.

If the symbols do not exist in PS Mode, then what DataSeries is my strategy code operating on in that mode? Is it perhaps the sum or average of the individual symbol prices at each bar, or something similar?

Yes, I had read about and tried "View the Average Result..." before but I still am not clear how to interpret it. The text says "each Scorecard metric" is being averaged. I can understand how one can calculate the average value of a each metric across all of the tested parameter value sets for a single or multiple symbols, which would produce a single row of data (the average of each metric) but there would no corresponding parameters (since this is taking the average results from all parameter sets). I assumed instead that Wealth Lab is presenting the average metric result for a single parameter set, averaged across multiple symbols. But that seems impossible the same parameter sets are not used across multiple symbols, as I am optimizing using genetic and monte carlo. In other words, it seems impossible to calculate the average result of a particular parameter set across multiple symbols if that parameter set was only used for one of the symbols. So, when viewing in "average" mode, what is the mathematical meaning and derivation of the displayed parameter values and "average" metric?

Thanks again,
Ron

QUOTE:
If the symbols do not exist in PS Mode, then what DataSeries is my strategy code operating on in that mode?

Sorry for not being clear: I only meant to say that there are no individual symbols, just portfolio as a whole.

QUOTE:
When you check "View the Average Result for all Symbols" in RP mode, exactly what is being averaged?

If you optimized a DataSet in RP mode, that calculates the arithmetic mean of each Scorecard metric for each parameter set.

QUOTE:
However, if each symbol is tested independently and we are using genetic or Monte Carlo optimization, how could identical parameter values appear in different symbol runs?

With MC optimizer, they may not appear across different symbols. In this case you may not see the parameter value at all in the list - before or after selecting "View the Average Results..."

Thanks, beginning to see the light.

QUOTE:
there are no individual symbols, just portfolio as a whole.

Which of the following is the way PS mode optimization works?

A.
1. Choose parameter values.
2. Backtest the parameter set on each symbol to generate trades on a per-symbol basis.
3. Repeat, attempting to optimize the metric for the entire portfolio.

B.
1. Choose parameter values.
2. Backtest the param set on the entire dataset by considering the dataset to be one symbol with the per-bar price being the average (or sum?) of all symbols in the dataset.
3. Repeat, attempting to optimize the metric for the "single symbol" portfolio.

If it is case B, how (mathematically) does Wealth Lab treat the group of symbols as one, i.e. does it just average the bar prices?

QUOTE:
With MC optimizer, they may not appear across different symbols. In this case you may not see the parameter value at all in the list - before or after selecting "View the Average Results..."

I ran a short MC optimization test on a 3 symbol dataset; RP mode. Every "average result" data row had a parameter set corresponding to a single-symbol row in non-average view (i.e. each average-view parameter set was used once, for one symbol). If a parameter set was only executed on 1 symbol, how does Wealth Lab calculate an average metric for that parameter set across all of the symbols?

QUOTE:
Which of the following is the way PS mode optimization works?

In PS mode, there is no averaging and no "each symbol".

Even for single-symbol runs in PS mode, optimization is a series of true portfolio backtests. (For more details on what a portfolio backtest is, see "Inside a Portfolio Simulation" in the User Guide.) Parameter choice is driven by selected Optimizer's logic.

In RP mode, when "View the Average Results for all Symbols" is checked, how does Wealth-Lab calculate the average metric (across multiple symbols) for a parameter set if that parameter set was only used for one of the symbols?

If a symbol wasn't included in a parameter set (MC/GA assumed), obviously there's nothing to average. As you could have read in the User Guide:

<<Same as the previous action, but the metric is averaged among all contributing symbols before finding the highest (or lowest) value.>>

Have a nice weekend, and don't forget to take a short break from optimizations! ;)

All of the symbols in my dataset are included in all of my optimizations. The issue is not symbols being left out of an optimization but rather that parameter sets are not common between symbols.

In RP mode optimizations, Wealth Lab creates different parameter sets during the course of optimizing on different symbols. If we have 2 symbols in a dataset then Wealth Lab generates 2 different streams of parameter sets as it individually optimizes on each symbol. If you then click "View the Average Results for all Symbols" Wealth Lab displays metrics averaged between the 2 symbols. But how is it possible to calculate an average metric for a parameter set across the 2 symbols if the parameter set was not used for both symbols?

If the same particular parameter set (1 row in the "non-average" optimization results) was coincidentally used for both symbols A and B, then certainly the average metric for the symbols can be calculated for that particular parameter set. But, by comparing the row-by-row optimization results (comparing Average view to non-average view) I observe that in most cases the parameter set values appearing in the "Average" view were used for only one of the symbols in the dataset. In such cases (which seem to be most cases) where a parameter set was not used on all of the symbols in the dataset, how is the average metric for that results row calculated, if there was never a metric calculated for all symbols in the dataset with that set of parameter values?

Thanks,
Ron

Ron,
If a parameter set was not used in a GA/MC optimization for a symbol, please agree with me that there's nothing to include for that parameter combination/symbol, and consequently, nothing to average. If a parameter set exists in another symbol (only), I believe that it is taken without averaging. In other words, if contributing symbols == 1, averaging is equal to dividing the performance metric figure by 1.

Hi Eugene,

Yes, I agree with you that if a parameter set was not included in all of the symbol backtests there is nothing to average.

I ran a short test. 2 symbols, RP mode, GA optimization on 2 parameters. The raw results produce 209 sets of parameters for the 2 symbols. Viewing Average Results produced 101 parameter sets. There were 5 cases where the same parameter set was used for both symbols. In other words, of the 101 Average parameter sets, 5 had values that were used for both symbols. In these cases the Metric value shown was the average of the metric values for the 2 symbols, as expected. In the other 96 cases the Average Metric value was half of the Metric value shown for the one symbol which used that param set. It seems that when calculating the average metric in Average view, Wealth Lab is considering the metric value to be zero for a symbol that did not use that parameter set, and is dividing by the total number of symbols rather than the number of contributing symbols.

In any case, I am still confused about the fundamental intent of Average View and RP optimization. Regardless of how WLab handles the cases where not all symbols contribute to the average metric, it seems that we can only base parameter selection decisions cases where all symbols do contribute. But that seems rare. In my test it was only 5% of the Average view rows. But the valid rows do not seem to be denoted in the output. Am I missing something?

My objective is to come up with a parameter set that works well across multiple symbols. I have been trying to figure out whether this should be done in PS or RP mode. I previously was using RP mode, viewing Average Results, and looking for the best metric. But now it seems that RP mode is suspect.

That leaves PS mode, but your previous posts in this thread emphasized that PS mode does not operate on individual symbols, but rather the entire portfolio, and I still am not clear what that really means. I understand about position sizing etc. and the other functions of PS mode but I am not concerned with or using those features yet. I am focused only on the core strategy and parameters. Could you please explain, in PS mode, if individual symbols are not used and if the prices of the symbols are not somehow combined into a single "portfolio data series" then what dataseries does my strategy operate on in PS mode?

Thanks,
Ron

Hi Ron,

QUOTE:
but your previous posts in this thread emphasized that PS mode does not operate on individual symbols, but rather the entire portfolio

That's not exactly true. You can run a backtest in PS mode on a single symbol (SSB), can't you? For sure PS mode operates on individual symbols.

There is no averaging in PS mode optimizations though like there is in RP mode. Why is that? Because Portfolio Simulation creates true portfolio simulation optimizations just like backtests in PS mode.

QUOTE:
Could you please explain, in PS mode, if individual symbols are not used and if the prices of the symbols are not somehow combined into a single "portfolio data series" then what dataseries does my strategy operate on in PS mode?

A SSB optimization run in PS mode is fully equivalent to a SSB backtest in PS mode. Of course, no modification to any DataSeries. The difference is in presence of the attributes of a portfolio: starting capital, ending capital, equity (not just net profit), margin, capital constraints (inability to purchase more than your buying power allows).

I am using multisymbol optimizations, not single symbol. In PS mode I a disable "portfolio effects" using very large initial capital (never run out of cash) and Fixed Dollar sizing. Is this equivalent to RP mode? If not, how is it different? This type of PS mode optimization runs faster than the equivalent RP mode optimization, so if they are equivalent then PS mode is better because it's so much faster, but the difference in execution time makes me thing they are quite different...?

A single-symbol (SSB) optimization in PS mode, with real-world position sizing rules "suppressed" by using Fixed Share sizing and never running out of cash, should deliver same results.

As already explained before, MSB optimizations are absolutely not comparable in PS / RP mode.

Please tell me if this is correct:
Assuming a multi-symbol PS mode optimization with Fixed Dollar position sizing and never running out of cash, WealthLab does this:
1. Choose a set of parameter values.
2. Individually backtest those param values on each symbol in the portfolio to get the optimization metric, e.g. Net Profit for each symbol.
3. Sum the per-symbol optimization metrics to get the total portfolio metric for this set of param values.
4. Based on the optimization algorithm (genetic or Monte Carlo) choose the next set of param values, and repeat.

When done, this results in a set of parameter values that provides the best performance for the selected metric, across the entire portfolio.

If this is correct, I think I may finally be at peace with the Universe.

If it's not correct, can you explain where I have it wrong?

Thanks,
Ron

Here we go again. Despite my suggestions to read the User Guide, you seem to be missing the idea of Portfolio Simulation mode.

Please look up "Portfolio Simulation Mode" in the Help menu > User Guide:

1) Reference: Glossary > "P" > Portfolio Simulation Mode
2) Reference: Data Panel > Position Size Control > Portfolio Simulation Mode

I will be at peace with the universe when after carefully studying the Guide, you will understand that #2-3 are wrong:

2) There is no "individual backtest on each symbol" in MSB (multi-symbol) mode as portfolio is treated as an entity.
3) There is no summing, dividing, multiplying, subtracting, averaging, or anything else in PS mode. Neither for SSB nor for MSB simulations.

Hope that helps (finally!)

Yes, it helps, thanks. Yes, I have read the documentation but from my understanding it seems to contradict a key concept above. User Guide > Section 7.6.4 ("Inside a Portfolio Simulation") > Main backtest processing functions > Executing System:

QUOTE:
Wealth-Lab executes the Strategy on each symbol individually in serial fashion and in a Raw Profit mode.

I assume that the User Guide is wrong and that your posts above are correct, i.e. there actually is no backtesting on each symbol as the User Guide says, but rather backtesting with the portfolio treated as an entity as you wrote.

So, the crux of the matter is: exactly what is this entity and where do its values come from? It must be a dataseries, correct? Then, if the per-bar price values of this dataseries are not a function (e.g. sum) of the individual symbols, then how does Wealth Lab calculate the values?

Thanks for your patience in explaining this to me.

Ron

QUOTE:
Wealth-Lab executes the Strategy on each symbol individually in serial fashion and in a Raw Profit mode.

In plain English, that's true - but under the hood. First goes a RP mode backtest on each symbol, then a position sizing overlay is applied. The trick is that this all happens before Wealth-Lab executes an optimization run. The UG is not wrong, "Inside a portfolio simulation" just suggests what goes "inside the mind" of WLP, the underlying mechanics of a real-life portfolio simulation that you can't "see".

To finally sum things up, in a PS mode MSB optimization, there is no backtesting on each symbol (and optional averaging etc). Backtesting on each symbol, and then applying portfolio-based position sizing, throwing out trades that don't meet capital constraints etc. -- is what makes a PS mode backtest internally before it's "presented" to you/optimizer.

Good luck, and I hope we're through with it finally. Good luck.