Hi LenMoz,

I read recently the following article (https://pdfs.semanticscholar.org/4183/f7485fc9d1a619a90a62957e5a5ff4662376.pdf) about the advantage to use the multi-objectives in Particle Swarm Optimization. The idea is to find the best combination between several fitness functions.

With this possibility, you can find for example the best net profit linked to the smallest drawdown.

It would be a very nice add-on to your extension if you consider it, isn't it ?

Hi Abegy,

Not a surprise request, I've actually considered that - to optimize an expression. It would be a big effort and somewhat unwieldy because the WL Results tab couldn't sort on the expression.

Perhaps a more useful design would be to implement a dynamic Performance Visualizer and Scorecard. That would be usable by all optimizers. That would integrate more smoothly with the Optimization host. (or am I just trying to get out of the work! lol)

Len

I agree with you that the best is what you say.But as there is no dynamic Performance Visualizer and Scorecard at this time, why not to offer a basic 2 fitness goals ;-) ?

QUOTE:
... a more useful design would be to implement a dynamic Performance Visualizer and Scorecard.

Are we talking about implementing a performance optimizer that can maximize some Scorecard metrics while minimizing others? And you're saying such an optimizer can be integrated into the Wealth-Lab framework as a Performance Visualizer?

If so, yes I would be interested in that. That's quite the madman scheme.

I have thought about composing a Scorecard metric that's a composite (linear combination) of several other metrics. But the coefficients of the linear combination would be static and conditional (e.g. include %win only if there are >5 trades). The problem there is choosing the best static weights for the linear combination.

Hi LenMoz,

I have investigated more about the MULTI-OBJECTIVES fitness feature. If I have understood correctly what I read in this website site (http://www.hvass-labs.org/projects/swarmops/cs/), the fitness feature works like a sum of individual objective that you can add a weight for each of them.

On this case, why not to add the possibility to your optimizer to select several metrics on the scorecard and offers the possibility for each to add a weight ? The sum must be equal to 100% and this is the responsibility to the user to be sure that metrics go in the same direction.

Wha do you think about that ? I don't see any incompatibility with the logic of the WLD optimization API. Right ?

Regarding graph, it is another story and not a requirement for me. But maybe, there is a simple solution : you can just display values in a table that we can copy/paste into excel.

Thanks in any case for your work around your very useful extension.

I have given your request some thought and can't offer good news.

It's very unlikely that I will be publishing any enhancements to Particle Swarm Optimizer any time in 2017. It functions adequately for many needs. On a personal note, this year I am getting married, downsizing, and selling my home. This leaves me little time to develop, document, test, and publish the requested enhancement. It would lead to a support requirement as well.

The multi-objective enhancement could take several forms - most significant being the weighting/balancing of the objectives, especially where the objectives differ in mean and standard deviation. It would be difficult to anticipate the needs of all users. I repeat that an alternate solution is to implement multi-objective as a Scorecard. The Scorecard would return the balanced multi-objective as a single metric which could be optimized by not only PSO, but any available optimizer. This fits WL's design and still seems to me to be the best solution.

The graph data could be parsed from the optimizer log.

Sorry to not offer more help.

Len

QUOTE:
On a personal note, this year I am getting married,

Congrats!

Vince

Thank you LenMoz. I understand now your point about the use of a private scorecard to bypass the limit.
I take this opportunity to wish you all the best for your mariage.

In finance, multi-objective fitness is really important. You can use a Pareto front to determine which are the best remaining candidates.
And typically, you won't stop at net profit and least drawdown, but these are primary factors.

3D or even 4D finesses aren't unheard of. You'll likely include some %activity, %deployment, or %exposure metric to help decide how reliable a strategy is.

QUOTE:
In finance, multi-objective fitness is really important. ... you won't stop at net profit and least drawdown, but these are primary factors.

So how are the multidimensional objectives (independent metrics) typically combined into a single merit term for this composite optimization? In numerical analysis, we typically combine multiple end-test metrics using the 1-, 2-, or infinity-norm. But metrics like net-profit, profit-per-bar, and drawdown have such different units and ranges that I question simply taking a norm is the best method.

I have thought about scaling each metric so their standard deviations are the same, then combining them with a norm. But just estimating their standard deviations is a problem because outliers will throw these estimates way off.

I also thought about using robust techniques such as performing a rank sort on each metric, then taking the 2-norm of all their rankings to get a composite merit term. Has any research been done with this approach?

I realize this is the wrong forum for a discussion about robust statistical methods and numerical methods. Can someone direct me to a research-oriented technical forum that understands numerical methods against stochastic variables? This is a tricky numerical problem.

-----

I agree, simply using net-profit or profit-per-bar for optimization alone is not ideal. But it does return "fair" results. However, in a choppy market, these results can be problematic with a great deal of trade turnover. I'm just wondering if further optimizing the strategy itself by including more external variables (oil prices, gold prices, index metrics) would provide more model stability than combining performance metrics?

Superticker,

See my post here : https://www.wealth-lab.com/Forum/Posts/Add-tanh-estimators-indicator-for-normalized-series-38889
There is an article that I recommend you to read about your subject.

Thanks for the article on tanh estimators. I'm not intuitively sure why hyperbolic tangent would be a good transform for normalizing stochastic variables. Perhaps if I converted tanh to its exponential form, it would be more apparent.

This approach is sharply different from my rank-sort suggestion.

Traditional multivariate statistics multiplies the model (system) matrix with its covariant matrix. The articles you cite refers to this process as employing z-score normalization. But this isn't a good option for stochastic data (e.g. stock data) because you can't reliably estimate the variances (or their standard deviations).

---

What I'm doing today, which works okay, is optimizing with the Particle Swarm optimizer against Net Profit. I have it using the Clerc Basic (OEP 0) algorithm with 13 particles. The Clerc Tribes algorithm works well too. After optimizing against Net Profit, via the Results tab, I assign PV parameters that maximize Profit per Bar. This does a good job for 90% of the stocks, especially the large cap stocks. For small cap stocks, this single-parameter approach (Profit per Bar) isn't as good.

Thanks again for the article.

There are a few schemes for keeping candidates in multi objective fitness. One of them being the use of Pareto front.



http://cms.horus.be/files/99936/MediaArchive/pictures/Pareto_Dominated.png

Next time please consider attaching images from external sources or image hosters directly to discussion thread. I've corrected this in your post. Thank you in advance.

Thanks for the graphic explaining Pareto frontier optimization. Yes, this frontier carves out a hyperbola, which explains why the hyperbolic tangent (tanh) is employed in tanh estimators to solve this problem. Nice graphic to answer my question. Points along the knee of the tanh line (frontier) would be most interesting for optimization, which tanh function locates.

This Pareto frontier methodology is not mutually exclusive from using a rank-sort transform of the data. In other words, you can rank your results, and then analyze the rank(objective 1) with the rank(objective 2) using the Pareto frontier. That would be very powerful.

Note: (obj 1)*(obj 2) also returns a hyperbola. This is a "minor reason" for including x*y interaction terms in empirical linear models.

I've seen this type optimization done before in finance before with efficient frontier analysis.
https://www.mathworks.com/help/finance/plotting-the-efficient-frontier-for-a-portfolio-object.html
https://www.mathworks.com/help/finance/-estimate-efficient-portfolios-and-frontiers.html

---

I'm still wondering if bringing in more external variables (factors) into the strategy itself--thereby making the strategy model smarter with more "meaningful" degrees of freedom--would be preferable over all this optimization? I haven't done that yet. There's only so much that can be gained from price and volume data alone.

For your information, I see in Ninja Trader 8 this type of optimization : http://ninjatrader.com/support/helpGuides/nt8/en-us/ --> See video Understanding Multi-Objective Optimization in Video Library

The major advantage of the multi-objective is to reduce overfitting probability during backtesting . Indeed, as there are several objectives to optimize, the algorithm cannot focus on the dominated solution.

This type of logic can be apply with any optimizer algorithms. But linked to Particle swarm, this gives a real advantage because the problem of this algorithm is that most of the time, it find the best solution (means probably that you will have an overfitting backtesting process).

There are some publics library/code on the web to help to develop this type of approach :
- Microsoft publication : http://visualstudiomagazine.com/Articles/2015/01/01/Multi-Swarm-Optimization.aspx?Page=1
- Nuget cs-moea package : http://www.nuget.org/packages/cs-moea/#

I’m not an expert to be able to develop this type of optimizer for the community. Sorry, but maybe Eugene ;-) ?

Please count me out.