Best practices benchmarking.

Most of us are just frustratingly tweaking code in the blind as to what might be reasonably accomplishable target goals.

It would nice to have some feedback as to what has been accomplished or seen by more seasoned WL practitioners. This would provide some hopeful incentivizing to spur us on with attempting continuous improvements in our studies, thinking, and practices.

Note, sharing your experiences may also result in some fresh and lively discussions.

So:

What are some of the highest APRs you have seen while running, say a two year back test of a purportedly "non-peeking," humanly tradeable WL strategy, on a broad index like the NAS 100, SP100, SP500 or other large or combo watch list ?

Any broad or particular comments on the strategies involved would be helpful and appreciated (e.g., long only, long and short, levered etfs, 800 to 1100 trades a year, pairs, uses multiple indicators, watchlist of 700, etc).

Please also let us know the name by which you reference such strategy.

I will just leave this pretty curve-fit one here --



Long+short, a handful of well known ETFs which either have a short counterpart or can be borrowed at Fidelity. Need to execute at close, which both pain in the ass to code in terms of indicator development (PartialValue, I'm looking at you and whomever invented you) as well as to execute (I'd expect real APR to be 5 to 10% lower because of execution issues/slippage).

365 trades over the course of last 2 years of backtest. First 252 bars are dead zone (indicator ramp-up).

I haven't traded it live and I don't know if I ever will. It is more likely than not that the whole contraption will fall apart out of sample. But since you've asked about what's feasible in backtest... :)

You can refer to the Market Order rankings on the WL4.com home page. These are scripts that were written years ago, but have provided outsized returns even over the last 5 or 6 years (1400 bars). The tradeoff for high APR is almost certainly high (or higher) drawdown, and even though Market Order scripts are easy to trade, living with DDs in excess of 20% are tough pills to swallow.

Note: A single $imulation is just that. Use Monte Carlo-Lab to get a better idea of what the most likely outcome of a Strategy may be as well as what is estimated to be possible (positive and negative extremes).

Re: WL4 - but be careful not to fall into the trap of comparing scripts that take single positions with multi-position scripts.

Avishn,
That strategy is great, fabulous. I love the smooth ascension, and relatively minimal drawdowns, and awesome gains.
Please also let us know the current name by which you would reference this strategy of yours.
Thanks for sharing this beautiful backtest.

Cone, Eugene,

Since WL4.X is being shutdown, and WLT translations are not quite so susceptible to mix and match for dropping into WL5 strategizing, are there / will there be / could there be / WL5 versions of the various current and or former top 25 or other top ranked WL4 scripts?

If not currently available, please consider doing so and maybe putting them into the various strategy folder types, or into a common WL5 folder of WL4 Hotties.

This way people will be able to continue to better appreciate and maybe make better use of some parts or all of these classic accomplishments.

Thanks again for all your wonderful ongoing work.

Re: WL4 -- I'm willing to spend my time to translate from WL4 to WL5 any equity trading strategy with Sharpe > 0.8, > 50 trades a year over last 3 years, <= 2 optimization parameters. Are there any candidates in WL4 Top 25?

You can run a screen for those parameters in the old ChartScript Center Performance Search.

What are the "Rules" for Avishn's above strategy?

The particulars of that strategy are irrelevant. The point was rather that even the most good-looking strategies (from the equity curve perspective) most often perform poorly out of sample. Monte-Carlo simulation doesn't provide much insight either. As somebody said -- I will take 1 month of live trading over 10 years of backtest any day.

Again, that was just a semi-serious reply to the question of what APR is feasible in backtest.

A note on the strategies from WL4 Top 25. There are only 9 strategies which satisfy my initial criteria of Sharpe ratio and the number of trades. I'm also filtering out anything below 15% APR. It is interesting to see that several of those follow the same "buying all the way down with limit orders" logic (multiple small orders for the same bar with decreasing price p1 = p0 * 0.999 or something along these lines).

Hi Avishn,
So how large is the watchlist on which you ran your ETF Curve Fitter?
How does it do between say 6/1/05 and 6/1/08 ?
Why haven't you tried trading with it?
As to the strategy that you are currently trading, what kind of two year APR backtest is it recntly achieving?
Thanks for sharing.

As to top ranked limit buyers, I really don't understand how one can humanly trade them as they are written or optimized. Maybe you might want to filter out those with a high number of alerts. Perhaps with addition of various modifiers they might present a more manageable number of fewer alerts, but doesn't that defeat their design purpose?

Cheers.

I've found about a dozen of ETFs which worked well for that strategy. Five of them included in the backtest above.

I've made an attempt to make strategy's parameters adaptive using some non-parametric statistics, but the results were terrible. In absence of walk-forward optimizer in WL, I have to assume that the strategy is not very robust.

On the subject of limit entry dip buyers. Trading those is actually pretty simple if you send all your alerts (even 100x of them) to the Quote Tool in WL. The real problem with any "catch a falling knife"-type approach is that the drawdowns are vomit-inducing. All of those strategies are guaranteed to be caught in _every_ market meltdown. I think somebody on this forum tried to hedge with short position in SPY, but I was not successful with that. It seems that it would be pretty difficult to design a good stop loss logic for these strategies too, after all they all built on the premise of buying _more_ when price moves down, so the only option left is to take very small positions which decreases efficiency and increases your trading cost exponentially.

Before I forgot. Just checked the results for the 6/1/05-6/1/08 range. APR 14%, Sharpe 1.54. Meh.

EDIT: Typo (date range)

Re: WFO. Frankly, the Optimizer API doesn't make it easy to implement walk-forward optimization.

Avishn,
I should have known better than to ask for your ETF Curve Fitter's APRs. The WL reported APR for it is completely inaccurate due to your long ramp up time to populate your indicators. For example, your equity curve says APR 74%, but that is over 3 years, and if focus on two years of trading, it has to be well over 200%. So whatever time frame you run it over, you probably need to exclude the first year from APR calculation.
Interesting about using WL quote manaager for limit down strategies. And thanks for the caveat about knife play.
Cheers.

This is a very good question, and it leads to a criticism of the WL5 performance table.

First what is a good "APR" ? By that term I think you mean Annualized Geometric Return, see the Bible and the hedge industry standard work "Practical Portfolio Performance Measurement and Attribution", 2nd ed, 2008, Carl Bacon. Much of WL5's performance terms are confusing and they use non-standard nomenclature so that we are left in the dark. The term APR is meaningless and I think it is the GeoReturn or "Annualised Return" ( The standard term, note Bacon is English.)

Anyway I've have strategies that are actually working and trading numerous times per week that yield 7-9% Annualised Return. However this not the statistic to focus on because it ignores risk. Much, better is a risk-adjusted ratio . WL5 gives as several: The WLscore
which is ok ( but hardly a industry standard), you should get better than 5 and for a really tradeable scheme, the score should be > 20. The Sharpe ratio should be > 1.

WL5 offers a statistic quaintly termed "Recovery Factor." I think, but I am not sure that this is the Calmar ratio, which is crude measure = Net Profit/Max Draw-Down; it should be > 1.
Much, much better ratio's are the Sortino which is not offered by WL5 (but should be) Sortino = Ann.MeanBar Return/Downside Std. Dev. Sortino should be > 2.2. The Sortino ratio is a very good ratio to ranking competing trading schemes, but WL5 does not offer anyway to compute it.
crude way is to export the Trade table to Excel and compute the down-side Stdev and mean bar returns from just the trade stats.

PLEASE, PLEASE, Wealth-Lab give us a way to export the equity curve data to Excel so that we can compute these superior stats, AND give us a direct computation of Sortino, M^2 return ( see below, and the Bacon Bible.)

Finally I should note that no ratio, Sharpe or Sortino or adjusted Sortino or adjusted Sharpe tells us how much better a superior trading scheme is. For that you need the M^2 adjusted return, and Wealth-Lab should provide that too. M^2 is a function of a designated benchmark, e.g. the SPX or Nasdaq. M^2 could be computed if WL allowed us to export the equity curve to Excel.

Paul3nt

QUOTE:
PLEASE, PLEASE, Wealth-Lab give us a way to export the equity curve data to Excel so that we can compute these superior stats, AND give us a direct computation of Sortino, M^2 return ( see below, and the Bacon Bible.)

This is old school. ;)

Just take a look what can be done with Wealth-Lab performance visualizers:

MS123 Visualizers

You can program your own too:

Wealth-Lab Version 6 (.NET) Development Guide

Requests for visualizers can be left in this thread and if feasible, implemented by MS123:

MS123 Performance Visualizer library