"Curve fitting" is a major risk. If you were going to select a few metrics for ranking parameter values which one(s) would be on your short list?
For me, this would apply to strategies derived from a symbol rotation code. When backesting, I do not want "luck" to have more to do with apr% than strategic concept put into action. On the other hand, some volatility is to be expected...8-15%. What concerns me more are other events that can make apr%, artificially, look "too good to be true" : a few number of good trades; outlier gains; and a surge in gains that is limited to a short span of time when compared to the larger period of back testing. I would want an index to put a premium on a high win percentage that does not have too low of a number of trades, an apr% that does not skewed by a few outliers (gains or losses), and where drawdowns are reliably offset with recovery. It is fairly easy to rule out parameter values by simply running them in the strategy and then looking at performance visualization. However, I would much prefer to run an optimization and then use a metric to do this screening for me. Currently, I might have parameter values that show very good apr%....but I would rather have parameter values that relatively look weaker but have better metrics as described above. Those pv's can get lost when optimizing and having the right metric might reveal them in a more methodical fashion. I do not expect to have a fool proof method...just something that gives me a little more confidence. Anyone have suggestions from the large array of metrics to choose from on the Community ScoreCard?