Measuring performance
Overfitting
Also called Curve fitting or Data mining
Tuning a strategy so precisely to past data that it captures noise instead of a real effect.
How it works
The more parameters a rule has and the more variants are tried, the more certain it is that the best-looking result is luck. A rule with six thresholds tuned on one decade describes that decade, not the world.
The defence is out-of-sample testing, keeping the number of parameters small, and having a reason the effect should exist before finding it in the data.
Where it misleads people
A backtest that looks too good almost always is. Sharpe ratios above 2 on a simple rule using public data would not survive contact with a market full of professionals.
Related terms
- BacktestRunning a strategy over historical data to see what it would have done.
- Out-of-sample testingChecking a strategy on data that was not used to design it.
- Look-ahead biasUsing information in a backtest that was not actually available at the time.
- Sharpe ratioReturn above the risk-free rate, divided by volatility — how much return each unit of risk bought.
Mentioned in
More in Measuring performance
- Performance attribution
- Price return
- Nominal return
- Real return
- Money-weighted return
- Realised profit and loss
- Mark to market
- Risk-free rate
All 29 Measuring performance terms · The full glossary, 433 terms
Educational reference only. Nothing here is investment advice or a recommendation to buy or sell anything, and tax treatment depends on your own circumstances and the country you are in.