Measuring performance
Backtest
Running a strategy over historical data to see what it would have done.
How it works
A backtest is a hypothesis test, and a weak one. It shows whether a rule would have worked on one sample of history — the only sample everyone has already examined thousands of times.
The credible ones state their assumptions: what costs were charged, what fill prices were assumed, whether delisted companies were included, and how many variants were tried before this one.
Where it misleads people
The number of untested variants is the missing figure in nearly every published backtest. Try enough rules on one dataset and something will look excellent by chance alone.
Related terms
- OverfittingTuning a strategy so precisely to past data that it captures noise instead of a real effect.
- Survivorship biasThe distortion caused by measuring only the things that made it, and ignoring the ones that failed.
- Look-ahead biasUsing information in a backtest that was not actually available at the time.
- Out-of-sample testingChecking a strategy on data that was not used to design it.
Mentioned in
More in Measuring performance
- Behaviour gap
- Annualised
- Benchmark
- Yield on cost
- CAGR
- Unrealised profit and loss
- Calmar ratio
- Tracking error
All 29 Measuring performance terms · The full glossary, 433 terms
Educational reference only. Nothing here is investment advice or a recommendation to buy or sell anything, and tax treatment depends on your own circumstances and the country you are in.