Backtesting Bias: Recognizing Look-Ahead, Overfitting, and Selection Errors
Summary
This article outlines common ways systematic trading experiments can mislead. It names look-ahead bias, where a test uses information unavailable at the time of a trade; overfitting, where rules or parameters are tuned to historical noise; and data-mining or selection bias, where repeated searches make a chance result look meaningful. It distinguishes measuring historical performance from estimating how a strategy may perform in the future.
The practical guidance is to use tests to check whether an idea could work, identify influential parameters, choose reasonable parameter ranges, and debug trade logic. Prefer simple rules that can be explained, avoid excessive precision in noisy markets, and judge strategies by robustness rather than their best in-sample result. The article also cautions against strategies whose expected edge barely covers retail trading costs, such as some scalping approaches. The supplied text is an outline rather than a full treatment: it gives no worked examples, diagnostics, or evidence quantifying the effects of each bias, so these recommendations need further methodological detail to implement.
Key ideas
- Backtests can describe historical results but cannot guarantee future performance.
- Look-ahead bias occurs when a simulation uses information unavailable at the decision time.
- Repeated parameter tuning can fit noise instead of a persistent market effect.
- Simple, explainable rules and robust parameter ranges reduce opportunities for overfitting.
- Trading costs can overwhelm strategies with only marginal gross returns.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.