Five Backtesting Biases and Ways to Reduce Them
Summary
The article explains five ways that backtests can mislead: overfitting, unrealistic transaction costs and fills, look-ahead bias, survivorship bias, and time-period bias. It recommends limiting parameter searches, preferring stable ranges over isolated performance peaks, and checking whether a strategy’s market logic remains plausible. It also calls for realistic assumptions about fees, spreads, slippage, partial fills, and delays, particularly for large orders and less liquid derivatives.
For data integrity, the article advises preventing access to information unavailable at the time, retaining delisted, failed, or acquired assets in the sample, and using periods that include varied market conditions. It notes that very old crypto data may not reflect current trading volume or market structure, so broad coverage still requires judgment about relevance. The guidance is practical but general: it provides no worked examples, quantitative tests, or evidence comparing specific mitigation techniques. Its suggestion to compare backtest results with live trading can reveal discrepancies, but does not itself establish that a strategy will continue to work.
Key ideas
- Repeatedly tuning many parameters raises the risk that a strategy fits historical noise rather than persistent market behavior.
- Stable performance across a parameter range is more credible than a sharp peak at one setting.
- Backtests should account for fees, spreads, slippage, partial fills, and execution delays.
- Historical tests must exclude future information and include assets that later failed or left the market.
- Samples should cover varied regimes while remaining relevant to the market’s current structure.
- Comparing live results with backtests can expose mismatches but cannot guarantee future performance.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.