Why Backtests Fail to Match Live Trading
Summary
This essay outlines common reasons a promising backtest may fail in live trading. It recommends checking calculations when returns look implausibly strong, including comparing the account’s initial and current states to derive profit and loss. It also warns about look-ahead bias and survivorship bias, especially when historical equity universes omit delisted companies or fail to reflect changing index membership.
The discussion emphasizes that live results face frictions absent from simplified simulations, such as slippage, market impact, network delays, and data or connectivity errors. It advises examining trade counts and investigating the strategy’s strongest winning and losing periods and individual trades to understand the sources of returns and vulnerabilities. Backtesting is framed as a way to expose defects and weaknesses, not to prove a strategy is excellent. The essay offers qualitative guidance rather than empirical comparisons or a formal validation procedure, and notes that strategies can depend on market regimes: trend methods may struggle in ranges, while range methods may struggle in trends.
Key ideas
- Unusually strong backtest returns can signal calculation or implementation errors.
- Look-ahead and survivorship biases can make historical results misleading.
- Live trading adds slippage, market impact, latency, and data or connectivity risks.
- Trade counts and the largest wins and losses help reveal how a strategy behaves.
- Backtests are useful for finding weaknesses, while strategy performance may vary across market regimes.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.