Common Backtesting Errors That Distort Trading Strategy Results
Summary
This article uses a series of trading examples to show how implementation choices can make backtests look stronger or weaker than a live strategy. It highlights omitted transaction costs and slippage, incorrect treatment of reinvestment and whole-unit sizing, overfitting, look-ahead bias, survivorship bias, use of the wrong price for a decision, and concept drift. Examples include intraday index signals, a short-only index strategy, a machine learning forecast, and a moving-average rule for a stock.
The reported examples show performance changing substantially after costs are included, accuracy falling from a perfect result to near chance after cross-validation, and returns declining when a signal is shifted to use information actually available at the time. The article recommends realistic fills and costs, careful return accounting, out-of-sample validation, and periodic review as markets change. These examples are instructional rather than a systematic study: some assumptions and calculations are only briefly described, and the article does not establish that any corrected strategy is profitable. It also flags an intentionally hidden error without resolving it in the text presented.
Key ideas
- Transaction costs and slippage can materially reduce results, especially for frequent trading.
- Backtests should account for compounding and whether instruments can be traded in fractional units.
- Signals must use only information available when the trade could have been placed.
- Cross-validation can reveal severe overfitting that a single evaluation split may conceal.
- Survivorship bias and concept drift can undermine conclusions drawn from historical data.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.