Building a Backtester and Avoiding Misleading Results
Summary
This article discusses whether to build a trading backtester, outlines common sources of bias, and contrasts simple for-loop simulations with event-driven systems. A basic loop applies calculations and trading rules to successive bars; it is straightforward and fast for screening ideas, but can omit realistic fills, costs, and separation between research and live code. An event-driven design processes market, signal, order, and fill events through system components, aiming to resemble live trading more closely.
The guidance covers in-sample overfitting, survivorship and look-ahead bias, regime changes, transaction costs, OHLC data limitations, capacity, benchmarks, robustness to start dates, and the psychological difficulty of enduring drawdowns. It recommends using simple tests as filters, scrutinizing assumptions, and building infrastructure knowledge. These are conceptual recommendations, not a measured comparison or a complete implementation guide. The provided excerpt is truncated, so portions of its event-driven discussion are unavailable.
Key ideas
- Backtests model historical performance and cannot establish how a strategy will perform live.
- Simple bar loops suit quick screening but are vulnerable to unrealistic fills and indexing errors.
- Event-driven systems process market, signal, order, and fill events in a workflow closer to live trading.
- Realistic tests should consider biases, trading costs, capacity, benchmark selection, and robustness.
- A backtester can be useful for learning, while impressive simulated results still require skepticism.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.