Diagnosing Backtest and Live Trading Differences
Summary
The document recommends plotting live and backtested equity, positions, strategy signals, and order prices together as an initial way to locate discrepancies. If the strategy logic is implemented consistently, it identifies latency and queue modeling as important sources of differences. Feed and order latency should be measured in the trader’s own environment or checked against the characteristics of an external data source. The article also suggests testing how assumed latency changes results to estimate the value of infrastructure improvements.
A queue model should reflect the likelihood of fills in live conditions, whether developed in-house or adapted from existing models. The discussion emphasizes a key limitation: the backtest assumes no market impact. Marketable orders and large limit orders can create impact, partial fills, and execution outcomes that are difficult to simulate. It recommends starting with small trade sizes, aligning live and simulated results, and increasing size gradually while monitoring both. These are diagnostic guidelines; the text gives no measured latency, model comparison, or empirical performance results.
Key ideas
- Plot live and simulated equity, positions, signals, and order prices to identify where results diverge.
- Measure feed and order latency in the relevant trading environment.
- Use a queue model that represents live fill behavior and calibrate it against observed outcomes.
- No-impact assumptions can misrepresent marketable orders and large limit orders.
- Start small and increase trade size gradually while comparing live and backtested results.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.