Why A-Share Backtests Diverge from Live Trading and How to Test Robustness
Summary
The article examines why A-share strategy results in historical tests can differ from live or simulated trading. It groups causes into data problems such as survivorship bias and price adjustments, model problems such as overfitting and crowded factors, market changes including regime shifts, and execution frictions such as trading limits, slippage, and delays. It recommends evaluating robustness with rolling out-of-sample tests, Monte Carlo paths, and parameter sensitivity analysis. A broad parameter plateau is presented as preferable to a narrow optimum, with deployment near the center of a stable region.
The article also reports BigQuant experiments comparing stock selections across overlapping prediction and validation periods. Selection agreement varied by model, factor, and number of stocks selected, with some configurations showing full agreement and others much less. These examples are platform-specific and do not establish that agreement guarantees live performance. The author notes that the experiments are incomplete and that much of the supporting industry evidence was gathered from summaries and secondary sources. Practical advice includes aligning live start dates with rebalance dates, accounting for costs and market rules, controlling model randomness, and reviewing market regimes.
Key ideas
- Backtest performance is vulnerable to data biases, overfitting, crowded signals, regime changes, and execution frictions.
- Monte Carlo testing and rolling out-of-sample validation can probe strategy robustness beyond a single historical path.
- A broad region of stable parameter results is treated as stronger evidence of robustness than one isolated optimum.
- Comparing selections on overlapping prediction and validation dates can expose sensitivity to strategy settings.
- The reported platform experiments are limited, and agreement between backtests does not guarantee live profitability.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.