Rolling Training Can Make Paper Trading Differ From Backtests

Summary

The post describes a discrepancy observed after adding a rolling-training module to a strategy on a Chinese quantitative trading platform. Without rolling training, the author says simulated live trades and later tracking backtests matched. With rolling training enabled, the trades and returns diverged. A platform representative reportedly attributed the difference to the simulation and backtest using different model update dates, and therefore different trained models.

The author asks whether changes to the rolling-training code could make both environments use the same model, but the document contains no answer or tested fix. It reports that rolling backtests looked strong across different start dates and that tracking performance was also favorable, while simulated live results were weaker. These are the author’s observations, not independently validated results. The post illustrates why model refresh schedules and evaluation timing can affect reproducibility, but it does not establish whether those differences fully explain the performance gap or rule out other causes.

Key ideas

• Different model update dates can cause a rolling-training backtest and a simulated live run to use different models.
• The author reports matching simulation and tracking results without rolling training, but divergent results after adding it.
• Strong rolling backtest results across start dates do not ensure that simulated live performance will match.
• The post asks how to align model schedules but provides no solution or independent analysis.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.