Diagnosing Backtest Changes When Increasing Historical Lookback
Summary
A user asks why backtest results change when the number of historical days fetched before a test is increased, even when the strategy calculates a moving average such as a ten-period average. The question assumes that fetching at least the indicator’s lookback should make results identical, but the discussion does not establish the cause in this particular case.
Replies suggest comparing the underlying data across lookback settings and checking the platform’s source or exported data. One proposed explanation is that preprocessing such as factor clipping can create tied values; without a deterministic tie-breaking rule, the selected stocks may then differ even when factor values match. The thread also recommends independently retrieving data and comparing it row by row. These are troubleshooting suggestions rather than a confirmed diagnosis, and the post supplies no example outputs or controlled test. Its practical lesson is to inspect data, preprocessing, and ranking behavior before attributing changing results to the indicator window alone.
Key ideas
- A moving average’s calculation window and the amount of historical data fetched for a backtest are distinct settings.
- Changing the fetched history can coincide with changes in backtest results, but the thread does not identify a confirmed cause.
- Compare exported data across settings to locate where values begin to differ.
- Factor clipping can create tied values that affect stock selection when ranking rules do not resolve ties deterministically.
- Independent data retrieval and row-by-row comparisons are suggested as debugging methods.
- The discussion provides troubleshooting ideas rather than controlled evidence or a definitive platform explanation.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.