Why Identical Factor Models Can Produce Different Backtest Results
Summary
The post raises a reproducibility question: two strategy templates reportedly use the same model, factors, parameters, and data, yet produce substantially different backtest results. The author asks which result is correct and whether one template contains an error. The two templates are described as a newly created strategy and a machine-learning example shared in a live session.
No code, performance figures, or explanation of the discrepancy is provided, so the post does not establish which output is valid or identify a cause. It illustrates why matching visible inputs may not guarantee comparable backtests: differences in template logic or execution details would need to be inspected before results can be trusted. Resolving the question would require comparing the full workflows and verifying that both use identical data handling, timing, and evaluation assumptions.
Key ideas
- The author reports different backtest outputs from templates said to share the same model, factors, parameters, and data.
- The post does not provide evidence that identifies which result is correct.
- A useful investigation would compare the complete template logic and data-processing steps.
- Backtest comparisons require checking that timing and evaluation assumptions also match.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.