Seven Common Quantitative Strategy Research Errors

Summary

This note surveys seven pitfalls in quantitative investing: survivorship bias, look-ahead bias, storytelling, data mining, signal decay and trading costs, outliers, and asymmetric long-short payoffs. It explains how current index constituents can distort historical tests, why revised or delayed data must be handled with point-in-time records, and how conclusions can change across periods and stock universes. It also warns that searching many factors can produce attractive in-sample results that fail out of sample.

The examples include comparisons of biased and less biased portfolios, factor results across different samples, and a fast-decaying signal whose apparent close-price performance depends on information unavailable until after the close. The note discusses turnover constraints, outlier treatment, and differences between long and short contributions. These examples illustrate research risks rather than establishing universal rules; the document gives limited methodological detail and does not provide enough underlying data to independently assess the reported comparisons. It recommends grounding factor hypotheses in financial reasoning and evaluating realistic timing and costs.

Key ideas

• Using only surviving companies in historical tests can inflate estimated returns and reverse factor conclusions.
• Backtests should use data as it was available at each historical decision point, including initial releases and publication delays.
• Factor results can depend strongly on the sample period and stock universe, while broad searches invite data snooping.
• Signal decay, turnover, and transaction costs must be considered together when setting portfolio adjustments.
• Outlier controls can materially change results, and their effects may vary by signal type.
• Long and short legs can contribute asymmetrically, with shorting costs and feasibility affecting realized returns.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.