Skip to content
All library documents

Walk-Forward Analysis and the Limits of Estimating Trading Edge

Article Quant Q&A · Author: Juan M. Almodóvar
Read the original

Summary

The discussion asks whether walk-forward analysis can estimate a trading system’s future predictability or alpha. It warns that even a holdout period becomes part of model selection once researchers inspect its results and use them to judge whether a model is useful. Repeated assembly, testing, and retesting can therefore leave an apparently promising system with limited evidence of future performance.

The practical recommendation is to test a system in real time and build criteria for abandoning it when performance deteriorates. One response says a carefully structured walk-forward process, described in a technical indicators reference book, can mitigate repeated testing and improve a system’s prospects. The document does not explain that process or provide empirical comparisons, so it offers no basis for quantifying how much alpha a strategy may capture. Its central caveat is that out-of-sample analysis can inform judgment but cannot prove future value; real-time performance remains uncertain.

Key ideas

  • Inspecting holdout results to choose whether a model is useful makes those results part of the selection process.
  • Repeated model testing can produce systems that look promising without establishing future performance.
  • Real-time observation is needed to assess a system’s value, and researchers should define conditions for dropping it.
  • A structured walk-forward process may reduce repeated-testing problems, but the document gives no details or evidence for the method.

Tags

BacktestingStatisticsRisk management
Full text
# Is Walk Forward Analysis a good method to estimate the edge of a trading system?


# Is Walk Forward Analysis a good method to estimate the edge of a trading system?












Do you think Walk Forward Analysis is a good method to estimate the predictability or edge of a trading system? Are there similar methods to know (estimate) how much alpha can capture an algo (in the future)?

## Answer by bill_080 (score 7, accepted)

https://quant.stackexchange.com/a/744

The term "Walk Forward Analysis" typically comes from technical analysis schemes. If that's the case here, I would be careful with whatever you're considering (or reading).

Even if you tune your model parameters (and I'm not talking about any TA scheme) with 80% of the data and then "check" your model with the remaining 20%, you're still using that last 20% to determine if your model is useful. As a result, whatever model you assemble/test/retest, the reality is, even if there is a solid basis for the structure of the model, the only way to prove its value is to use it in real time. So, it is worthwhile to build tests that allow you to dump your model at the first sign of trouble.

From personal experience, I can tell you that you'll dump far more models than you'll keep.

## Answer by B Seven (score 4)

https://quant.stackexchange.com/a/1977

I found a very good process for running a walk forward analysis in The Encyclopedia of Technical Market Indicators, Second Edition: http://www.amazon.com/Encyclopedia-Technical-Market-Indicators-Second/dp/0070120579 .

The approach in the book helps mitigate the problem described above of assemble/test/retest. When you finally implement a trading system, it should have a good chance of success.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.