Skip to content
All library documents

Why Historical Backtest Win Rates Need Statistical Testing

Article Quant Q&A · Author: Curnelious
Read the original

Summary

The document asks whether a trading rule that would have made money on most days in a short historical sample should be used in live trading. It cautions that a simple count of profitable outcomes is not enough to establish that a rule has predictive value. Historical price behavior alone does not guarantee future results, and the response invokes the stochastic character of prices as one reason past observations may not provide a dependable signal.

A second response notes that researchers do use historical data to assess strategies, but typically apply more rigorous statistical analysis than counting correct calls. The discussion does not specify particular tests, address transaction costs or overfitting, or provide evidence about the example rule. It therefore motivates careful evaluation without supplying a complete backtesting methodology or a conclusion about any specific strategy.

Key ideas

  • A high historical fraction of profitable days does not by itself show that a trading rule will work in the future.
  • Historical observations are used in strategy research, but simple outcome counts are inadequate evidence.
  • Statistical analysis is needed to assess whether apparent historical performance reflects a repeatable signal.
  • The document does not prescribe tests or evaluate costs, overfitting, or the specific trading rule.

Tags

BacktestingStatisticsEquities
Full text
# Does historical backtest data mean anything?


# Does historical backtest data mean anything?












Sorry for this being a basic question.

If I take a stock’s historical data and check for some rule I have found to buy/sell = what happened if I bought and sold this stock according to this rule in the past 2 months every day? then I get a clear answer - “you would make profit in 75% of the days”.

Would this be any indication I should trade based upon this rule ? Or is it a bad approach because the past always produce a certain result?

If not- why, if yes- does algo companies use historical data like that?

## Answer by aalberti333 (score 2, accepted)

https://quant.stackexchange.com/a/41901

If your model is only relating to historical price data of that single stock, then the model wouldn’t be useful. Historical price data is stochastic, and a lot of theory in financial mathematics is based on this idea, meaning the expected value of a stock at any point in the future has no memory of (and is completely independent of) past prices.

## Answer by LazyCat (score 2)

https://quant.stackexchange.com/a/41903

There's a field of study called Statistics, which to a large extent tries to answer questions like that both in a financial setting and in experimental sciences. Try to read something about it. To your question, yes, people use the historical data this way, but usually, they perform a more rigorous statistical analysis, than just counting the number of times when the model is correct.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.