Assessing Statistical Significance in Backtested Long–Short Strategies

Summary

The document asks how to assess the statistical significance of a monthly rebalanced long–short value strategy over a long historical lookback. The answer recommends comparing several performance measures, including Sharpe and Sortino ratios, yearly profits, and annual maximum drawdowns. Suggested reference points are a buy-and-hold portfolio of the strategy’s stock universe and several randomly generated strategies rebalanced on the same schedule.

For a more formal comparison, the answer points to methods for testing whether the strategy’s Sharpe ratio exceeds that of a buy-and-hold benchmark based on an index of the same universe. This frames significance as a comparison with relevant alternatives, rather than an isolated score. The document does not give the formulas, address multiple testing or strategy selection, or discuss how dependence and market regimes affect inference. Its recommendations are a starting point, not a complete validation procedure.

Key ideas

• Evaluate a backtest against benchmarks drawn from the same investment universe.
• Compare performance measures such as Sharpe and Sortino ratios, yearly returns, and annual drawdowns.
• Randomized strategies with matching rebalancing frequency can provide additional comparison cases.
• A formal Sharpe-ratio comparison can test performance against a buy-and-hold index benchmark.
• The brief answer does not address multiple testing or other limits to backtest inference.

Tags

# how to I get the statistical significance of a backtested result


# how to I get the statistical significance of a backtested result












If I have a simple long/short value strategy (say long stocks with high e/p and short stocks low e/p or any other parameter) rebalanced monthly, and a look back window of say 15 years. How do I calculate the statistical significance of my result?

thanks,

## Answer by alexprice (score 5, accepted)

https://quant.stackexchange.com/a/50536

Compare Sharpe , Sortino Ratios, yearly Profit,Max Drawdowns per year of your strategy to

1) buy and hold all of the stocks in your universe

2) few strategies (with different random seeds) which randomly buy /sells stocks in your universe with monthly re-balancing

If you want to go more mathematical and get p-values that your strategy's Sharpe Ratio if higher than buy and hold then compare buy and hold of index comprising your universe's stocks to your Sharpe ratio by using formulas (2-sided) from

http://www.datamineit.com/Sharpe%20Ratio%20Comparisons%20-%20J.D.%20Opdyke%20-%20preprint%20-%20Journal%20of%20Asset%20Management,%20Vol8(5),%202007.pdf

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.