Skip to content
All library documents

Estimating the Maximum Sharpe Ratio Across Many Backtests

Article Quant Q&A · Author: not2qubit
Read the original

Summary

The document explains how to reproduce a plot illustrating the False Strategy theorem: as more independent strategies are backtested, the best observed Sharpe ratio can rise even when the strategies have no genuine edge. The example computes the 25th, 50th, and 75th percentiles of the maximum Sharpe ratio using a beta distribution, then plots the median and interquartile ribbon against the number of backtests on a logarithmic scale.

The calculation assumes each backtest covers one year of daily observations and uses a specified annualization convention. It describes a theoretical distribution-based construction, rather than a simulation or a test on real market data. The resulting curve helps show why selecting the strongest result from many trials can exaggerate apparent performance. Its interpretation depends on the assumption that the backtests are independent and share the modeled conditions; correlated strategies, different sample lengths, or other selection procedures may require a different analysis.

Key ideas

  • Testing more independent strategies can increase the maximum observed Sharpe ratio through selection alone.
  • The example derives quantiles of the maximum using a beta distribution and a Sharpe ratio distribution function.
  • The plotted center is the median, with the 25th and 75th percentiles forming an interquartile ribbon.
  • The example assumes one year of daily returns and annualizes the Sharpe ratio.
  • The result is conditional on independent backtests and does not establish the performance of real strategies.

Tags

BacktestingStatisticsRisk management
Full text
# How can I reproduce the experimental verification of the “False Strategy” theorem plot?


# How can I reproduce the experimental verification of the “False Strategy” theorem plot?












I recently came across the following blog post talking about the importance of back-testing overfitting, and a plot claiming to be an experimental verification of the False Strategy theorem.

The plot shown is:

I would like to reproduce this plot to better understand the issue, but cannot find the original source, nor any info how to produce such a plot.

How can I reproduce this plot?

PS. This question may also be related to this one.

## Answer by shabbychef (score 2, accepted)

https://quant.stackexchange.com/a/40398

Up to the presentation details, the combination of beta and Sharpe distribution function gives the plot data. Below is the code to compute and plot the median value and a ribbon between the 25th and 75th quantile, where the backtests are over a single year.

```
require(SharpeR)
require(dplyr)
require(ggplot2)

bt_len <- 252     # length of backtest
days_py  <- 252   # number of days per year
back_lens <- exp(seq(log(1),log(1e6),length.out=1000))

# compute 0.25, 0.5, and 0.75 quantiles
qv <- data_frame(nbacktest=unique(round(back_lens))) %>%
    mutate(q25=SharpeR::qsr(qbeta(0.25,nbacktest,1),df=bt_len-1,ope=days_py),
                 q50=SharpeR::qsr(qbeta(0.50,nbacktest,1),df=bt_len-1,ope=days_py),
                 q75=SharpeR::qsr(qbeta(0.75,nbacktest,1),df=bt_len-1,ope=days_py)) 

ph <- qv %>%
    ggplot(aes(x=nbacktest,y=q50,ymin=q25,ymax=q75)) +
    geom_line() + geom_ribbon(alpha=0.25) + 
    scale_x_log10() +
    labs(x='number of independent backtests',
             y='maximal Sharpe, annualized',
             title='maximal Sharpe over many independent 1 year backtests, median and IQR')
print(ph)
```

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.