Using the Deflated Sharpe Ratio to Compare Strategies
Summary
The document introduces the Probabilistic Sharpe Ratio (PSR) and Deflated Sharpe Ratio (DSR) in the context of comparing two trading strategies. PSR estimates the probability that an observed Sharpe Ratio exceeds a chosen benchmark while accounting for sample length and the return series’ skewness and kurtosis. DSR adjusts the benchmark for the maximum Sharpe Ratio expected when many candidate strategies or backtests have been tried, addressing the risk of selecting a lucky result.
The question is whether the excess returns of strategy A over strategy B can be treated as a return series and evaluated with DSR to test relative performance. The document provides formulas and a conceptual description, but it does not include an answer or empirical comparison confirming that procedure. Its discussion therefore motivates adjustment for multiple trials but leaves open how to define the benchmark, trials, and return observations for a relative strategy test.
Key ideas
- PSR estimates whether an observed Sharpe Ratio exceeds a selected benchmark while incorporating return-distribution shape and sample length.
- DSR raises the benchmark to account for the best Sharpe Ratio expected across multiple trials.
- The document asks whether strategy A’s returns in excess of strategy B’s can be evaluated using DSR.
- No answer or empirical evidence is provided to validate that relative-performance procedure.
Tags
Full text
# Backtesting of outperformance of a benchmark using the Deflated Sharpe Ratio
# Backtesting of outperformance of a benchmark using the Deflated Sharpe Ratio
I want to test whether, let's say, strategy A outperforms strategy B.
In Marcos López de Prado's book Advances in Financial Machine Learning he presents the following statistics:
- The Probalistic Sharpe Ratio: the probability that a strategy's observed Sharpe Ratio $\hat{SR}$ exceeds a benchmark Sharpe Ratio $SR^*$ follows a normal distribution when we consider the skew $\hat{\gamma_3}$ and kurtosis $\hat{\gamma_4}$ of the strategy's return: $ \hat{PSR}[SR^*] = Z[\frac{(\hat{SR} - SR^*)\sqrt{T-1}}{\sqrt{1-\hat{\gamma_3}\hat{SR} + \frac{\hat{\gamma_4} - 1}{4}\hat{SR}}}] $
However, if we were to run many simulations, just by random chance, at some point we would find some strategy that outperforms our benchmark significantly. So a new statistic is introduced accounting the multiplicity of trials:
- The Deflated Sharpe Ratio, $\hat{DSR}=\hat{PSR}[SR^\star]$ where $SR^\star=\sqrt{Var(\{\hat{SR_n}\})}((1-\gamma)Z^{-1}[1-\frac{1}{N}]+\gamma Z^{-1}[1-\frac{1}{N}e^{-1}])$, where $\gamma$ is the Euler-Mascheroni constant and $e$ is Euler's constant. In fact, $SR^\star$ is the maximum expected Sharpe Ratio in the series of Sharpe Ratios (calculated over all backtests). The Deflated Sharpe Ratio tests whether a given strategy's excess returns are significantly higher than zero.
You can refer to this article for more information.
So far so good. In the book, López de Prado summarises saying that the DSR can be computed on both absolute as relative returns.
My question then is fairly simple. I have strategy A and strategy B, and ran $N$ simulations. If I want to test whether strategy A is expected to outperform strategy B, I can simply take A's excess returns over B, and calculate the deflated Sharpe Ratio over that return series?
Thank you all!Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.