Approaches to Backtesting Expected Shortfall Risk Estimates
Summary
The document considers how to assess expected shortfall, also called conditional value at risk, when it is not directly elicitable as a standalone forecast. It discusses a proposal to approximate expected shortfall by averaging several value-at-risk quantiles at different confidence levels, then evaluate the related quantile forecasts. The author reports that a simple comparison using a Student-t loss model showed a sizable difference between this approximation and analytical expected shortfall, raising a question about the proxy's adequacy.
Responses point to alternative lines of work: published backtesting methods for expected shortfall and regulatory capital, and a Berkowitz tail test that evaluates the distribution's tail and thereby bears on CVaR indirectly. The material is a brief collection of a question and references, not a comparative study. It supplies no implementation details, test statistics, or evidence establishing which method is preferred in current industry practice, so the cited approaches require further evaluation for a particular risk model and data set.
Key ideas
- Expected shortfall is difficult to backtest directly because it is not elicitable on its own.
- One proposed approach approximates expected shortfall with several value-at-risk quantiles and backtests those forecasts.
- The author reports that this quantile approximation differed from analytical expected shortfall in a Student-t example.
- Published alternatives include expected-shortfall backtests for regulatory capital and tests of the modeled density tail.
- The document does not compare methods empirically or establish an industry-standard approach.
Tags
Full text
# Expected Shortfall (CVaR) Backtesting
# Expected Shortfall (CVaR) Backtesting
I am writing my thesis on VaR and ES risk measurements and have encountered some issues with how to best test the accuracy of ES estimates.
My understanding of the topic is that backtesting ES adequately is extremely difficult or even impossible, as it is not an elicitable risk measure. And I also assume the number of approaches in the literature are scarcity because of exactly this property.
However very recently D.Tasche et.al. (http://arxiv.org/pdf/1312.1645v2.pdf) uploaded a paper where it was argued that ES is not directly elicitable, but indirectly elicitable because it can be approximated by several VaR estimates. Thus, they state ES can be backtested reasonably by backtesting several VaRs (at different confidence levels) related to the ES estimate. Their specific proposition is:
$ E{S_\gamma }(L) = {1 \over {1 - \gamma }}\int_\gamma ^1 q u(L)du \approx {1 \over 4}\left( {{{\rm{q}}_\gamma }{\rm{(L) + }}{{\rm{q}}_{0.75\gamma + 0.25}}(L) + {{\rm{q}}_{0.5\gamma + 0.5}}(L) + {{\rm{q}}_{0.25\gamma + 0.75}}(L)} \right)$ where L is the loss distribution and gamma the chosen confidence level.
From my simple calculations, proxying financial data with a student-t(6) the accumulated VaRs with this approach seems to deviate much from the analytical ES.
I wonder if anyone find the Tasche et.al. approach appealing/adequate for backtesting ES? Further I am also interested in knowing what the "state of the art" approach is in the industry (if any particular) for backtesting ES.
Any help would be greatly appreciated.
## Answer by user15083 (score 2)
https://quant.stackexchange.com/a/16227
You can also check out the expected shortfall backtesting methodology proposed here:
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2514403
## Answer by user7448 (score 1)
https://quant.stackexchange.com/a/10472
You can find a backtest for expected shortfall detailed in the paper below
Kerkhof, F.L.J., & Melenberg, B. (2004). Backtesting for risk-based regulatory capital. Journal of Banking and Finance, 28, 1845-1865.
Best, JK
## Answer by emcor (score 0)
https://quant.stackexchange.com/a/16232
The Berkowitz tail test allows to test the density tail, and hence indirectly evaluate the CVaR risk measure:
http://www.ims.nus.edu.sg/Programs/econometrics/files/kw_ref_2.pdfShown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.