Backtesting VaR Forecasts with Overlapping Horizons
Summary
The document considers VaR forecasts made monthly for losses through a fixed calendar year-end. Because each forecast covers a different, overlapping interval, a single late-year market decline can breach several forecasts, making the breach indicators dependent. This complicates the interpretation of a combined breach count and differs from the usual rolling one-period backtest, where each forecast is evaluated over its own subsequent, non-overlapping period.
The answers describe standard VaR diagnostics: unconditional coverage checks whether the violation rate matches the target rate, while independence tests look for clustering. One response argues that rolling forecasts must be re-estimated and tested according to their own horizons; fixed year-end forecasts should be assessed separately by horizon after the year ends. Another suggests estimating confidence intervals for overlapping-return backtests through Monte Carlo simulation, with a semi-analytical alternative mentioned but not detailed. The discussion does not supply a full procedure or settle all inference choices, and emphasizes that overlapping calendar-year forecasts are not interchangeable with ordinary one-period backtests.
Key ideas
- VaR forecasts with a common year-end date have overlapping holding periods, so their breaches can be dependent.
- A single large loss can trigger violations across multiple forecasts made earlier in the year.
- Coverage tests assess the violation frequency, while independence tests assess whether violations cluster.
- Rolling one-period VaR backtests differ from forecasts evaluated over shrinking horizons to a fixed date.
- Monte Carlo simulation can be used to estimate confidence intervals for overlapping-horizon backtests.
Tags
Full text
# Backesting VaR on overlapping intervals to year's end
# Backesting VaR on overlapping intervals to year's end
Let us assume that each month of the year (up to November) we calculate a VaR (say 99%) with holding period to the end of the year. Thus the holding period starts with 12 months and goes down to 1 month in November.
Obviously the event of a breach of my June VaR is not independent of the event of a breach of my Augus VaR (similar to here).
So how can I calculate the significance of the number of breaches? How can I take the dependence into account?
PS: At first sight it looks a bit artificial but a lot of institutions want to measure their risk in an (accounting) year. The next step is VaR and its backtest.
If look at monthly log-returns then in October I estimate $$ P[R_1+ R_2 + R_3 \le VaR_1] $$ and in November I look at $$ P[R_2 + R_3 \le VaR_2]. $$
Another edit: Say I estimated my VaR in Jan, Feb, up to Novenber and in December the market drops by 50%. Then I can have a breach in all my VaR estimates.
In the usual setting I estimate VaR and check on the following period. Then I estimate VaR again and check on the next period. Thus there is no overlap and the above problem can not happen!
I could only form series of "estimate yearly loss in Month x" and e.g. look at all January VaRs for the year 2000, 2001, .. 2016. They would be (more or less) independent. Then I could look at Februaries ... this would take away the overlap. But I would only have a few observations.
## Answer by Malick (score 1)
https://quant.stackexchange.com/a/25557
If I'm correct Backtesting VaR usually boils down to two conditions:
- The unconditional coverage hypothesis : the probability of an ex-post violation must be equal to the coverage rate. (ie : if 0.01 confidence level, you should get 1% violation). You can test it with the Kupiec Test .
- The independence hypothesis, your VaR violations should be independent. (ie : you shouldn’t observe violations clustering- past violations should not influence current and future violations.) Engle and Manganelli (2004) develop a test for it.
So, normally if your model is correctly specified you shouldn’t observe dependence in your series of violations.
The point I think you underestimate is that every month you re-estimate your model with new data. It means that your $VaR_{2}$ forecast will implicitly take into account your past violations. Your $VaR_{2}$ already integrate your $VaR_{1}$ violations since your $VaR_{2}$ is based on a larger sample. So you can not consider a kind of square root of time method to extend your VaR forecast because you need to re-estimate VaR parameters every months.
For more details about the independence hypothesis : Christoffersen, Peter, Backtesting (November 19, 2008). Available at SSRN: http://ssrn.com/abstract=2044825 or http://dx.doi.org/10.2139/ssrn.2044825
Edit :
I’m not sure I really got you point but I just clarify my talk :
- If you perform a rolling window VaR approach, ie every month you are estimating the next month VaR : then you perform 1 Month horizon VaR Forecast and not yearly VaR evaluation. There is no overlap. When you evaluate your VaR accuracy your are performing 11 VaR backtesting on a single period (one month) window. In doing so you can not obtain « yearly (calendar) loss estimate »
- If you perform VaR estimation with different horizon but a fixed ending date (the calendar year), then to evaluate the accuracy of you VaR forecast, for each month the accuracy is evaluated also on a different (decreasing) window. That means that a large drop in December may be classified as a violation in the January VaR estimation, not classified as a violation in February VaR estimation and again as a violation in a March VaR estimation. The accuracy of your backtests should be conducted separately for each horizon , then your are performing 11 VaR backtesting on 11 diffferent periods. Edit : you cannot evaluate the VaR accuracy every month, you have to wait the end of the calendar year to evaluate the 11 VaR.
## Answer by reverse_engineer (score 1)
https://quant.stackexchange.com/a/60287
I know this is an old question, but I encountered this problem too, years ago, and the answer is that you can actually compute confidence intervals for VaR backtesting of overlapping returns.
From what I understand, banks usually derive them with heavy Monte Carlo simulations (so they generate the overlapping returns a large amount of times and look at the excesses compared to a percentile and take the confidence intervals based on those excesses' distribution). This is actually a quite fine approach given the ready available computation power nowadays.
I had a slightly better semi-analytical approach that I had written up here:
https://share.reverservices.eu/legacy/reverse-engineerbe/posts/foolingaroundwithgauss/Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.