Diagnosing and Improving a Historical VaR Model After Backtest Failure

Summary

The discussion asks what to do when a historical Value-at-Risk model fails conditional coverage or independence backtests. It recommends treating the failure as a prompt to investigate the model and its inputs, rather than assuming that changing the return lookback period is the only available adjustment. Possible causes mentioned include mismatched positions, omitted risk factors, and parameters that were not updated or calculated correctly.

Potential improvements include changing the observation period, applying volatility scaling, improving valuation accuracy, or adding relevant risk factors. The answer also suggests filtered historical simulation, which incorporates conditional volatility into historical simulation. The document points readers to general backtesting guidance and resources on filtered historical simulation, but provides no worked example, comparative evidence, or criteria for choosing among remedies. Its recommendations are therefore a starting point for diagnosis rather than a demonstrated fix; any change would need to be assessed against the model’s intended use and subsequent backtests.

Key ideas

• A failed VaR backtest should lead to investigation of the model and its inputs.
• Position mismatches, missing risk factors, and calculation errors can affect historical VaR.
• Lookback length is one possible adjustment, alongside volatility scaling and valuation improvements.
• Filtered historical simulation adds conditional volatility to a historical simulation approach.
• The discussion offers general recommendations but no evidence comparing their effectiveness.

Tags

# Procedures to follow when VaR model fails backtest


# Procedures to follow when VaR model fails backtest












I was wondering what the correct procedure is to follow when a VaR model fails a backtest (either conditional coverage and/or independence tests)?

Assuming I am restricted to using a historical VaR model, it seems that the only parameter than can be tuned is the lookback period of returns.

## Answer by Magic is in the chain (score 1)

https://quant.stackexchange.com/a/41909

Pages 8-11 of the BIS 1996 paper on backtesting provide very useful general guidance:

https://www.bis.org/publ/bcbs22.pdf

Hope this helps.

Many of these (positions mismatch, risk factors not captured, parameters not updated or calculated incorrectly) are as applicable to historical VaR as to any other VaR calculation approach.

In terms of the solution, in addition to the length of the observation period that you mentioned, one can also look at volatility scaling, improving the accuracy of the valuation, brining in additional risk factors etc. Understanding the causes is therefore very important when devising an improvement plan.

## Answer by AK88 (score 1)

https://quant.stackexchange.com/a/42399

Obviously, you biggest issue is the volatility. If you are restricted to use only Historical Simulation VaR, then consider incorporating conditional volatility to the model. Your backtesting will definitely improve if you do this.

Resources:

- An overview of Filtered Historical Simulation (FHS)

- Using Bootstrapping and Filtered Historical Simulation to Evaluate Market Risk

- Filtered historical simulation Value-at-Risk models and their competitors

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.