Evaluating GARCH Volatility Forecasts Requires a Realized Volatility Proxy
Summary
The document asks how to compare volatility forecasts from ARMA(1,1)-GARCH(1,1) models fitted to S&P 500 returns under normal, Student-t, and GED innovations. The author describes generating rolling forecasts with an out-of-sample period and asks whether mean squared error, root mean squared error, or mean absolute error can be used to compare the models.
The answer says forecast evaluation requires realized volatility or another proxy for actual volatility, since true volatility is unobservable. It points to a review by Poon and Granger that surveys approaches used in prior research to construct such proxies. The discussion does not explain how to calculate the error metrics, specify a particular proxy, or assess whether the stated forecast procedure is correctly configured. Its main lesson is that a volatility forecast needs a suitable observed benchmark before error-based comparisons are meaningful.
Key ideas
- True financial volatility is unobservable, so forecast evaluation requires a realized volatility measure or proxy.
- The proposed comparison concerns GARCH forecasts under several innovation distributions.
- Error metrics such as MSE, RMSE, and MAE require a volatility benchmark to compare forecasts meaningfully.
- The answer refers readers to a literature review but does not specify a proxy construction method.
Tags
Full text
# how to calculate RMSE, MAE, given ugarchforecast results?
# how to calculate RMSE, MAE, given ugarchforecast results?
Given S&P500 returns for the past 20 years I fitted an ARMA(1,1)-GARCH(1,1) model using the rugarch package, so using ugarchspec() and the ugarchfit(), with different innovations distributions, i.e. norm, std, ged. My task would be to evaluate and compare the forecasting performance of the different models but I have problem to figure out how to do it. I then used the ugarchforecast as:
```
spec <- ugarchspec(variance.model = list("sGARCH", garch0rder = c(1,1),
submodel = NULL, external.regressors = NULL, variance.targeting = F),
mean.model = list(arma0rder = c(1,1), include.mean = T, archm = F,
archpow=1, arfima = F, external.regressors = NULL, archex = FALSE),
distribution.model = "norm", fixed.pars = list(ar1 = 0.6170, ma1 =
-0.6824, mu = 2e-04))
garch <- ugarchfit(spec, ret, out.sample = 100, solver = "solnp",
fit.control = list(stationarity = 1, fixed.se = 0, rec.init = "all"))
fore <- ugarchforecast(garch, n.ahead = 100, n.roll = 100)
```
Is that procedure correct? what should I do now to evaluate the forecast performance by comparing MSE, RMSE, MAE?
thank you!
## Answer by Neeraj (score 2, accepted)
https://quant.stackexchange.com/a/24571
To compare the performance among various model, you require realized volatility or proxy of actual volatility (actual volatility is always unobservable).
Go through this paper (link provided below) where authors have explained how previous authors constructed realized volatility or actual volatility.
Paper: Forecasting Volatility in Financial Markets: A Review by Poon and Granger. This is one of most finest paper, where authors reviewed 93 papers related to volatility forecasting and cover each and every aspects in details.Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.