Diagnosing Autocorrelation in Time-Series Forecast Errors
Summary
The document discusses how to evaluate residuals when comparing time-series forecasts, including volatility forecasts from models such as GARCH. It recommends examining autocorrelation: if errors contain predictable structure, the model may be missing a pattern that could be represented with a revised specification. Ljung–Box is named as a standard test, with Box–Pierce and Durbin–Watson also mentioned.
A key qualification is that residual autocorrelation within a fixed model window differs from dependence among errors produced by rolling or expanding forecasts. In-sample residual autocorrelation can indicate a model weakness, while overlap across successive multi-step forecast errors can create dependence by construction. The document notes that an h-step forecast error process has an MA(h−1) structure. Thus, a diagnostic finding must be interpreted in light of the forecast horizon and evaluation design; the discussion offers no worked test or correction for overlapping errors.
Key ideas
- Autocorrelation tests can reveal predictable structure in a model’s residuals.
- The Ljung–Box test is cited as a standard diagnostic, alongside Box–Pierce and Durbin–Watson.
- Autocorrelation in fixed-window residuals differs from dependence in errors across rolling forecasts.
- Overlapping h-step forecast errors may follow an MA(h−1) process by construction.
- Residual dependence should be interpreted in the context of the forecast horizon and evaluation method.
Tags
Full text
# What type of analysis is appropriate for assessing the performance time-series forecasts? # What type of analysis is appropriate for assessing the performance time-series forecasts? When using time-series analysis to forecast some type of value, what types of error analysis are worth considering when trying to determine which models are appropriate. One of the big issues that may arise is that successive residuals between the 'forecast' and the 'realized' value of the variable may not be properly independent of one another as large amounts of data will be reused from one data point to its successive one. To give an example, if you fit a GARCH model to forecast volatility for a given time horizon, the fit will use a certain amount of data, and the forecast is produced and then compared to whatever the realized volatility was observed for the given period of time, and it is then possible to find some kind of 'loss' value for that forecast. Once everything moves forward a time period, assuming we refit (but even if we reuse the data parameters), there will be a very large overlap in all the data for this second forecast and realized volatility. Since it is common to desire a model that minimises these 'losses' in some sense, how do you deal with the residuals produced in this way? Is there a way to remove the dependency? Are successive residuals dependent, and how could this dependency be measured? What tools exist to analyse these residuals? ## Answer by Richard Herron (score 14, accepted) https://quant.stackexchange.com/a/62 I think you're looking for some way to test for autocorrelation in your residuals. If your model is good -- let's say you have an ARMA(1, 1) model for your forecast -- then the residuals from this model will be white noise. Which is to say that the difference between your forecast and the realization can not be predicted any better. The residual is some zero mean normally-distributed error. Let's pick an extreme example. If your residual (the diff between forecast and realized) were always 1, then the residuals would be autocorrelated. Clearly if your model is always off by 1, then you can do better. So if the residuals in your model are autocorrelated, then you can do better. The standard test for this these days in Ljung-Box, but in the past Box-Pierce and Durbin-Watson were also used. ## Answer by RockScience (score 5) https://quant.stackexchange.com/a/59 I am not sure I clearly understand your question. But definitely you can do some analysis on the residuals, especially autocorrelation. If you find any significant autocorrelation, I suggest you add a ARMA process to your model to increase the accuracy of your forecast. ## Answer by Richard Hardy (score 1) https://quant.stackexchange.com/a/30739 I am not sure I understand your question, but you might be conflating two different things: - autocorrelation in model residuals in a fixed sample (window) and - autocorrelation in forecast errors across samples (rolling or expanding windows). (1) is undesirable as it indicates the model misses a pattern which it should ideally capture. This can be remedied, for example, by changing the model. One may add an ARMA structure to the model's error term (to get ARMA-GARCH from pure GARCH, for example), change the model's autoregressive order, or do some other changes. (2) can happen by construction and need not indicate any problem with the forecasts or the modelling scheme that is generating them. Indeed, forecast errors of $h$ steps ahead will necessarily be MA($h-1$) processes; see e.g. Diebold "Forecasting in Economics, Business, Finance and Beyond" Chapter 10 "Point Forecast Evaluation", section 10.1 "Absolute Standards for Point Forecasts" (version of 14 December 2015; the linked version might change over time).
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.