Regression Inference with Non-Normal, Heteroskedastic Time-Series Errors
Summary
The note distinguishes the distribution of regression errors from the sampling distribution of estimated coefficients. It explains that OLS does not require normally distributed errors to estimate coefficients, while heteroskedasticity and autocorrelation affect the usual covariance estimates and the efficiency conditions for OLS. Robust standard errors can support hypothesis tests using OLS coefficients: Huber–White addresses heteroskedasticity, and Newey–West addresses both heteroskedasticity and autocorrelation. The response says no special alteration to Newey–West is needed merely because residuals appear t-distributed.
Alternative models can address dependence or changing variance directly, including ARMA for autocorrelation and GARCH or stochastic volatility for heteroskedasticity. The note cautions that critical values depend on the hypothesis test; the error distribution alone does not determine them. It cites the Augmented Dickey–Fuller test as a case with special critical values. It does not specify a Newey–West lag choice, finite-sample correction, or a diagnostic procedure, so those implementation decisions remain outside its guidance.
Key ideas
- OLS coefficient estimation does not require normally distributed errors, though standard inference assumptions still matter.
- Heteroskedasticity and autocorrelation can be handled in covariance estimates using robust standard errors.
- Newey–West standard errors address both heteroskedasticity and autocorrelation for hypothesis testing.
- The distribution of errors differs from the sampling distribution of estimated coefficients.
- Some tests, including Augmented Dickey–Fuller, require test-specific critical values.
Tags
Full text
# Inferences with non-normal data # Inferences with non-normal data I have data of index closing values. I later will use to run some regressions on the percent changes. When examining the data, I find heteroscedastic residuals and that the distribution is non-normal. In fact, it more looks like a t-distribution. Number of observations is > 6000. My question is how I properly make inferences from this data if I would choose to not consider normality and instead go with t-distribution. How do I determine the critical values in such a distribution? After running the regressions and checking for autocorrelation in the residuals, say I find autocorrelation, do I need to tweak the Newey-West SE in any way for it to work? Anything else I need to consider? ## Answer by John (score 1, accepted) https://quant.stackexchange.com/a/17303 You are concerned about non-normality, heteroskedasticity, and autocorrelation in your data. The normality of errors is not an assumption of OLS (it is for MLE regression). That is, you can conclude that OLS is the best linear unbiased estimator (BLUE) without assuming normality. Nevertheless, there are a number of techniques within the context of robust regression to handle outliers and t distributed data, such as a Bayesian regression assuming t distributed errors. Lack of heteroskedasticity and autocorrelation are required for OLS to be BLUE. However, with some adjustments, you can still use OLS coefficients in hypothesis testing (though they will no longer be BLUE). All you have to do is adjust the standard errors (or more generally the covariance matrix of the OLS parameters). With new standard errors, you can make new t statistics and run any hypothesis tests you want. Huber-White is common in regression packages. It can correct for heteroskedasticity in the errors. Newey-West errors are a subsequent development. They can correct for autocorrelation and heteroskedasticity. This is particularly important for some time series data that is common in finance. If you're already using Newey-West errors, then you can construct the t statistics and run whatever hypothesis tests you need to. Another approach to autocorrelation is fit an ARMA model to the data. Similarly, with heteroskedasticity, you can fit a GARCH or SV model to the data. Before I end, you also ask about the critical values of the distribution of errors. People are sometimes confused in frequentist statistics that the distribution of the parameters is not the same thing as the distribution of the errors. For instance, assume the errors are normal. The distribution that you use for hypothesis testing is the t distribution. What distribution is this? It is the distribution of the OLS coefficients. That being said, there are a number of cases where the hypothesis testing requires different critical values. For instance, in the Augmented Dickey Fuller test you have to make adjustments because the coefficient being sufficiently different could mean that the underlying data will explode to infinity.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.