Skip to content
All library documents

Non-Normal Regression Errors: Focus on Exogeneity and Dependence

Article Quant Q&A · Author: Harry
Read the original

Summary

The document addresses a common interpretation of residual normal probability plots in multiple linear regression, using value at risk as an example dependent variable. Its central point is that normality of the errors is not required to estimate regression coefficients by least squares. More fundamental conditions discussed are exogeneity, constant error variance, and lack of correlation; under these assumptions, the response describes the estimators as unbiased, efficient within a class, and consistent.

Normal errors help justify conventional finite-sample standard errors and confidence intervals, but non-normality does not by itself make a model useless. The discussion mentions bootstrapping as one way to construct intervals without Gaussian errors and notes that large samples can make inference less sensitive to non-normality. Heavy tails may still affect uncertainty estimates, so residual shape should be assessed alongside other assumptions. The answers do not provide a model-specific diagnostic workflow, and one brief suggestion to try regularization is not developed or supported as a remedy for non-normal errors.

Key ideas

  • Least-squares regression does not require normally distributed errors to estimate coefficients.
  • Exogeneity, homoscedasticity, and uncorrelated errors are more central assumptions in the discussion.
  • Normality supports conventional finite-sample inference but is not essential to coefficient estimation.
  • Bootstrapping can provide confidence intervals when errors are not Gaussian.
  • Large samples can reduce the impact of non-normality on test statistics, though heavy tails still affect uncertainty.

Tags

StatisticsRisk management
Full text
# Ran multivariate linear regression, checked normal probability plot, residuals are not normal. What can I do?


# Ran multivariate linear regression, checked normal probability plot, residuals are not normal. What can I do?












One of the required assumptions for multiple linear regression is that residuals are normally distributed, correct?

After running my regression, my normal probability plot is showing the typical 'heavy tail' S shape.

Does this inability to satisfy the assumption deem my whole model useless? Is there anyway I can get normal residuals?

My dependent variable is VaR, and my independent variables are Average Return, Log of Market value, dummy variable 1 and dummy variable 2.

Edit: I've tried transforming the independent variable (VaR)(Square root, Log, reciprocal), but it doesn't seem to make sufficient difference

## Answer by jenny (score 0, accepted)

https://quant.stackexchange.com/a/16112

This means that a linear regression is not the best model for your data. You may want to try a regularized regression (LASSO/Ridge) to see if penalizing the coefficients will help.

## Answer by Kiwiakos (score 7)

https://quant.stackexchange.com/a/16084

Regression analysis, as a minimization of the sum of squared errors, does not require normality of the error term.

The requirements are that errors are homoscedastic and uncorrelated. And these are the fundamental assumptions (together with exogeneity). Then estimators are unbiased, optimal (exhibit the minimum variance within the class of unbiased estimators) and consistent (the variance also goes to zero with sample size). Normality is not required.

If errors are normal, then we can also say something about the standard errors of these estimates and build confidence intervals. However, there are ways to build these confidence intervals even if the errors are not Gaussian, for example by bootstrapping them. Therefore I would focus on the other assumptions which are more material.

## Answer by user1483 (score 1)

https://quant.stackexchange.com/a/16124

If your errors are non-normal and your sample is large non-normality is not important. You can rely on the Central Limit Theorem which implies that the test statistics (t and F statistics) have approximately the same distribution as in the normal case. Standard errors, in your case will be larger because of the fat tails. For a good textbook treatment of this treatment see, for example, Wooldridge (2013), Introductory Econometrics, Fifth Edition, South Western.

You could do your tests in Excel but you would have to do a lot of work and I would not recommend it. You should have a look at gretl http://gretl.sourceforge.net/ which is an easy to use econometrics package and is free.

If you wish to do some serious econometrics you need to obtain a better understanding of the underlying theory. You could start with Wooldridge or any of the other excellent test books available. Many introductory texts will give the normality requirement in an early section and then generalize this in a later section.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.