Linear Regression Assumptions, Diagnostics, and Limitations
Summary
The article explains the conditions commonly used to justify ordinary least squares regression and why they matter for estimation and inference. It covers linearity in the model parameters, lack of perfect multicollinearity, independent errors, constant error variance, and a zero conditional mean. Normally distributed errors are presented as an additional assumption mainly relevant to inference. The text also distinguishes linearity in parameters from the shape of the predictors, which may be transformed or expanded with terms such as powers.
Suggested diagnostics include residual plots, predictor correlations and variance inflation factors, Durbin–Watson and Ljung–Box tests, and tests for unequal variance. Possible responses include transforming variables, adding predictors, reducing correlated dimensions, and using procedures for autocorrelated errors. The article notes that violations can undermine efficiency, standard errors, inference, or fit, and that regression can be too simple, sensitive to outliers, or overfit in complex forms. It provides a general checklist rather than a complete treatment; remedies depend on the data and modeling goal.
Key ideas
- OLS estimates are most useful when the model is correctly specified and its core error assumptions are reasonable.
- Linearity refers to the parameters, so transformed predictors can still be used in linear regression.
- Residual plots and statistical tests can help detect misspecification, dependence, and changing error variance.
- Multicollinearity makes it difficult to separate predictor effects and can inflate coefficient uncertainty.
- Assumption violations affect estimates, standard errors, efficiency, or inference, and remedies depend on the problem.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.