Skip to content
All library documents

Regression Coefficient Tests and Multiple Testing Risk

Article Quant Q&A · Author: Harry
Read the original

Summary

The document distinguishes a joint hypothesis about regression coefficients from tests of individual coefficients. The null that all slope coefficients are zero is typically assessed with an overall F-test; individual p-values address separate coefficient hypotheses. It cautions that removing a predictor because its p-value exceeds a threshold and then re-estimating does not by itself establish a sound final model.

The answer illustrates multiple-testing risk: when three coefficients are tested separately at a 5% level, the chance of at least one false significant result can exceed 14% if all coefficients are truly zero. This highlights why nominal per-test significance does not equal the error rate across a set of tests. The source gives a brief conceptual warning rather than a full model-selection procedure, and notes that interpretation should also account for the underlying theory and how categorical indicators are represented.

Key ideas

  • An overall F-test evaluates whether all regression slopes are jointly zero.
  • Individual coefficient p-values test separate hypotheses about each parameter.
  • Repeated coefficient testing can raise the chance of a false positive across the model.
  • Model decisions should consider theory and the structure of included indicators, not p-values alone.

Tags

StatisticsMachine learning
Full text
# Null and Alternative hypothesis for multiple linear regression


# Null and Alternative hypothesis for multiple linear regression












I have 1 dependent variable and 3 independent variables.

I run multiple regression, and find that the p value for one of the independent variables is higher than 0.05 (95% is my confidence level).

I take that variable out and run it again. Both remaining independent variables have $p$-value less than 0.05 so I conclude I have my model.

Am I correct in thinking that initially, my null hypothesis is

$$H_0= β_1=β_2 = \dots =β_{k-1} = 0$$

and that the alternative hypothesis is

$$H_1=\textrm{At least one } β \neq 0 \textrm{ whilst } p<0.05$$

And that after the first regression, I do not reject, as one variable does not meet my confidence level needs...

So I run it again, and then reject the null as all $p$-values are significant?

Is what I have written accurate?

Edit: Thanks to Bob Jansen for improving this aesthetics of this post.

## Answer by user1483 (score 3)

https://quant.stackexchange.com/a/16127

The hypothesis $H_0: β_1=β_2=\dots =β_{k−1}=0$ is normally tested by the $F$-test for the regression.

You are carrying out 3 independent tests of your coefficients (Do you also have a constant in the regression or is the constant one of your three variables?) If you do three independent tests at a 5% level you have a probability of over 14% of finding one of the coefficients significant at the 5% level even if all coefficients are truly zero (the null hypothesis). This is often ignored but be careful. Even so, If the coefficient is close to significant I would think about the underlying theory before coming to a decision.

If you add dummies you will have a beta for each dummy

## Answer by Andrew (score 2)

https://quant.stackexchange.com/a/16066

These are independent variables so the hypothesis applies to each parameter independently.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.