Interpreting Non-Rejection and Confidence Intervals in Hypothesis Tests

Summary

The document explains what it means when a test statistic falls within the non-rejection region at a stated confidence level. The result is a failure to reject the null hypothesis; it does not establish that the null is true. A non-rejection may reflect limited data or low power, so the test alone cannot distinguish a plausible null from an inconclusive result.

To assess whether a parameter is near a hypothesized value, the answer recommends examining its confidence interval and whether the interval is sufficiently narrow for the practical question. The examples show how intervals around a mean can become narrower as more data are collected, while a very small but statistically detectable difference may still be negligible in practice. The answer also notes the equivalence between a two-sided test and checking whether the null value lies outside the corresponding confidence interval. Its illustration describes a confidence interval as having a 95% probability of covering the true mean; under the usual frequentist interpretation, the confidence level applies to the long-run coverage of the interval procedure, not to the probability that a particular computed interval contains the parameter.

Key ideas

• Failure to reject a null hypothesis does not prove that the null is true.
• A non-rejection may result from insufficient data or low statistical power.
• Confidence intervals show the range of parameter values compatible with the data and model.
• Narrow intervals can help assess whether an effect is practically close to a hypothesized value.
• For a two-sided test, the null value is rejected at a given level when it falls outside the corresponding confidence interval.

Tags

# Can a null be inconclusive?


# Can a null be inconclusive?












My Null for the T-test is

h0: -tcritical < Tstat < +tcritical

I require confidence level of 95%.

If my empirical result satisfies the null, but not my p-value requirements,

does this mean that the test is inconclusive? Or that the null is not rejected/rejected?

## Answer by airguru (score 2)

https://quant.stackexchange.com/a/16092

That means the null is not rejected and therefore test is inconclusive. With this type of testing you can only try to reject the null. Your non-rejection could have been due to lack of data, therefore you cannot conclude anything from it.

If you want to somehow support your null, compute the confidence interval for your parameters and show that its sufficiently narrow around your null. This will also give you a good overview about the quality of your data.

For example: Suppose you have null that the mean of your distribution is 0 (and the real distribution variance is for example ~1), and you compute the 95% confidence interval for mean as (-0.2,0.3). This means that with 95% probability the confidence interval covers the real mean, which sort of means, that the real mean is not far from the 0 in the sense of those numbers. When you add more data and your confidence interval shrinks to (-0.005,0.01) you can immediately see how the data adjusted and possibly supported your conclusion.

And more of a philosophical note : The real mean is most probably not zero, because the world is most probably not ideal. If you measure ton of data in the above example and get the confidence interval of (0.0001,0.0002), the null is rejected, but i assume that for all practical purposes the mean is effectively negligible.

I hope you are aware of the equivalence: null is rejected on X% confidence level <=> null value is outside of X% confidence interval.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.