Why Separate Mean Tests Can Agree with an Equality Test
Summary
The document presents an apparent contradiction from three two-sided t-tests: one fails to reject a zero mean for a company’s monthly stock returns, another rejects a zero mean for the market index, and a third fails to reject equal means for the stock and index. These outcomes can coexist because the tests ask different questions. The equality test allows both means to be nonzero while still being equal, and its result does not follow mechanically from the separate tests against zero.
A coin-flipping example illustrates the point: samples from two coins may lead to different conclusions about each coin’s bias relative to a fair coin, while the evidence remains insufficient to distinguish their biases from each other. The answer offers intuition, not the underlying return data, test statistics, assumptions, or a detailed account of dependence between stock and index returns. In practice, failure to reject a hypothesis is not proof that it is true; it means the sample does not provide enough evidence against that particular null hypothesis.
Key ideas
- Each hypothesis test answers a distinct question about the means being compared.
- Two means can each be nonzero and still be equal to one another.
- Rejecting a zero-mean hypothesis for one series does not determine the result of an equality test.
- Failure to reject a hypothesis is not proof that the hypothesis is true.
- The coin example illustrates how separate significance decisions can differ from a direct comparison.
Tags
Full text
# Hypothesis Test Contradiction? # Hypothesis Test Contradiction? I have a question regarding hypothesis testing. I used the t-test (2-tailed) for these hypotheses: - Whether the (monthly) mean return of company A's stock is different from 0 - Whether the (monthly) mean return of S&P500 is different from 0 - Whether the (monthly) mean return of company A's stock is different from (monthly) mean return of S&P500. The result of my tests showed that the mean return of company A's stock is 0, while rejecting the hypothesis the the mean return of S&P500 is 0. However, the result of the last test showed that the mean return of the company A's stock is the same as that of S&P500. Can someone explain what happened here? And what is the reason behind that? Thank you so much for your help. ## Answer by Arshdeep (score 1) https://quant.stackexchange.com/a/76167 The third test considers possibility of both means being non 0 but equal and that may as well be quite plausible, as in the coin example below. Consider flipping 2 coins and getting 9 and 6 heads on each. You will conclude (say 90% confidence level) that first coin is biased, second is unbiased but the hypothesis both have the same bias cannot be rejected. Obviously this is a very likely outcome if the joint bias is $Pr(head)$ in $[0.7,0.8]$.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.