Interpreting CAPM Alpha P-Values and Model Assumptions

# CAPM alphas have unexpected p-value distribution


# CAPM alphas have unexpected p-value distribution












I am trying to "test" whether the EMH holds by testing for every stock in the S&P 500 whether it has a "significant" CAPM alpha.

If the EMH is true, then the null-hypothesis (alpha = 0) should hold for all stocks, which implies that the p-values for alpha should follow a uniform distribution. If the EMH is false, then I would still expect the histogram of p-values to be mostly flat (most stocks are efficiently priced) with a peak near p = 0 (under and overperforming stocks).

Instead I am getting the following histogram of p-values:

I am having trouble understanding why the distribution of p-values looks like this. I could come up with two possible explanations. The first is that my code and/or data is faulty. The second possible explanation is that the market is interfering with my "experiment" and is "rerandomizing" any alphas that appear to be significant.

Could someone help me understand what is going on here?

Here is the code that I used to produce the histogram:

```
library(tidyquant)
library(tidyverse)
library(broom)

# Compute returns of the S&P 500
returns_spy = tq_get("^GSPC") %>%
    tq_transmute(adjusted, periodReturn, period = "weekly", col_rename = "returns_weekly")

# Compute returns for all (individual) stocks in the S&P 500
sp500 <- tq_index("SP500")
stocks <- sp500 %>%
    select(symbol) %>%
    tq_get() # Fetch from Yahoo Finance
returns_stocks <- stocks %>%
    group_by(symbol) %>%
    tq_transmute(adjusted, periodReturn, period = "weekly", col_rename = "returns_weekly")

# Get the 10 year T-Bond rate from FRED
DGS10 <- tq_get("DGS10", get = "economic.data")
weekly_riskfree_rate <- DGS10 %>%
    fill(price) %>%
    transmute(date, returns_annual_riskfree = price / 100) %>%
    transmute(date, returns_weekly_riskfree = (1 + returns_annual_riskfree)^(1 / 52) - 1)

get_model <- function(stock_returns) {
    excess_returns <- stock_returns %>%
        left_join(returns_spy, by = c("date"), suffix = c("_stock", "_spy")) %>%
        left_join(weekly_riskfree_rate, by = c("date")) %>%
        transmute(
            date,
            excess_returns_weekly_stock = returns_weekly_stock - returns_weekly_riskfree,
            excess_returns_weekly_spy = returns_weekly_spy - returns_weekly_riskfree)
    model <- lm(excess_returns_weekly_stock ~ excess_returns_weekly_spy, excess_returns %>% tail(104))
    tidy(model)
}

pvalues <- returns_stocks %>%
    group_by(symbol) %>%
    nest() %>%
    transmute(symbol, model = map(data, get_model)) %>%
    unnest(model) %>%
    filter(term == "(Intercept)") %>%
    select(symbol, p.value) %>%
    arrange(p.value)

ggplot(pvalues, aes(p.value)) + geom_histogram(bins = 30)
```

## Answer by Dave Harris (score 1)

https://quant.stackexchange.com/a/63193

The distribution of p-values is uniform only when the null hypothesis is true and all assumptions are met. The difficulty with p-values is that they are composite functions.

Finding that an empirical set of p-values is far from uniform tells you one of two things.

The first is that your null is false.

OR

The probability assumptions behind the model are completely off.

An important mathematical assumption underlying the math involved in these models is that the parameters are known. The excuse for this is that the market behaves as if they are known.

However, there was a proof in 1958 that models like the CAPM or Black-Scholes have no solution in Pearson and Neyman's Frequentist or Fisher's Likelihood-based methods if the parameters are not known.

The MVUE and the MLE for $R$ in $w_{t+1}=Rw_t+\epsilon_{t+1},R>1$ is the least squares estimator, but its sampling distribution is the Cauchy distribution. That is no different than being a non-existence proof. The Cauchy distribution has no population mean and so the estimator diverges perfectly under any squares minimizing routine.

Even though $\epsilon$ is assumed to be centered on zero with finite variance, the sampling distribution of $R$ has no mean or variance. The Cauchy distribution is Student's t-distribution with 1 degree of freedom, regardless of the sample size. A million observations have no more information about the center of location when a sample mean issued than one observation.

The National Institutes of Standards and Technology says the following about the Cauchy distribution,

> The Cauchy distribution is important as an example of a pathological case. Cauchy distributions look similar to a normal distribution. However, they have much heavier tails. When studying hypothesis tests that assume normality, seeing how the tests perform on data from a Cauchy distribution is a good indicator of how sensitive the tests are to heavy-tail departures from normality. Likewise, it is a good check for robust techniques that are designed to work well under a wide variety of distributional assumptions. The mean and standard deviation of the Cauchy distribution are undefined. The practical meaning of this is that collecting 1,000 data points gives no more accurate an estimate of the mean and standard deviation than does a single point.

Models like the CAPM or Black-Scholes or any Ito model with equity securities only have solutions under Kolmogorov's axiom system if the parameters are known. Under other axiom systems, models like the CAPM fall apart. It appears that models like the CAPM are fragile and dependent on Kolmogorov's third axiom of countable additivity.