Testing Whether One Portfolio Has Higher Factor-Model Alpha

# Interpreting Statistically Significant (or Insignificant) Difference in Alpha Between Two Portfolios


# Interpreting Statistically Significant (or Insignificant) Difference in Alpha Between Two Portfolios












Suppose that I have two portfolios: A and B. The factor model used to compare these two portfolios is the same.

Now, suppose model A has an alpha of 0.3 while model B has an alpha of 0.8. Assume that both are statistically significant. So, both portfolios have a positive (statistically significant) alpha. Next, assume that I run the difference between these two portfolios (B minus A) on the same factor model. Obviously, the alpha of this portfolio will be 0.5

I have the following questions. Suppose that the alpha of 0.5 on the B minus A portfolio is not statistically significant. Does that mean that I cannot claim that portfolio B has a superior alpha of 0.8 compared to portfolio A which has an alpha of only 0.3? In other words, is the B minus A portfolio estimation a way to test whether the difference in the alpha between the A and B portfolios is statistically significant? Or does it simply mean that it doesn't make sense to form a long-short portfolio comprised of portfolios A and B, but I can still claim that portfolio B is superior to portfolio A (without attempting to create a long-short portfolio)?

To restate, I am trying to understand whether I can claim that portfolio B has a superior return in excess of the risk-free rate compared to portfolio A or whether I cannot make that claim if the B minus A portfolio alpha is not statistically significant.

Lastly, how would the interpretation change if the alpha of the B-A portfolio is statistically significant while both portfolios A and B have positive alpha (and both are statistically significant)?

I would very much appreciate anyone's help on this issue!

## Answer by Dave Harris (score 1)

https://quant.stackexchange.com/a/71781

No, your logic is not correct. As a disclosure note, I am an opponent of the method you are using.

Let us assume that you run a Frequentist regression of any kind on phenomena $A$ and $B$. You are asserting a null hypothesis in order to get significance. $A$ could be that gravity has no effect on the motion of objects, and $B$ is that Mendelian inheritance has no impact on future generations of peas. Those statements are a test of something but do not imply anything about the other. There is nothing in the structure of a linear regression to imply that $A$ is the perfect fungible substitute for $B$. There is also nothing to imply that they are imperfect substitutes. Your equations in no way link them. You are adding math that you did not use in tests.

If you are relating $A$ and $B$, as they logically are, then you need to build a mathematical bridge between them.

Also, your portfolios are not random samples. When you do not randomly sample, you have to be very careful what questions you are and are not asking.

The simplest way to test if there is a net difference would be to go long one portfolio and short the other in a combined portfolio. Alternatively, you could build a vector regression and make the linkages explicit.

There is another, deeper issue, though. Finance constructs portfolios as the outcome of an optimization problem. If you get a specific portfolio out of using a tool, such as the Black CAPM, for example, you have chosen a specific form for your utility function.

If portfolio $A$ is optimal for person $X$ and portfolio $B$ is optimal for person $Y$ under the Black-CAPM, again as a neutral example, then any movement from $A$ to $B$ by $X$ is individually irrational.

Now, let us start with a known fact that the Black-CAPM is exactly true. That isn't a known fact, but we are pretending it is known. If it is true, then regardless of the measurements of $\alpha$, we must ignore them because $\alpha\equiv{0}$ no matter what has at times been observed.

Now let us take the opposite position, let us start with the perfect knowledge that the Black-CAPM is strictly false. We calculate $\hat{\alpha}$ and, as expected, we falsify the null that $\alpha\equiv{0}$. So, knowing that it is false, we ignore the entire model as it falsifies the entire thing. That being the case, we abandon the entire approach because we successfully falsified the null.

With factor models, it is a bit more complicated because there are multiple uses of the term factor in statistics and finance.

Finally, if you did perform some type of vector regression, you will want to be very careful about assumption violations. If they are partial substitutes, then they are not independent anymore. If they are perfect substitutes and $\alpha\equiv{0}$, then they are collinear.

Finally, in answer to your specific question, given the structure of your models, that you ran two separate, independent, linear regressions, you cannot rank-order $\alpha$. Your lines are functions.

Frequentist methods, in the general case, minimize the maximum amount of risk from using an estimator. Because risk is a function, there is no guarantee that you will find a function that dominates another function in every region. You cannot talk about a significance test between two portfolios unless you build that relationship into the mathematics of the problem so that you can then build a mini-max estimator. You will need to estimate the coefficients for that mathematical linkage. For example, is there a portfolio $C$ that is optimal for $Z$ that contains some of $A$ and some of $B$?

EDIT In response to the comment, if your null is $\alpha=0$ and it is rejected, since the slope is of no interest to you, your interpretation would be no different than any other hypothesis test. If the null is true, that $\alpha=0$, then it is unlikely to see data as extreme or more extreme than the $\hat{\alpha}$ that you observed.

Frequentist tests do not permit greater interpretation because, by assumption, the null is true. It is not possible to separate out the null is true but the data is unusual from the null is false. There is another technical issue that I will deal with after we handle it not being falsified.

Now let us assume that $\hat{\alpha}$ does not cause the rejection of the null. If you are using a Fisherian interpretation of probability, then the interpretation would be that you discovered nothing and you should move on with your life. The weight of the evidence against the null was insufficient to reject it; however, Fisher did not permit an alternative hypothesis. As such, all you know is that you know no more than before you began.

If you are using a Pearson and Neyman interpretation, particularly if you are working in a decision-theoretic perspective, which it appears that you are, then the interpretation is that $\alpha=0$. You will accept the null. You should treat your measured $\hat{\alpha}$ as an idiosyncratic result.

Now to the technical issue.

It does not sound as if your portfolios were randomly selected. You didn't randomly grab 3500 Americans and poll them for their income to determine if it was greater than some value. It is as if you grabbed two specific classrooms of school children, probably in the same grade, and decided to compare their achievements. You decided to compare them on two measures, $(\beta,\alpha)$, but you only care about $\alpha$, making $\beta$ a nuisance parameter.

That probably cannot be interpreted without quite a bit more information. Even with a significant amount of information, it may have no interpretation.

For example, if portfolio $A$ is the DJIA and $B$ is the S&P 500, they are going to be nearly collinear. The intercept may be functionally meaningless as are all of your coefficients. It will depend on your methodology, but your violation of assumptions would be so significant that you would need an abundance of caution.

Again, that may change if you change methodologies. If you did not randomly select your portfolios, it isn't possible to really discuss what $\alpha$ means without discussion of things like validity, model assumptions, how you arrived at the factors and so on. If you changed from least squares to ridge regression, quantile regression or some other tool, for example, you would change the discussion quite a bit.

As a general rule of thumb, your methodology and your stated hypothesis will drive any interpretation.

On the Frequency side of the probability fence, a hypothesis statement, such as $\alpha=0$, written in Keynesian notation as $\Pr(X;Y|\alpha=0)$ explicitly incorporates the null but implicitly incorporates the loss function when you chose an estimator, the model itself, and the sampling method. When business statistics is taught, that gets glossed over or ignored entirely.

Without an explanation of literally everything, the most you could say is that if $\alpha$ were rejected as different from zero, that it is the amount that you could not explain from the other coefficients. At best, it reflects missing relationships or missing information. At worst, it just worked out that way.

## Answer by TwoII (score 0)

https://quant.stackexchange.com/a/71756

The way I interpret your question is:

How can you compare portfolio A and portfolio B in deciding which portfolio is the better (superior) choice.

A simple answer to this question is:

You would need to make portfolio A and portfolio B comparable. If the portfolios evaluated are the entire risky investment, such a measure could be the Sharpe ratio or M^2. Using these measures, you care about reward to total risk.

With regards to the factor regression

I do not fully understand what you are initiating. $\alpha$ in the regression is simply the part of the returns of the portfolio (e.g., portfolio A) that the factors do not 'account' for. If you were to take $portfolio_A - portfolio_B$, this would be the same as having a new portfolio, $portfolio_C$. Running the regression on $Portfolio_C$ would not necessarily return a significant $\alpha$. And I do not really believe that your statement

> Obviously, the alpha of this portfolio will be 0.5

would necessarily hold.

EDIT: Follow up on the subsequent questions:

Significance Interpretation

Following in the spirit of the first part of my answer, what your regression results implies is that while you find $portfolio_A$ and $portfolio_B$ to have significant alpha, the combined portfolio - $portfolio_C$ (long-short) does not.

What you can say, given that you believe in the model framework that you have used, is that both $portfolio_A$ and $portfolio_B$ yields significant $\alpha$. Given the level of significance, it should also imply how confident you are in the implications. However, you cannot say that one $\alpha$ is better than the other, or more significant than the other - referring to what you imply in your question.

Using a measure with the assumptions mentioned above, however, you could say that one portfolio is a better choice then the other based on a preference of achieving a higher reward to total risk.