Regression and Mean-Variance Optimization for Combining Signals

# Linear Regression vs Mean Variance Optimization


# Linear Regression vs Mean Variance Optimization












Assume I have `n` signals, which I would like to linearly weight and combine to form an aggregate signal. Two possible ways of doing this based on historical data are:

- Regress the `n` signals on historical returns. Use the betas estimated from the regression as the weights on the signals.

- Estimate a covariance matrix of the signals based on historical signal values. Estimate future returns (mean) of the signals based off of the historical returns of the signals. Perform a mean-variance optimization and obtain optimal weights of signals

I understand the technical differences between these two approaches (different objective functions) but I'm trying to grasp (in practice) what is a better approach, as empirically they come up with similar answers.

In both cases, you are using historical signal performance to estimate future signal performance (e.g. momentum of the signal returns). In both cases, you can constrain the optimization to achieve different objectives (restrict weights to be positive, sum to 1 etc.). The linear regression objective - squared loss - is somewhat similar to the mean variance objective in that you are maximizing return. Not quite sure if the minimizing variance part is as obvious

It seems as though the mean variance approach is a little more flexible, you can detangle the expected returns from the covariance estimates (or even having no estimates of expected returns - e.g. minimum variance). Wondering if anyone has any insight

## Answer by Richi Wa (score 8, accepted)

https://quant.stackexchange.com/a/38770

In a linear regression approach you do the following: $$ (X \beta - y)^2 \rightarrow Min $$ thus you try to predict something. Your objective is quadratic. You usually add constraints on $\sum \beta_i^2$ or $\sum |\beta_i|$. Without constraints the estimator is: $$ \hat{\beta} = (X^T X)^{-1} X^T y, $$ where $X^T y$ has to do with the covariance of $X$ and $y$ and $(X^T X)^{-1}$ normalizes for the co/variance of $X$. This is the beta we all know from school.

Thus we can interpret $(X^T X)^{-1}X^T = P$ as matrix and $\hat{\beta} = P y$ - meaning that we try to get the information how much $X$ is there in $y$. $P$ is the "information extractor".

Finally the estimate $\hat{\beta} X = \hat{y}$ and see how much of $y$ can be explained using the "information extractor" and the value of $X$ to $y$. This has a lot to do with a projection on the space spanned by $X.$

In the case of portfolio optimization your problem is (note that $1/2$ is just there to ease further computations) $$ \frac12 w^T \Sigma w -w^T \mu \rightarrow Min, $$ where you have constraints on $w$, $\Sigma$ is positive definite. $\Sigma$ can be the estimate of asset returns $\Sigma = \frac{1}{n-1} R^TR$ where $R$ are asset returns. We can follow e.g. Roncalli and find the explicite solution in the case of no constraints (comparable to the setting above): $$ w = \Sigma^{-1} \mu, $$ meaning that $w_i$, your bet on asset $i$, is driven by the expected value $\mu_i$ divided by the risk. We could also say this is a porjection of $\mu$ on the space $\Sigma^{-1} = (\frac{1}{n-1} R^TR)^{-1}$.

As a last idea: If we do not care for absolure performance but we want to track some return $y$ and the objective is tracking error, then we try to find weights $w$ such that $$ (Rw - y)^2 \rightarrow Min. $$ Thus in the setting of tracking error optimization you get problems that match the regression setting pretty closely.