Why Random Portfolio Sampling Misses the Minimum-Variance Portfolio

# Cant replicate minimum variance portfolio variance by simulating many random portfolios in R


# Cant replicate minimum variance portfolio variance by simulating many random portfolios in R












I have computed the theoretical minimum variance portfolio using the 30 stocks in the Dow. The formula used is:

$$\underset{N\times 1}{\omega_{mvp}}=\frac{\lambda}{2}\cdot \Sigma^{-1}\iota=\frac{\Sigma^{-1}\iota}{\iota'\Sigma^{-1}\iota}$$

Where $\iota$ is a $N\times 1$ vector containing 1's.

For the data downloaded i get approximatly 0.0002712748, which i call Sigma_mvp in the script.

Then i generate one million different vectors containing 30 weights each for the assets. The weights can be negative, and i make sure they sum to one by dividing with the column sum.

My problem is: The smallest variance i manage to get with these random weights is 0.0004467729, so something must be wrong.

Any ideas? Hope the question is clear.

My code is provided below:

```
library(tidyquant)
library(tidyverse)

############ Getting Data for DOW ##############

tickers <- tq_index("DOW")

N <- length(tickers$symbol) # number of assets = 30

ones <- as.matrix(rep(1,N), ncol = 1) # one vector for later use

data <- tickers %>% tq_get(get = "stock.prices")

# calculate weekly returns

returns <- data %>% 
    group_by(symbol) %>% 
    tq_transmute(select = adjusted,
                 mutate_fun = to.weekly,
                 indexAt = "lastof") %>% 
    mutate(return = (log(adjusted) - log(lag(adjusted)))) %>% 
    na.omit()

# mean return vector

asset_returns <- returns %>% group_by(symbol) %>% 
  summarise(expected_return = mean(return)) %>% 
  select(expected_return) %>% as.matrix()

rownames(asset_returns) <- tickers$symbol %>% sort()

# create covariance matrix

Sigma <- returns %>%
  select(-adjusted) %>%
  pivot_wider(names_from = symbol, values_from = return) %>% # reorder data to a T x N matrix
   na.omit() %>% # remove NA that got generated by "DOW"
  select(-date) %>%
  cov(use = "pairwise.complete.obs")

Sigma <- Sigma[rownames(asset_returns),rownames(asset_returns)] # reorder matrix to match asset_return vector sequence

############## Generating random portfolios ###################

# random weights

w_rdm <- matrix(runif(n = 1000000 * N, min = -3, max = 3), nrow = N)

w_rdm <- apply(w_rdm,2,function(x){x/sum(x)})

# Create points

eff_frontier_rdm <- matrix(0, nrow = 1000000, ncol = 2)

for(i in 1:ncol(w_rdm)){
  
  eff_frontier_rdm[i,1] <- t(w_rdm[,i, drop = F]) %*% asset_returns
  
  eff_frontier_rdm[i,2] <- t(w_rdm[,i, drop = F]) %*% Sigma %*% w_rdm[,i, drop = F]
  
}

colnames(eff_frontier_rdm) <- c("return", "variance")

eff_frontier_rdm <- eff_frontier_rdm %>% as_tibble()

# smallest variance achieved with random portoflios

min(eff_frontier_rdm$variance)

# Computing the minimum variance portfolio

lambda <- 2 / as.numeric((t(ones) %*% solve(Sigma) %*% ones))

w_mvp <- (solve(Sigma) %*% ones) * lambda/2

Sigma_mvp <- t(w_mvp) %*% Sigma %*% w_mvp

# theoretical min variance portoflio

Sigma_mvp
```

I have added a picture of the simulated portfolios with the code below, with the theoretically correct efficient frontier.

Based on the answers, i managed to create the following

Thanks for the different suggestions. Based on the above answers, I resolved my issue by the following procedure:

Given some covariance matrix Σ, and an expected return vector, I call asset_returns, I used the following steps:

Use the two mutual fund theorem to choose some random weight matrix, from a portfolio located on the frontier. Add some noise to each weight, by drawing from a standard normal. Normalize the created vector by dividing with the sum. Compute the portfolio variance and expected return with the created weight vector. After creating 1 million of such random weights, I manage to fill the area within the frontier. Creating more points would fill it all out.

The solution is based on the answers I got here on the post.

```
eff_frontier_rdm <- matrix(0, nrow = 1000000, ncol = 2)

for(i in 1:nrow(eff_frontier_rdm)){
  
  c <- runif(1, min = -4, max = 4) # draw random number
  
  w = c * w_mvp + (1-c) * port_2$w_eff # create weight
  
  eps <- matrix(rnorm(N, mean = 0, sd = 0.1), ncol = 1)
  
  w = w + eps
  
  w = w / sum(w)
  
  eff_frontier_rdm[i,1] <- t(w) %*% asset_returns
  
  eff_frontier_rdm[i,2] <- t(w) %*% Sigma %*% w
  
}
```

## Answer by nbbo2 (score 0, accepted)

https://quant.stackexchange.com/a/61248

I think I understand your issue. Your weights are not uniformly distributed over the desired space. Instead you weights are bunched together, there are fewer extreme weights (weights close to 1) than there should be.

For example let's consider how often the 1st weight (or any other weight for that matter) should be between 0.8 and 1.0. Since the length of the interval [0.8,1.0] is one tenth of the interval [-1.0,1.0], ten percent of generated vectors should have a first weight between 0.8 and 1.0. (Agreed?). If you test your randomly generated vectors I think you will find that far far less than 10% of your first weights are in this range (perhaps none at all).

How can this be, you might ask, given that in the first step you generate weight randomly between -1 and 1. The problem comes in the second step, where you divide by the sum of weights. As a Japanese saying goes "The longer a nail is, the more it gets hammered down". If one (or more) random weights are close to 1, the sum of weights will be large and the division will "hammer down" the big weight accordingly. Result: very few big weights. But don't trust my intuition, check your generated data.

What is the solution? How do you generate portfolio weights uniformly? The Patrick Burns web site linked by Mr. Schumann may be helpful. Personally I discovered this issue in a context where I had to generate weights between 0 and 1. I found this CS StackExchange question helpful https://cs.stackexchange.com/questions/3227/uniform-sampling-from-a-simplex. I even wrote up one of the algorithms here Random Portfolios vs Efficient Frontier (I think the pictures in this link will look familiar to you, too few points close to the frontier). Your problem is a little different, since you allow short positions so your weights have to be between -1 an +1. The algorithms will have to be modified accordingly.

My thought on this is that, since a long/short portfolio consists of a long portfolio and a short portfolio, you could generate the positive and the negative weights separately. But I haven't finalized exactly how this would work. I hope my remarks above help somewhat.

## Answer by Enrico Schumann (score 1)

https://quant.stackexchange.com/a/61245

It's normal that it takes very long to come close to the efficient frontier with random portfolios.

How close you come how fast will be strongly influenced by how you sample the portfolios. In your code, you sample uniformly. You may want to look at the weight distributions of the portfolios on the frontier, and then consider how likely it is that you arrive at such portfolios through your sampling procedure.

Patrick Burns (https://www.burns-stat.com/, https://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=59330 ) has written lots of insightful stuff about random portfolios; also about how to construct them.

## Answer by Nick (score 0)

https://quant.stackexchange.com/a/61230

Why did you expect to hit the minimum variance portfolio in your simulations?

There are infinitely many vectors of size Nx1, such that the sum of its elements adds up to 1. Even with 1 million simulations, it is very difficult to hit the exact minimum variance allocation vector.

Moreover, your simulations might have hit some allocation vector "very near" to the minimum variance one (very near meaning a low value of the norm of the vector difference). If your covariance matrix is ill-conditioned, then the allocation vector that is "very near" to the minimum variance one might result in a simulated portfolio with a significantly higher variance than the optimal one.

## Answer by demully (score 0)

https://quant.stackexchange.com/a/61235

Not sure why you would seek to hit this via trial-and-error sampling in the first place.

By definition, the MinVar portfolio is the tangency / MaxSharpe portfolio if all the returns are assumed to be constant/equal. See pp.7-10 in e.g. https://faculty.washington.edu/ezivot/econ424/portfolioTheoryMatrix.pdf