Optimizing a Sparse Basket to Track an Index

# Nearly replicate a basket with a few of its constituents


# Nearly replicate a basket with a few of its constituents












### Motivation

I have a basket with 30 constituents each with a weight which I want to nearly replicate with less than 30 trades for reducing trading costs.

### Better definition

- Better replication equals higher correlation

- I have an cost function relating how much correlation I am willing to forgo for lower trading costs

- Nominal dollar amount should be the same

- No negative weights

- You can only trade items of the basket. Nothing outside is allowed, no matter how correlated it is.

### What I have tried

- Linear regression with a predefined adhoc subset: breaks defs #3-4, and altough has higher historical correlation by definition, the weights don't seem robust for the future and look somewhat overfitted.

- Using lasso regression and tuning the lambda parameter for higher or lower number of selected items: still breaks defs #3-4, but the selection is no longer adhoc

- Linear regression restricted so the weights sum to 1 and no negative weights allowed on an adhoc subset: passes all the rules, however the estimation process seems finicky. minor differences in the starting coefficients leads to different ending weights. also sometimes the estimation does not converge

- Dropping the lowest weight constituents first, redistributing the weights: altough it works, it lacks formality to me, and could lead to a very suboptimal basket once many items are removed. Also the redistribution of the weights is up to debate. Is it done by weight or by relative correlation?

Also,I have not found any formal material (papers or books) on this subject and also if you could point me it would be immensely helpful

## Answer by Enrico Schumann (score 2, accepted)

https://quant.stackexchange.com/a/46779

You may want to look at the literature on index tracking; perhaps you find useful ideas there.

I would write and solve it as an optimisation model. Since you seem most interested in the number of assets that are required to closely replicate your basket (i.e. the cardinality of the replicating basket), you could solve the model for different cardinalities and then look at the trade-off correlation/cardinality.

Let me sketch how such an optimisation model could be handled in R, using a method called Threshold Accepting (implemented in the NMOF package). I use correlation, as you described, even though correlation may not be the best measure for closeness, since you may have baskets that are highly correlated but have very different volatilities.

```
n <- 30  ## number of assets in original basket
k <- 3   ## number of assets in replicating basket
```

The original basket: I assume equal weights, but just plug in other weights as desired.

```
w.basket <- rep(1/n, n)

library("NMOF")        ## https://github.com/enricoschumann/NMOF
library("neighbours")  ## https://github.com/enricoschumann/neighbours
```

I start by creating some random returns for your 30 assets. The result is a matrix `R` of size `ns` times `na`.

```
random_returns <- function(na, ns, sd, mean = 0, rho = 0) {
    ## na   = number of assets
    ## ns   = number of scenarios
    ## sd   = vol of returns
    ## mean = means of returns
    ##      ==> sd and mean may be scalars or
    ##          vectors of length na    

    ans <- rnorm(na*ns)
    dim(ans) <- c(na, ns)

    if (rho != 0) {
        C <- array(rho, dim = c(na, na)) 
        diag(C) <- 1
        ans <- t(chol(C)) %*% ans
    }
    ans <- ans*sd
    ans <- ans + mean
    t(ans)
}

R <- random_returns(na = n, ns = 250, 0.01, rho = 0.1)
```

Threshold Accepting is an optimisation method, so we need an objective function: the correlation between your basket and a replicating portfolio `x`. For simplicity, I also use equal weights for `x`, but only for the `k` included assets. All other weights are zero.

```
cor_basket <- function(x, R, w.basket, k, ...)
    -c(cor(R %*% (x/k), R %*% w.basket))
```

The best you can achieve is a correlation of 1. Since we follow the convention to minimise, we use minus the correlation.

```
cor_basket(w.basket, R, w.basket, k)
## [1] -1
```

Next, the key part of Threshold Accepting: the neighbourhood function. The neighbourhood takes a solution, makes a copy, modifies the copy slightly (and randomly), and returns this modified copy. In the case here, the function randomly selects on asset in the basket and replaces it with an asset that was not in the basket.

```
nb <- neighbourfun(type = "logical",
                   kmin = k,
                   kmax = k)

x0 <- c(rep(TRUE, k), rep(FALSE, n - k))
data.frame(x0, n1 = nb(x0), n2 = nb(x0))
##       x0     n1     n2
## 1   TRUE   TRUE  FALSE
## 2   TRUE  FALSE   TRUE
## 3   TRUE   TRUE   TRUE
## 4  FALSE  FALSE  FALSE
## 5  FALSE  FALSE  FALSE
## 6  FALSE  FALSE  FALSE
## 7  FALSE  FALSE  FALSE
## [...]
## 12 FALSE   TRUE  FALSE
## [...]
## 23 FALSE  FALSE   TRUE
## [...]

sol.ls <- TAopt(cor_basket,
                list(x0 = x0,
                     nI = 5000,
                     neighbour = nb),
                R = R,
                k = k,
                w.basket = w.basket)
## Threshold Accepting
##   [...]
##   Best solution overall: -0.6534397
```

Recall that we minimised: so the highest correlation with a tracking portfolio of three assets is 0.65. Now, to see how the trade-off correlation/number-of- assets looks like, simply run a loop.

```
for (k.i in 2:20) {
    x0 <- c(rep(TRUE, k.i), rep(FALSE, n - k.i))
    sol.ls <- TAopt(cor_basket,
                    list(x0 = x0,
                         nI = 5000,
                         neighbour = nb,
                         printDetail= FALSE,
                         printBar = FALSE),
                    R = R,
                    k = k.i,
                    w.basket = w.basket)
    message(format(k.i, width = 3),
            " | ",
            round(-sol.ls$OFvalue, 3))
}
##   2 | 0.566
##   3 | 0.653
##   4 | 0.721
##   5 | 0.765
##   6 | 0.8
##   7 | 0.833
##   8 | 0.86
##   9 | 0.876
##  10 | 0.89
##  11 | 0.9
##  12 | 0.913
##  13 | 0.921
##  14 | 0.931
##  15 | 0.938
##  16 | 0.946
##  17 | 0.95
##  18 | 0.955
##  19 | 0.96
##  20 | 0.965
```