Estimating Tracking Error Uncertainty with Bootstrap Resampling

# N Needed for Statistically significant tracking error


# N Needed for Statistically significant tracking error












Let's say I have the tracking error calculation for a portfolio:

How would I determine the N-obervations required for a statistically significant tracking error? Alternatively, how would I determine if the tracking error itself were statistically significant?

Some illustrative code here

```
import statsmodels.stats.moment_helpers as mh
import pandas as pd
import numpy as np
def generate_correlated_random_return_matrix(annual_means, annual_vols, corr, t_periods, n_samples, period_adjust=12.):
    """
    Generates a return matrix from a multivariate random normal distribution.

    **Args**:
        *annual_means*: An array of mean annual returns.

        *annual_vols*: An array of annual vols.

        *corr*: Correlation matrix. An example being:
            >>> [[1,0],[0,1]]

        *t_periods*: How many months would you like to simulate?

        n_samples**: How many times do you want to run this simulation?
    """
    means = np.divide(annual_means, period_adjust)

    vols = np.divide(annual_vols, period_adjust ** .5)

    cov = np.asmatrix(mh.corr2cov(corr, vols), float)

    sim_array = np.random.multivariate_normal(means, cov, [n_samples, t_periods])

    return sim_array

te_tests = generate_correlated_random_return_matrix(annual_means=[.03,.03],annual_vols=[.1,.1],corr=[[1,.8],[.8,1]],t_periods=10000,n_samples=1)

df = pd.DataFrame(te_tests[0])

expanding_te = pd.expanding_std(df[0] - df[1])

mu = (df[0] - df[1]).std()

true_te = (df[0] - df[1]).std()

vol_of_expanding_TE = expanding_te.std()

z_score_of_TE_at_obs_N = ((expanding_te - true_te)/vol_of_expanding_TE).plot()
```

This I guess would give me a way to state the "measured TE is statistically indistinguishable from the `TRUE` TE", I suppose. Unsure if what I am using as standard deviation is correct, though.

## Answer by Attack68 (score 2, accepted)

https://quant.stackexchange.com/a/50990

Suppose you have a portfolio that has some unknown real mean tracking error, $t$, relative to the benchmark, with some real variance $\sigma_t^2$.

You have a sampling process generated from your data that determines the estimated mean tracking error, $\hat{t}$ according to your formula, and of course you can derive an estimated variance in the tracking error also, $\hat{\sigma_t^2}$.

How can you assess the confidence in your dataset relative to the unknown real values?

1) You can run a non-parametric bootstrap sample: In this method you create multiple bootstrap sample datasets via resampling (with replacement) of the tracking errors from your dataset. Then you derive a confidence interval from the statistics of the bootstrap estimators.

As an example suppose you have 5 datapoints, 5 days worth of tracking errors: [ 1, 2, 1, 3, 20 ], has a mean of 5.4 and variance of 67. How accurate or misleading might these estimators be? I perform 100 bootstrap samples (with replacement) and the resulting distribution I get of mean estimators is depicted:

Because of the small number of samples (and potential outlier) this is quite dramatic. I would suggest you cannot confidently assess you have an accurate estimator of the mean tracking error at 5.4. However with much larger N I think you will derive a fairly confident value.

For example, suppose I extend the dataset of tracking errors to 20 datapoints: [1, 2, 1, 3, 20, 5, 3, 2, 8, 9, 4, 4, 7, 16, 2, 2, 2, 7] has a mean of 5.4. Now 100 bootstrap samples will yield the following distribution of mean estimators:

2) You can do the same process for a parametric bootstrap sample: where you assume the tracking error has some underlying distribution, and you can also do this for the variance estimator also.

In your case this might be more appropriate, since you have an underlying multivariate normal distribution.

In this case I would rephrase the question to what is the width of range of TE estimator values that is statistically significant given N takes different values.

For example suppose I create 20 parametric bootstrap samples for the cases where N is 3, 6, or 24 in each sample. I have a simple benchmark portfolio with weights [1,1] and a tracking portfolio of [0.9, 1.1]. I have simulated market movements with simple uniform distribution and no correlation and calculated the TE according to your formula. The distribution of TE estimators that I got looked like the following:

Clearly as N increases you have less variance in the estimators distributions and by performing this kind of analysis you can reference definitive confidence intervals that you are comfortable to work with. I.e. in this case it is statistically unlikely that the tracking error will be +- 0.2 from true value with N=24, but with N=3 there is a reasonable chance that will occur.

### These methods form part of the field of computationally intensive statistics.