Why High-Frequency Equity Return Autocorrelation Changes Over Time

# Memory effect of log returns of S&P 500


# Memory effect of log returns of S&P 500












I am trying to reproduce the analysis discussed in https://arxiv.org/pdf/cond-mat/9905305.pdf where they use high-frequency data (1-minute frequency) of S&P500 from 1984 to 1996. In particular, they found the log returns for this period follow an exponential decay behaviour for the first 20 minutes with a characteristic decay time of 4 minutes, see figure below (that is the Fig.3 of the paper)

I don't have the data of the period from 1984 to 1996, but I have downloaded data from 2010 to 2021 (here the link) and I have tried to perform the same analysis with these data, but I find a completely different behaviour (see Figure below).

Question: Why am I not seeing the exponential decay? Do I really need more data?

The python code that I use to generate this plot is the following:

```
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

def multiplyLag(series,lag):
    return series * series.shift(periods=lag)

df = pd.read_csv('SPX.csv',parse_dates=['time'],index_col='time',skipinitialspace=True)
df=df.reset_index()
df['time'] = pd.to_datetime(df['time'],utc=True)
df=df.set_index('time')
df=df.sort_values('time')

logDataSet=np.log(df)
logDataSetDiff = -logDataSet.diff(periods=-1)
denominator = multiplyLag(logDataSetDiff,0).mean()['4']-((logDataSetDiff).mean())['4']**2
y=[]
for i in range(1,50):
    print(i,end='\r')
    y.append(1/denominator*(multiplyLag(logDataSetDiff,i).mean()['4'] - ((logDataSetDiff).mean())['4']**2))
    
x=range(1,len(y)+1)
plt.scatter(x, y)
plt.show()
```

and the file `SPX.csv` that I am using can be downloaded here for convenience.

## Answer by Pleb (score 3, accepted)

https://quant.stackexchange.com/a/69182

### You will not be able to replicate positive decaying autocorrelations in current markets:

Due to the increased popularity in high-frequency trading, there's been a likewise increased focus on studying the underlying empirical properties of high-frequency data. One of the stylized facts in high-frequency data is the significant first-order negative autocorrelation of the log-returns — which you have rightfully captured in your `Python` generated autocorrelation plot — that is said to arise from non-synchronous trading and bid-ask bouncing, as seen from the perspective of Econometricians [1] [2] [3].

There's been a good amount of literature describing this fact for the past 25 years:

- Aït-Sahalia, Y., Mykland, P. A., & Zhang, L. (2011), show in their preliminary empirical study how 30 different US stocks on average exhibit first-order negative autocorrelation. They find strong evidence of the negative autocorrelation at horizons up to about 15 transactions (they are using transaction-based tick data).

- Zhou (1996) find the same fact under tick-by-tick exchange-rate returns and moreover Bandi, F. M., & Russell, J. R. (2008) and Hautsch, N., & Podolskij, M. (2013) provide evidence of first-order negative autocorrelations under second- and transaction-based frequencies for US-based stock returns.

Therefore your second depicted plot, is more in line with current empirical literature than the study you're trying to replicate. Also, increasing the time-horizon or frequency of your current data will not increase the chances of producing positive decaying autocorrelations.

### Why don't you experience the same decay of positive autocorrelations?

- Different data-cleaning procedure: Even if you were to obtain SPX minute data from 1985 via your own data-vendor, you could still end up with slightly different results, since your data-vendor might use different cleaning rules, than the ones used by the authors (or their corresponding data-vendors).

- The underlying characteristics has changed (as also described by the above comment): The underlying characteristics of the SPX index has changed since the 1990s, which can be attributed to high-frequency trading becoming more predominant in todays markets. Moreover, the constituents of the S&P 500 index have likely changed within this time-span. One of the cited papers on high-frequency returns exhibiting positive autocorrelations was Cutler et al. (1991). They showed how autocorrelations became more positive as the sampling frequency increased, even though the authors never used higher frequencies than a daily sampling rate. However, they specifically argue that different time-horizons and data constructions can affect the sign of the autocorrelations: While these findings appear in many markets, they are not universal. Schwert (1989), for example, finds negative serial correlation in daily U.S. stock returns before 1917. An obvious issue for future research is whether differences in data construction, or variation in market structure through time or across markets, can explain some of the differences in the stochastic properties of returns.

In conclusion, it is unlikely that you would get the same results using the minute-returns of SPX from 2010 and beyond.

The best way for you to reproduce the results of the article: Ask the authors for the source of their data and if they have used a cleaning procedure. Or buy/get pre-cleaned SPX data in the time interval 1984 to 1996 from your own data-vendor and try to reproduce the results.

$\:$ Articles commenting on the first-order negative autocorrelation:

[1]: Andersen, T. G., & Bollerslev, T. (1997). Heterogeneous information arrivals and return volatility dynamics: Uncovering the long‐run in high frequency returns. The journal of Finance, 52(3), 975-1005.

[2]: Koutmos, G. (1997). Feedback trading and the autocorrelation pattern of stock returns: further empirical evidence. Journal of international money and finance, 16(4), 625-636.

[3]: Roll, R. (1984). A simple implicit measure of the effective bid‐ask spread in an efficient market. The Journal of finance, 39(4), 1127-1139.