Skip to content
All library documents

High-Frequency Methods for Estimating and Forecasting Equity Volatility

Article Quant Q&A · Author: Tal Fishman
Read the original

Summary

The document asks how to forecast volatility across a large cross-section of equities from high-frequency returns. The accepted response says that, at the time discussed, a mature theory for forecasting cross-sectional realized volatility was lacking; much of the research focus remained on estimation. The challenges include irregularly spaced transaction data, market microstructure noise, asynchronous trading, high dimensionality, and covariance estimation issues.

Two estimation approaches are highlighted: pre- and post-averaging methods, and kernel smoothing. The answer identifies multivariate realized kernels as a more developed published technology among these approaches, while conventional VAR methods are described as less central in the cited work. A separate answer points to blocking and regularization for high-dimensional realized covariance estimation: assets are grouped by liquidity, estimated block-wise with realized kernels, then regularized. The cited work reports simulation and index-universe applications, but the post does not establish that this method solves the distinct problem of forecasting volatility across all equities.

Key ideas

  • High-frequency cross-sectional volatility forecasting is more difficult than estimating realized volatility and covariance.
  • Irregular observations, market microstructure noise, asynchronous trading, and high dimensionality complicate estimation.
  • Pre- and post-averaging and kernel smoothing are presented as estimation approaches.
  • Multivariate realized kernels provide a developed method for high-frequency covariance estimation.
  • Blocking by liquidity and regularization are proposed for high-dimensional covariance matrices, but forecasting remains a separate challenge.

Tags

EquitiesVolatilityStatisticsMachine learning
Full text
# How to forecast expected volatility from high-frequency equity panel data?


# How to forecast expected volatility from high-frequency equity panel data?












I'm wading through the vast sea of literature on realized volatility estimation and expected volatility forecasting (see, e.g. Realized Volatility by Andersen and Benzoni, which cites 120 other papers, and Volatility by Bandi and Russell, which cites a slightly overlapping set of 120 papers).

I'm having a tough time finding research that specifically addresses the simultaneous estimation of a broad cross-section of equity volatility from high-frequency returns time-series. I'm looking for something along the lines of Vector Autoregression (VAR), but applying both sophisticated techniques developed for large equity panel estimation (thousands of volatilities and potentially millions of correlations being estimated) and using recent advances developed for efficient estimation using high frequency data.

What papers address the specific problem of forecasting the cross-section of equity volatility from high frequency data?

## Answer by Ryogi (score 7, accepted)

https://quant.stackexchange.com/a/2174

As far as I know the short answer is negative: there isn't a well developed theory of how to forecast cross-sectional realized volatility. From the perspective of statistics/econometrics, most of the recent research is still trying to find its way around estimation of cross-sectional realized volatility, and so far even in these area the progress is slow.

Bringing modern techniques to panel data amounts to being able to:

- extract information irregularly spaced transaction data (UHFT or tick level),

- deal with "microstructure noise",

in a multivariate setting where the additional problems of non-synchronous trading and high-dimensionality complicates the analysis, together with the usual hassle that comes with HAC estimators (such as the dimensionality issues that @QuantGuy mentions).

There are two main tools for tackling estimation: (1) the pre/post averaging approach of Ait-Sahalia, Mykland, Renault (and others) and (2) the kernel smoothing of Barndorff-Nielsen, Hansen (and others) [the third child, i.e. VAR and its crew, seems on the sideline of late, but I'd be happy to be proven wrong here]. Of these two approaches, only the second has matured a technology (Multivariate realised kernels) that is published (here).

## Answer by Ram Ahluwalia (score 5)

https://quant.stackexchange.com/a/2181

Check out A Blocking and Regularization Approach to High Dimensional Realized Covariance Estimation.

Abstract:

> We introduce a blocking and regularization approach to estimate high-dimensional covariances using high-frequency data. Assets are first grouped according to liquidity. Using the multivariate realized kernel estimator of Barndorff-Nielsen et al. (2010), the covariance matrix is estimated block-wise and then regularized. The performance of the resulting blocking and regularization (‘RnB’) estimator is analyzed in an extensive simulation study mimicking the liquidity and market microstructure features of the S&P 1500 universe. The RnB estimator yields efficiency gains for varying liquidity settings, noise-to-signal ratios and dimensions. An empirical application of estimating daily covariances of the S&P 500 index confirms the simulation results.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.