Evaluating Parameter-Uncertainty Methods with Forecast Errors

Summary

The document considers how to compare approaches intended to address parameter uncertainty in portfolio optimization, including Bayesian methods with diffuse, asset-pricing-based, or economic-objective priors. Its response recommends evaluating forecasts with mean squared forecast error or another suitable forecast metric. For a parameter that can be observed later, forecasts can be tested on a rolling basis against realized values, and historical forecast errors can be used to form confidence intervals.

For model assessment, the response also suggests comparing rolling forecast errors with those of an ideal benchmark that has perfect information. This shifts the comparison from debating broad theoretical categories to measuring predictive performance. The advice is concise and does not specify a portfolio objective, define the parameters or benchmark construction, or address transaction costs and downstream allocation outcomes. Forecast accuracy can inform uncertainty assessment, but it does not by itself establish that a method produces better realized portfolio performance.

Key ideas

• Parameter-uncertainty approaches can be compared using mean squared forecast error or another evaluation metric.
• Rolling forecasts can be checked against subsequently observed parameter values.
• Past forecast errors can inform confidence intervals without relying solely on an assumed distribution.
• Model errors can be compared with an idealized benchmark that has perfect information.
• Predictive accuracy alone does not establish superior portfolio outcomes.

Tags

# Critical Appraisal of Approaches countering Parameter Uncertainty in Portfolio Optimization


# Critical Appraisal of Approaches countering Parameter Uncertainty in Portfolio Optimization












It is very hard to come up with legit and solid advantages and drawbacks of the various approaches wich are trying to counteract parameter uncertainty in portfolio optimization procedures. In my opinion all bayesian approaches, i.e. the use of diffuse priors, asset pricing based priors and economic objective priors are somewhat similar and may or may not lead to favorable results. How would you assess the advantages and disadvantages of each approach in practice?

## Answer by Kyle Balkissoon (score 1)

https://quant.stackexchange.com/a/15880

Use Mean Squared Forecast Error (or any other forecast evaluation metric).

Your question appears to complicate the problem: If your goal is to forecast a given parameter you can test the rolling forecast against the actual observed values. This will also give you a metric of uncertainty as you can then create confidence intervals around your forecasts based on past error (as opposed of using distributional estimation and it's assumptions).

If you want to test models you can check the forecast error of the model vs optimal with perfect information on a rolling standpoint.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.