Choosing Fit Metrics for Return Replication Models

Summary

The document considers how to measure the fit of a portfolio or linear combination of assets intended to reproduce another asset’s returns. It notes a weakness of relative error: dividing by the target return becomes unstable when that return is near zero. It also expresses a preference for a measure that penalizes predicting the wrong return direction more heavily than a miss in magnitude.

The response points to mean squared error, R-squared, and adjusted R-squared as standard fit measures, and names Mallows’ Cp, AIC, and BIC as criteria that account for model complexity and help guard against overfitting. It does not define these measures, compare their behavior on return data, or propose a specific sign-sensitive loss. The guidance is therefore a starting point rather than a complete solution; practitioners still need to choose a metric suited to their objective and validate it on relevant data.

Key ideas

• Relative error can become distorted when the target return is close to zero.
• Mean squared error and R-squared are common measures for assessing return replication fit.
• Adjusted R-squared and information criteria account for model complexity when evaluating fit.
• The response does not supply a metric that explicitly penalizes sign errors more heavily.

Tags

# goodness of fit metric


# goodness of fit metric












I am trying to approximate the returns of asset A by means of a linear combination of other assets A'=aB0+bB1+c*B2....

I have this quite figured out but I'm not sure what a good metric for goodness of fit would be, so far I am only considering relative error (e=(rA-rA')/rA), and I'm concerned with distortions when rA is close to 0.

What would a better metric could be? Ideally it would penalize sign errors more than absolue value errors (ie, it is worse that rA' is positive when rA is negative).

## Answer by Magic is in the chain (score 2)

https://quant.stackexchange.com/a/42108

Please look up goodness of fit measures such as MSE (mean squared error) , R-squared , and adjusted R squared. There are also a number of others measures that have been developed to penalise complex model to avoid overfitting. These include mallow $C_P$, AIC, and BIC. This note would be a good start:

https://people.duke.edu/~rnau/compare.htm

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.