Why Regression Metrics Can Favor Mean Forecasts in Trading

Summary

The article argues that common regression losses such as RMSE, MSE, and MAE can favor a forecast near the target’s historical mean. In noisy financial data, a model that predicts the average return may score well without learning a signal useful for trading. It extends the concern to classification, where always choosing the most common class can also appear effective under simple accuracy measures.

For evidence, the author describes comparing a constant historical-average forecast with a Ridge model using price data across markets, with time-series validation and a ten-day target horizon. The constant forecast had lower error in most of the tested markets. The article recommends evaluating predictive models against a mean baseline and relating error to total target variation, while ultimately judging models in terms relevant to trading outcomes. Its results are presented as one broker’s sample, and the excerpt does not fully show the experiment details or establish that alternative metrics will reliably identify profitable strategies.

Key ideas

• Regression error metrics can reward predictions close to the target mean, even when they provide little trading information.
• A constant average-return forecast is a useful baseline for evaluating more complex models.
• The article reports that this baseline achieved lower error across most of the tested markets.
• Classification systems can likewise appear skilled by repeatedly predicting the most common class.
• Trading model evaluation should consider decision usefulness and profit and loss, not error scores alone.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.