Why Negative Out-of-Sample R² Can Coexist with Profitable Trading
Summary
The discussion explains how a forecasting model can have negative out-of-sample R² while still supporting profitable long-short positions. If the forecast ranks assets well but misestimates the overall return level, the squared-error measure can be poor even as a portfolio earns gains from the spread between selected long and short returns. A shift in the unconditional mean between training and test periods can contribute to this mismatch.
It also emphasizes that R² is only one summary of predictive performance. A metric based on average fit may be poorly aligned with a strategy whose value depends on correctly identifying a rare but consequential outcome. The examples are conceptual; the discussion does not establish that any particular forecast is robust or profitable after costs. Performance evaluation should therefore reflect the trading objective and portfolio construction, alongside standard statistical measures.
Key ideas
- A model can rank returns usefully even when its level forecasts are biased.
- A shift in the out-of-sample mean can produce negative R² without eliminating long-short portfolio gains.
- Long-short profit depends on return spreads, while R² summarizes forecast errors relative to a benchmark.
- Forecast metrics should be selected to match the economic objective of the strategy.
- A favorable metric or illustrative example alone does not establish robust profitability.
Tags
Full text
# Model is economically significant, but has negative $R^2$? # Model is economically significant, but has negative $R^2$? I'm reading a paper by Rama Cont that says (Page 25): > We remark that negative R^2 values do not imply that the forecasts are economically meaningless. To emphasize this point, we will incorporate these return forecasts into two common forecast-based trading strategies, and showcase their profitability... The authors proceed to do just that, showing that a strategy that takes forecast-weighted positions in the stocks under considerations earns economically meaningful profit, despite the forecast model having an OOS r^2 of -0.11 to -0.18. Intuitively, how can a forecast with a negative r^2 generate positive returns? ## Answer by Thomas Johnson (score 1, accepted) https://quant.stackexchange.com/a/74110 After thinking about this problem more, I believe I understand what drives this behavior of a negative R^2 and positive PnL. Suppose that you are trading a long-short portfolio (as in the paper), and your out-of-sample data has an unconditional mean that is very different from your in-sample. But, suppose your forecast still ranks the OOS returns accurately while being wrong about the mean. Then your OOS r^2 might be negative, but you don't care because your long-short portfolio pnl is driven by the spread of the returns, not their mean. I've put together a Colab notebook demonstrating this behavior: https://colab.research.google.com/drive/17NHxfxTnO3eEM0UC9j7zkIGNKoHUkXd2?usp=sharing ## Answer by g g (score 2) https://quant.stackexchange.com/a/74104 You ask in a very general way, which makes it hard to pin down without studying the paper since there are plenty of possibilities. Just stating three here: - In the specific setup all returns are positive - The example is cherry-picked. - For the example $r^2$ is an irrelevant indicator. I'll just discuss option 3. because this may be relevant to a larger audience. $r^2$ is just one possible summary of the full distribution of outcomes. It may miss totally what is relevant or as expressed by Cont economically meaningful in a specific situation. To give an intuitive explanation in a simplified setting: Imagine you predict binary outcomes (say 0 being everything is fine and 1 being all hell breaks loose) and the probability of outcome "1" is very low. Then the prediction "always 0" will be very good as measured by all indicators relying on some notion of average fit. But imagine in your specific setup that getting the "1" right is much more desirable than erring in a "0" situation. In that case a strategy may be "good" (predict some "1"s correctly) even though it may mispredict many "0"s on average. tl;dr: Your quality measure needs to be aligned with your interests other measures can be ignored.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.