Why Accurate Forecasts Can Fail to Produce Profitable Trades
Summary
The document explains why predictive accuracy alone does not establish that a trading strategy will earn money. Forecasts and backtests can omit execution costs, commissions, order-book depth, bid and ask prices, and the effect of the strategy’s own orders on market supply and demand. A trade that appears attractive at historical prices may not have been filled at those prices or at the assumed size. As participation grows, other traders and market makers may also adapt, changing liquidity and the strategy’s expected returns.
The responses also criticize models that apply ordinary regression to rare extreme outcomes or weak historical data, where tail-focused methods may be more appropriate. The discussion argues that market outcomes are path-dependent and strategic interactions matter, especially for large orders or less liquid securities. It offers conceptual cautions rather than a quantitative demonstration, and it does not specify a universal model or prove that any particular forecast will fail. A sufficiently accurate model that incorporates its own market impact could still support profitable trading.
Key ideas
- Forecast quality does not account for whether orders can be filled at assumed prices and sizes.
- Transaction costs, liquidity, and bid-ask information can turn an apparent edge into an unprofitable trade.
- A strategy’s own orders can affect market prices and alter its realized returns.
- Other market participants may adapt when a strategy becomes a regular source of trading activity.
- Models of extreme outcomes may need methods suited to tail behavior rather than central tendency.
Tags
Full text
# Why Good forecasting != Good trading? # Why Good forecasting != Good trading? In AI for algorithmic trading: 7 mistakes that could make me broke the author talks about why good forecasting cannot equal good trading based on the strategy selected. While reading the article I didn't quite understand his examples. Can anybody explain more? ## Answer by Dave Harris (score 2, accepted) https://quant.stackexchange.com/a/43997 The article probably could have used a bit of editing. Instead of talking about predictions, it should have talked about the impact of poor modeling decisions and data on predictions. Most predictions do not consider the impact of adding one more actor to the results. For example, if you backtest a set of trades, then you are ignoring the fact that for your trade to be included for purchases, then it would have had to outbid all the trades that actually happened. Likewise, to sell, you would have to underbid all the trades that actually happened. Adding a player to the system changes the system. Had your algorithm been in the system and shifted both the supply and the demand curve, then definitionally your return would have to be less than or equal to the existing returns. In addition, also not mentioned, is that if your algorithm becomes a regular feature of the market then the market-maker will adapt its inventory and liquidity position to your existence, at least for thinly traded securities. As with Long Term Capital Management, you could end up becoming the market and the illusion would be that your algorithm is working when in fact others are now playing off of it. Except for small players, the illusion is that events are independent, except that is mathematically impossible as there is strong path dependence in this math. I have noticed that most discussions of such algorithms ignore the game theoretic elements and are straight regressions with no sense of price elasticity, liquidity costs, or the profit maximization and risk transfer strategies of the market makers. I recently saw one of the worst algorithm design mechanisms that could be built. It attempted to maximize profit based on historical events. The historical profit-maximizing set of trades would have been purely random events. There would have been little parametric information to gather and even there it was captured wrong. The maximizing events would need to use extreme value theory not measures of central tendency, yet they used regressions looking for central tendency when they were looking at data sitting in the extrema. In doing this, they lost most hope of finding the drivers. I think a lot of models are bad because many people are ignoring first principles, collecting weak data, and using cookbooks that may not apply to their problems. The author is wrong about one thing though. If you had a perfect predictive tool that included the impact of your response, then you would make a lot of money. What the author is trying to convey is that people are not building those predictive algorithms. ## Answer by Iñaki Viggers (score 5) https://quant.stackexchange.com/a/43994 > Why Good forecasting != Good trading? I am not yet familiar with the F1 score the author compares with the Sharpe ratio. But the article rightly points out at least two grounds on which good forecasting does not imply good trading. The first issue has to do with the operative minutiae of trading. Models typically exclude aspects such as commissions, transaction costs, or any complications arising from market depth (I think this is what the author calls "bet sizing") once an order has been submitted. These operational hurdles can defeat the purpose of an otherwise profitable trade. The second concern I perceived from the article is the "need [for] bids and asks from the order book", traders' focus being typically the HOLC (high, open, low, and close) prices. Incorporating & modeling bids and asks is certainly complex, but their due consideration can make the difference between an order being timely filled and being pointless (even if the model is systematically accurate about forecasting an average price).
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.