Signal Timing, Return Labels, and Tradability in Short-Horizon Prediction
Summary
The document examines an hourly machine-learning signal that predicts the direction of returns measured from an interval’s average price. The key timing issue is that the signal becomes available only after that interval ends, so a backtest that assumes entry at the interval’s average price may use an unavailable execution price. The response argues that trading may still be possible at a later executable price, such as a following interval’s VWAP, while accounting for slippage.
It also explains why predictions for returns sharing an earlier price anchor can appear strong while the next interval’s return prediction fails. If features and the anchor price both incorporate overlapping past observations, the target may contain overlap or information leakage that inflates apparent accuracy. Alternative labels, such as close-to-close returns, can help examine the issue. The response considers a one-hour directional forecast from past candle data difficult in an efficient market and says higher-frequency data might help, with added processing cost. No out-of-sample trading results or execution-cost analysis are provided, so the reported directional accuracy alone does not establish profitability.
Key ideas
- A signal computed at an interval’s end cannot be traded at a price from earlier in that interval.
- Backtests should use a plausible post-signal execution price and account for slippage.
- Overlapping historical data in features and return labels can make prediction accuracy look misleadingly strong.
- Changing the return label can help reveal whether apparent predictability comes from a shared price anchor.
- Directional accuracy alone does not demonstrate a profitable strategy after execution costs.
Tags
Full text
# Signal update frequency and predicting directional return one step ahead
# Signal update frequency and predicting directional return one step ahead
I am trying to get some insights on this specific sort of problem from experienced people, as I do not have much experience in this field.
I have a family of features that for simplicity I will just denote it by $S_{i-1}$ (we can think of it as a single feature.). Here $i-1$ means that this feature/signal corresponds to the time interval $[t+(i-1)d, t+id]$. Here $t$ is a fixed starting time and $d$ is the length of the each interval that our data gets updated. Note that $S_{i-1}$ becomes available at the end of the aforementioned interval i.e. at time $t+id$.
Let's denote the average price of an asset over the interval $[t+(i-1)d, t+id]$ by $P_{i-1}$. I have built an ML model that given the signals $S_{i-1}$ can predict $ln(P_i/P_{i-1})$. Note that the actual errors of the predictor can be high but the model predicts the direction (the sign of the return) correctly with a high accuracy (let's say $70$ percent).
Note that this prediction is not tradable, because $S_{i-1}$ becomes available at the end of the interval at a time that is too late to take positions (we should have started to take positions, short or long during $[t+(i-1)d, t+id]$ ).
Note that the same model is capable of predicting the direction of $ln(P_{i+1}/P_{i-1})$ with a very high accuracy as well (this is not tradable as well) but when trying to predict $ln(P_{i+1}/P_{i})$ everything breaks down and model completely loses its capability. Note that this prediction if it had worked would have been tradable. We could have started to take positions in the interval $[t+id, t+(i+1)d]$ and potentially get out of the position in the interval $[t+(i+1)d, t+(i+2)d]$.
So my question is, are these kinds of signals common when trying to predict the returns? (or am I fooling myself to think there is something special happening here) My hypothesis was that if I can manage to get high resolution data and work in higher frequency it might start to work but getting higher frequency data won't be easy, so my intention was to see whether these types of signals have tendency to work in higher frequencies or not. My current frequency is 1 hour.
## Answer by autoencoder (score 1)
https://quant.stackexchange.com/a/80232
Posting as an answer here for better readability.
Given what you mentioned in the comments, here's a simple example of your context:
- K denotes a 15min candle
- P denotes the price for return calculation, P is the average of OHLC of four candles
- S denotes the feature set, which might also involve previous candles
As time goes by, one gets the following:
t1: {K11, K12, K13, K14}, S1, P1;
t2: {K21, K22, K23, K24}, S2, P2;
t3: {K31, K32, K33, K34}, S3, P3; ...
At time t1, we gathered market data and calculated feature S1, and we can get a prediction for label ln(P2/P1) using S1. Now OP said the signal is not tradable, but I would argue that it is. However we won't be able to trade the asset at price P1, or the open price of K21, but we can often assume that we could trade at the VWAP of K21, or a VWAP of the next several candles. We will have slippages but we'll be able to trade.
Now, the other problem OP has is that, prediction for ln(P2/P1) is good, prediction for ln(P3/P1) is good, but the prediction for ln(P3/P2) is terrible. The problem is that, the P1 part in label ln(Pn/P1), is calculated using too much past information. Note that P1 is the mean of four OHLCs in the past hour, and although OP didn't mention his/her feature set, I would guess S1 are also features derived from past price/volume data. So there could be large overlap/info leak in the label. A simple adjustment would be to use close to close return as labels, or, as the OP suggests, use ln(P3/P2) as the label.
And we come to the last question: prediction for ln(P3/P2) is bad, which in my opinion is normal. One hour prediction horizon is acutally pretty challenging, if ones tries to do it using just past candle data. Although I have no clue what market/asset class OP is trading at, the market should be decently efficient nowadays. Adding higher frequency data would help, but it again requires more cost to process it.Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.