Intraday Backtests and the Risk of Selective Shutdowns
Summary
The document asks whether backtests of intraday strategies that start and end each day flat understate performance because a trader could stop trading after recognizing a poor market day. It notes that backtest results depend on assumptions about transaction costs, market impact, and order timing, as well as strategy parameters. The proposed intervention is to exit positions and disable trading for the rest of a day judged unfavorable.
The author observes that poor results may cluster on particular days rather than in particular stocks, but gives no tested shutdown rule, evidence, or procedure for deciding when to stop. A historical backtest can model a shutdown only if its trigger is defined using information available at that time; choosing bad days after seeing outcomes would introduce hindsight and inflate results. The discussion is an open question rather than a validated method, and it leaves execution costs, trigger design, and the effect of missed profitable trades unresolved.
Key ideas
- Backtest results depend on assumptions about costs, market impact, and order execution.
- The author suggests that intraday performance may vary more by market day than by individual stock.
- A shutdown rule needs an explicit trigger that uses only information available at the time.
- Selecting days for shutdown after observing their outcomes risks hindsight bias.
- The document raises the idea but provides no tested rule or performance evidence.
Tags
Full text
# Seemingly under-estimated backtest performance of intraday trading
# Seemingly under-estimated backtest performance of intraday trading
Suppose I have an intraday strategy that is flat at the beginning of the day and flat at the end of the day. I run the backtest and generates thousands and thousands of trades over say one year. Obviously, many parameters-decisions have to be made for the backtest:
A transaction cost estimate ( entering and exiting ) number in basis points needs to be used of course. Also, vague approximations of how much can be bought while balancing market impact. Things like, how long does it take to get in and at what price. Similarly, how long does it take to get out and at what price ? The whole thing is just one big vague estimate. No doubt about it.
And then there are all the parameters associated with the trading strategy itself. thresholds, flags, Z-scores, stop-loss amts, volatility estimates and all sorts of other things. If you get a poorly performing backtest, you can probably change enough things to eventually make it look good. At the same time, if you don't try to trick yourself, you can tell by specific statistcs whether the thing is working or not. Anyway, that's getting off the track.
The one thing that's special about intraday in my eyes is that one can get flat during bad market periods by shutting the whole process down for the day. At whatever time XXX that it looks like it's gonna be a "non-noise filled day", get out of everything and enforce that there is no more trading for the day. This would improve performance of a backtest because I can see, by looking at it, that a lot of times its poor days rather than poor specific stocks.
Clearly, there's no way to put the "shut it off" ability in a backtest so it seems like the backtest performance will be consistently under-estimated. Am I missing something here ? Thanks for any wisdom or references. I've never run an intraday trading strategy in real life so I could definitely use any kind of wisdom.
```
Mark
```Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.