Lessons on Backtest Bias, Live Trading Frictions, and Strategy Allocation
Summary
This retrospective contrasts rule-based stock selection with machine-learning ranking and describes backtesting as a way to evaluate a strategy on historical market data. Its central caution is that a strong fit on a small sample can reflect an irrelevant feature rather than a durable relationship. The author recommends testing across more periods and market conditions, including unusual rises and falls, and treating historical curves as preliminary evidence rather than proof of future performance.
The account also identifies practical gaps between simulated and live results: slippage, missed orders, discretionary departures from exit signals, and day-to-day execution constraints. It argues that strategies can behave differently across market regimes and discusses combining defensive and momentum-oriented approaches with varying allocations. The examples are personal experience, not a controlled study; reported returns lack enough context to assess comparability, and discretionary allocation can introduce its own judgment risk. The piece is useful chiefly as a set of validation and implementation cautions.
Key ideas
- A backtest can appear accurate when a feature fits a small sample by coincidence.
- Strategies should be evaluated across multiple periods and market conditions, including unusual moves.
- Slippage, missed trades, and deviations from exit rules can make live results differ from simulated results.
- Different strategies may suit different market regimes, but combining them and adjusting allocations requires judgment.
- The reported experiences are anecdotal and do not establish general performance expectations.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.