Why Trading Research Needs Mechanisms Alongside Backtests
Summary
The article argues that using large language models to generate strategies and run backtests can accelerate technical work while leaving the trader without an understanding of why an opportunity might persist. It frames sound research as a cycle of forming a hypothesis, testing it, learning from the evidence, and refining the next question. A central test is to identify the counterparties on the other side of a trade and the constraints that might lead them to keep trading there.
The author proposes that a credible edge needs both a plausible market mechanism and supporting data. Backtests can help assess evidence, but parameter tuning alone risks fitting historical patterns; an appealing explanation without data is also insufficient. Understanding the mechanism can help a trader judge whether a drawdown reflects normal variation or a change in the market structure. The article is an opinionated critique rather than an empirical comparison of LLM-assisted and conventional research, and it acknowledges that LLMs can still help with data work, coding, experiments, and interpretation.
Key ideas
- A trading hypothesis should explain who is on the other side and what constraints may sustain the opportunity.
- Backtesting can assess evidence but does not establish a market mechanism by itself.
- A plausible mechanism and supportive data are both needed to build confidence in an edge.
- Repeated research should develop the trader's judgment, rather than only produce more strategies.
- Understanding the mechanism can help assess whether a strategy's drawdown reflects changed market structure.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.