Testing Simple Trading Strategies Against AI Traders More Thoroughly
Summary
The paper challenges published claims about AI and machine-learning trading agents. It argues that some earlier evaluations relied on a limited number of test sessions and market scenarios, leaving too little evidence to judge the agents’ performance reliably. To address this, the authors report testing across broad parameter ranges and many sessions using parallel cloud computing, creating a larger body of experimental results.
According to the document, simple strategies without AI or machine learning routinely outperform some public-domain AI or machine-learning traders from the research literature. The proposed lesson is methodological: comparisons need extensive evaluation across parameters and market conditions, and a sophisticated model should be tested against simple baselines. The excerpt does not identify the strategy rules, markets, performance measures, or specific AI agents, so its findings cannot be independently assessed or translated into a deployable trading system from this text alone. Its reported results concern the tested research agents and settings; they do not establish that simple strategies generally outperform AI-based approaches in live markets.
Key ideas
- The paper questions whether earlier evaluations of AI trading agents tested enough sessions and market scenarios.
- The authors report extensive testing over broad parameter ranges using parallel computing.
- In the reported tests, some simple non-AI strategies outperformed published AI and machine-learning traders.
- Simple baselines can help reveal whether a complex trading method adds value.
- The excerpt omits strategy details, markets, metrics, and evidence needed to assess live performance.
Tags
Full text
# Methods Matter: A Trading Agent with No Intelligence Routinely Outperforms AI-Based Traders # Methods Matter: A Trading Agent with No Intelligence Routinely Outperforms AI-Based Traders There's a long tradition of research using computational intelligence (methods from artificial intelligence (AI) and machine learning (ML)), to automatically discover, implement, and fine-tune strategies for autonomous adaptive automated trading in financial markets, with a sequence of research papers on this topic published at AI conferences such as IJCAI and in journals such as Artificial Intelligence: we show here that this strand of research has taken a number of methodological mis-steps and that actually some of the reportedly best-performing public-domain AI/ML trading strategies can routinely be out-performed by extremely simple trading strategies that involve no AI or ML at all. The results that we highlight here could easily have been revealed at the time that the relevant key papers were published, more than a decade ago, but the accepted methodology at the time of those publications involved a somewhat minimal approach to experimental evaluation of trader-agents, making claims on the basis of a few thousand test-sessions of the trader-agent in a small number of market scenarios. In this paper we present results from exhaustive testing over wide ranges of parameter values, using parallel cloud-computing facilities, where we conduct millions of tests and thereby create much richer data from which firmer conclusions can be drawn. We show that the best public-domain AI/ML traders in the published literature can be routinely outperformed by a "sub-zero-intelligence" trading strategy that at face value appears to be so simple as to be financially ruinous, but which interacts with the market in such a way that in practice it is more profitable than the well-known AI/ML strategies from the research literature. That such a simple strategy can outperform established AI/ML-based strategies is a sign that perhaps the AI/ML trading strategies were good answers to the wrong question.
Shown in full with attribution under the source's licence. Licence: abstract CC0
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.