Skip to content
All library documents

Why Deep Learning Has Limited Success in Algorithmic Trading

Article Quant Q&A · Author: Chris D
Read the original

Summary

The answer explains why published deep learning results in trading are relatively scarce and why performance depends on the market and available data. It argues that rich equity data, including news and company information, may support more complex models, while fragmented fixed income markets make useful signals harder to identify. Limit order book analysis is another possible application, though the evidence described is mixed.

The discussion cautions that long financial histories do not necessarily provide the breadth of data needed to fit large networks, especially when markets are non-stationary. It cites equity factor research where neural networks beat linear models but the preferred networks were shallow, and limit order book work where pooled observations outperformed separate single-name models. These examples support using domain knowledge and model simplicity. The response is an informal explanation with references to selected studies, not a systematic survey, and it does not establish that deep learning cannot work in other settings.

Key ideas

  • Deep learning may be better suited to equities with rich cross-sectional information than to fragmented fixed income markets.
  • Long time series do not necessarily provide enough independent information when financial data are non-stationary.
  • Limit order book applications have mixed evidence, and pooled data can outperform models fitted separately to individual securities.
  • Selected equity factor research found neural networks outperforming linear models with shallow architectures.
  • Domain knowledge and simpler models can help reduce overfitting in financial applications.

Tags

Machine learningEquitiesFixed incomeHigh-frequency tradingStatistics
Full text
# Why are there so few published research papers that apply Deep Learning to Algorithmic Trading?


# Why are there so few published research papers that apply Deep Learning to Algorithmic Trading?












The only related papers I can find are:

Financial Trading as a Game: A Deep Reinforcement Learning Approach (2018) Deep Neural Networks in High Frequency Trading (2018) MACHINE LEARNING FOR TRADING (2017) A Deep Learning based Stock Trading Model with 2-D CNN Trend Detection (2017)

...and then a lot of older papers. Am I bad at searching or is there a lack of published research in this area?

I expect a lot of research goes unpublished (especially if it is profitable), but I am interested in published research.

## Answer by NBF (score 14, accepted)

https://quant.stackexchange.com/a/42753

I would say that most ML methods risk overfitting and it depends very much on the asset class. The only area where more sophisticated ML methods such as deep learning appear to make a major difference is in cash equities, where the feature space is very rich (NLP, news and announcements, corporate earnings, other financials) and the data is relatively good, Corporate bonds are too fractious a market (so many more products than in the equities space) but would be the next area worth considering.

But a lot of the ML guys who start trying to do rates get sorely disappointed. I heard from a guy who had a decent equity fund and wanted to do rates, and set his mind to parsing fed statements (bfd, been there done that, so what), then started asking why they were talking about hiking when PMIs were so low (duh! hadn't been looking at the news). Basically, FI is much harder. Too many products and not enough differentiation--does the announcement affect the 2y or the 5y? I'm not saying it's not feasible at some point but it isn't feasible now.

Simply speaking, DNNs work by dimension reduction. You have the possibility of thousands, perhaps tens of thousands or more parameters. If you have petabytes of image data, then you are doing a good job with dimension reduction. We do not have big data in most trading. Having more data (longer time series) isn't as good as increasing the cross-sectional dimension because we have non-stationary series anyway. Other than equities, analysis of limit order books may be an area where DNN can have applications although the evidence is mixed (domain knowledge is always more important than just throwing data into some model...GIGO).

Non-stationarity alone will reduce the complexity of any good model. The recent paper by AQR on using neural nets (for EQUITY factor extraction) showed that NNs outperformed linear models but that the optimal nets were in fact quite shallow (see Empirical Asset Pricing via Machine Learning).

In general, naive applications of NNs to finance are doomed from the get-go. The paper by Cont and Sirignano applying LSTMs to LOBs on a single-name and pooled basis shows the limits of the approach, where they found pooled data worked far better (pooled data = 1/300 or so the number of parameters as the single name models). They call it 'universal rules' but their interpretation is very generous! it's just that there wasn't enough data (see NN learned universal model. It's pretty clear that a big model applied to tons of data seems to result in just a lack of any understanding. Domain knowledge and lighter touches make for far more robust results.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.