Skip to content
All library documents

Why Neural Network Choice Does Not Ensure Cross-Asset Forecasting

Article Quant Q&A · Author: Eka
Read the original

Summary

The exchange considers whether a recurrent neural network (RNN) trained on one asset’s OHLC history should outperform a multilayer perceptron (MLP) when asked to forecast another asset. The answer argues that naming an architecture alone says little about predictive performance. Model depth, activation functions, weight initialization, loss, learning rate, and training procedure all affect what is learned, while apparent in-sample accuracy does not establish reliable out-of-sample forecasting.

Applying a model trained on SPY to a different stock such as Google adds a domain shift: the target asset may have different behavior and need not be correlated with the training asset. The response recommends treating this as an additional source of failure and suggests beginning with methods suited to the prediction task, including robust linear models, rather than choosing a fashionable neural architecture. It offers general cautions rather than comparative experiments or a validated rule that RNNs should never be used. Model selection still requires asset-appropriate data, careful validation, and evidence on the intended forecasting task.

Key ideas

  • The RNN or MLP label alone does not determine predictive accuracy.
  • Training choices such as architecture, loss, initialization, and optimization materially affect results.
  • A model trained on one asset may not transfer to a different asset with distinct behavior.
  • In-sample accuracy does not demonstrate dependable out-of-sample performance.
  • Choose and validate models for the specific forecasting task instead of relying on algorithm popularity.

Tags

Machine learningEquitiesBacktestingStatistics
Full text
# When and how to use RNN for stock analysis or trading


# When and how to use RNN for stock analysis or trading












I am learning about neural network and created some small networks in feed forwarding network myself. I was curious about Recurrent Neural Networks (RNN) and read some papers about RNN in trading. The results in those paper using RNN seems promising but I have some general and hypothetical doubts regarding RNN. I learnt that RNN is best used for learning sequential and time series data.

Lets assume I made two models for predicting future price of stocks, one trained in RNN and other in MLP (Multi Layer Perceptron) using 10 years (OHLC) of data from SPY with good accuracy. I then wants to use these trained models to predict another stock's future price (may or may not be correlated with SPY) eg GOOGLE's tomorrows price.

(first question) Which algorithm has more chances to give an accurate prediction?

I know there is a good chance both models will fail to predict future price of a different product (here GOOG) correctly because the model is trained in SPY data.

But the question is; because RNN is used for training model for sequential and time series data. Is RNN have more probability of a failure prediction than MLP because it trained explicitly for SPY?

## Answer by Chris Taylor (score 5, accepted)

https://quant.stackexchange.com/a/25373

> Lets assume I made two models for predicting future price of stocks, one trained in RNN and other in MLP (Multi Layer Perceptron) using 10 years (OHLC) of data from SPY with good accuracy. Which algorithm has more chances to give an accurate prediction?

The choice of which model to use for training matters far, far less than the specific parameters used for the training. How many levels do your RNN and perceptron models have? What are the activation functions of the RNN? What algorithm did you use to initialize weights? What is your loss function? What is the training rate? Did you train using monolothic backpropagation or in batches? Did you use stochastic gradient descent?

The idea that you can just "train an RNN" or "train a MLP" and expect it to make good out of sample predictions is a fantasy, even when you aren't trying to make predictions for a different asset than the one you trained on. The fact that you want to train on SPY and predict on GOOG just adds extra complication (and more ways for your model to fail).

If you want to learn about neural networks, you should start with a problem them are known to be good at (e.g. text classification). If you want to predict stock prices, use an algorithm known to perform well on predicting stock prices (it's harder to find these, but maybe start with some kind of robust linear model).

Just picking a trendy machine learning method and a trendy problem to work on is a recipe for failure.

In answer to your question

> When and how to use RNN for stock analysis or trading?

The when is "never" and the how is irrelevant.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.