Adaptive and One-Cycle Learning Rates for MLP Trading Models

Summary

The article compares adaptive learning rate methods for multilayer perceptrons with a one-cycle schedule. It describes adaptive gradient, RMS, mean exponential, and delta approaches, which derive parameter-specific updates from training gradients while requiring relatively few inputs. The one-cycle method instead raises the rate from a minimum to a peak during warm-up, then reduces it during the remainder of training.

The examples use an Expert Advisor tested on NZDUSD for 2023, with sigmoid activations and raw price inputs selected to fit the activation range. The author reports that changing the learning rate affected network performance, including a run that produced short trades. These results are exploratory rather than evidence of a robust edge: the setup changed from the prior article, inputs were not batch normalized, and the examples are not presented as optimized settings. The text recommends tuning with quality historical data and checking results in forward testing before deployment.

Key ideas

• Adaptive learning rates use gradients to vary updates across network parameters.
• The article describes adaptive gradient, RMS, mean exponential, and delta update formats.
• A one-cycle schedule warms the learning rate up before reducing it during training.
• Activation choice and input scaling affect whether network outputs remain valid.
• The reported trading tests are exploratory and require further tuning and forward evaluation.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.