Adapting ADAM into a Population Optimizer for Numerical Search
Summary
The article describes converting ADAM, originally a gradient-based optimizer used in neural networks, into a population-based method for general optimization problems. It reviews ADAM’s first- and second-moment updates, then outlines how the population version initializes candidate solutions, estimates numerical gradients from changes in objective values and coordinates, and updates candidates using moment estimates. Randomness is introduced through population initialization and later modifications to the search dynamics, allowing the method to explore objectives without analytical gradients.
The article reports tests of the optimizer and a modified variant, ADAMm, across benchmark functions, with results summarized in comparison charts and a rating table. It characterizes the methods as producing good results on low-dimensional problems with relatively low result dispersion, while identifying the number of external parameters as a drawback. The benchmark evidence is specific to the test setup; the article cautions that algorithm implementations may differ from canonical versions and that experimental conclusions do not establish performance on trading data or live markets.
Key ideas
- ADAM updates parameters using bias-corrected estimates of gradient means and squared gradients.
- A population version can estimate gradients numerically, extending the approach to objectives without analytical derivatives.
- Random initial candidate positions provide population diversity, and the article also explores randomness in search dynamics.
- The reported tests favor performance on low-dimensional benchmark problems but do not establish trading effectiveness.
- The method has several external parameters that require selection.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.