Tuning Neural Network Hyperparameters with Nelder–Mead
Summary
The article demonstrates tuning an MLP regressor for Brent oil price forecasting with the derivative-free Nelder–Mead algorithm. It defines a target using a future close, evaluates candidate settings by time-series cross-validation RMSE, and searches regularization strength, initial learning rate, and tolerance. A preliminary line search is used to choose a starting region before optimization, and the article explains how to interpret the optimizer’s output and evaluation counts.
The reported comparison across test folds is mixed: the customized model has lower errors on some folds but substantially higher errors on others, while the text characterizes it as outperforming the default. The author cautions that tuning and assessment used the same dataset, weakening that comparison, and notes that the optimizer stopped after reaching its function-evaluation limit. The example is a demonstration rather than evidence of reliable out-of-sample trading performance; the described forecast and model errors do not establish profitability.
Key ideas
- Nelder–Mead searches for lower objective values without requiring derivative information, but results can depend on the initial point.
- The example minimizes average time-series cross-validation RMSE for three MLP settings.
- A preliminary parameter sweep is used to select a plausible starting region for the search.
- Each objective evaluation fits the model across multiple time splits, so optimization can require substantial computation.
- Using the same data for tuning and performance comparison risks an optimistic or unreliable assessment.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.