Interpreting Stepwise BHHH and Levenberg–Marquardt Optimization
Summary
The document asks how a paper’s description of maximizing a likelihood by iterating between the Marquardt, or Levenberg–Marquardt, algorithm and BHHH should be understood. The practical distinction is whether each optimizer is run to completion before switching, or whether the methods alternate during optimization.
The response infers that the procedure is stepwise: each algorithm supplies an iteration, followed by a step from the other method, rather than each being run as a separate full optimization. Its reasoning is that both methods are expected to stop when the Kuhn–Tucker condition is satisfied, which makes full sequential runs a less likely interpretation of the paper’s wording. This is an informed interpretation of a brief description, not a definitive account from the paper’s authors. The exchange gives no implementation details, convergence analysis, or empirical comparison of the algorithms.
Key ideas
- The question concerns alternating BHHH and Levenberg–Marquardt steps during likelihood maximization.
- The reply interprets the description as switching algorithms step by step rather than running each to completion.
- The interpretation is based on the expectation that each optimizer stops at a first-order optimality condition.
- The exchange does not establish the authors’ exact implementation or give convergence evidence.
Tags
Full text
# Combining BHHH and Levenberg Marquardt # Combining BHHH and Levenberg Marquardt I already asked a question related to this here: How to apply Levenberg Marquardt to Max Likelihood Estimation I know understand how Levenberg Marquardt (LM) can be applied to the objective function. In the paper on p315: http://www.ssc.upenn.edu/~fdiebold/papers/paper55/DRAfinal.pdf the authors also state that: "We maximize the likelihood by iterating the Marquardt and Berndt–Hall–Hall–Hausman algorithms, using numerical derivatives, optimal stepsize, and a convergence criterion of 10^-6 for the change in the norm of the parameter vector from one iteration to the next." Does anyone know what precisely they mean by this? Do you run the LM fully and then take the optimal parameter set found and use those as initial guesses for the BHHH algorithm, then run the BHHH and take those solutions as initial guesses to the LM and repeat until convergence? Or are they iterating step by step? So you do the same as above but rather than running a full optimization each time, you simply so one step at a time? Even your best guesses would be most welcome. Baz ## Answer by Brian B (score 1) https://quant.stackexchange.com/a/12826 Well, given that either LM or BHHH is supposed to stop when the Kuhn-Tucker condition is satisfied, I infer it has to be stepwise. I would say otherwise if, say, they were potentially using something like SALO (simulated annealing with local optimization), where one algorithm could profitably run in full as a sub-step of the other.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.