Metropolis–Hastings Sampling for Bayesian and Statistical Models

Summary

The article introduces Markov chain Monte Carlo sampling and explains the Metropolis–Hastings algorithm for approximating a target distribution when its normalized density is difficult to compute. Each iteration proposes a candidate state and accepts it with a probability based on the ratio of target densities and, for asymmetric proposals, a Hastings correction. Because the normalization constant cancels, the method can use an unnormalized density. The article also distinguishes symmetric random-walk proposals from independent proposals and describes an object-oriented implementation in MQL5.

It discusses practical diagnostics and tuning: accepted and rejected moves create correlated samples, so burn-in, thinning, acceptance-rate monitoring, trace plots, histograms, and autocorrelation checks are covered. Proposal scale must balance exploration against rejection and autocorrelation. The article presents univariate and multivariate examples, but the excerpt supplies no detailed numerical results. It cautions that basic Metropolis–Hastings can perform poorly in higher dimensions or with strongly correlated variables, where methods such as Hamiltonian Monte Carlo or slice sampling may be more effective.

Key ideas

• Metropolis–Hastings constructs a Markov chain whose long-run distribution can match a target distribution.
• The acceptance probability uses unnormalized target densities, with a proposal-density correction for asymmetric proposals.
• Proposal choice and scale affect both acceptance and how efficiently the chain explores the state space.
• Autocorrelation reduces the effective information in a sample, making convergence diagnostics important.
• Basic Metropolis–Hastings may require substantial tuning and can struggle with high-dimensional or correlated targets.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.