Bayesian Inference Explained Through Coin Tossing and Maximum Likelihood

Summary

This article introduces Bayesian inference by estimating the unknown probability of heads for a coin. It contrasts the frequentist view, where the parameter is fixed but unknown, with the Bayesian view, where uncertainty about the parameter is represented by a probability distribution. Using a hypothetical sequence of tosses, it explains how the likelihood represents the observed data under a parameter value, how the prior encodes beliefs before observing data, and how Bayes’ theorem combines them into a posterior. The marginal probability of the data normalizes that posterior.

The article also outlines maximum likelihood estimation for independent Bernoulli observations, including maximizing the likelihood or its logarithm. It presents the posterior as proportional to likelihood times prior and describes the posterior as a summary of prior knowledge updated with evidence. The discussion is conceptual and algebraic; the displayed derivations are partly omitted, and it does not provide the promised simulation or demonstrate a trading application. Its claim that Bayesian and frequentist estimates converge with increasing sample size is presented without qualifications.

Key ideas

• Bayesian inference updates a prior distribution using the likelihood of observed data to form a posterior.
• For coin tosses, the unknown probability of heads is modeled as a parameter between zero and one.
• The frequentist approach treats the parameter as fixed, while the Bayesian approach represents uncertainty about it probabilistically.
• Maximum likelihood estimation selects the parameter value that best accounts for the observed outcomes.
• The data's marginal probability normalizes the posterior distribution.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.