Bayesian Updating for a Coin with an Unknown Bias

# bayes theorem probability Jar


# bayes theorem probability Jar












I am trying to come up with different theoretical answers below.

I believe the standard one is based on Bayes theorem, but I am struggling to prove it.

A jar has 1000 coins, of which 999 are fair and 1 is double headed. Pick a coin at random, and toss it 10 times. Given that you see 10 heads, what is the probability that the next toss of that coin is also a heads? Prove it.

Does anyone have any thoughts?

## Answer by Bob Jansen (score 3)

https://quant.stackexchange.com/a/44737

This is just an extension of problem 4.22 in interview practice book "Heard on the Street".

Maybe they expected you to reason along these lines:

$$\frac{1}{2^{10}}= \frac{1}{1024} \approx \frac{1}{1000}.$$

So the odds of fair coin vs non fair coin are about even and the probability of heads is approximately

$$0.5 \times 0.5 + 0.5 \times 1 = 0.75.$$

The above argument can be made more precise, let $u = \frac{999}{1000}\frac{1}{2^{10}}$ be the probability that an unbiased coin is selected and 10 heads are observed, $b = \frac{1}{1000}$ the probability the biased coin is selected and $p = u + b$. Either $u$ or $b$ happened so the probability of heads on the next throw is

$$P(\textrm{Next throw is heads}) = \frac{1}{2} \frac{u}{p} + \frac{b}{p}.$$

You can verify that $u \approx b$ and thus the back of the envelop calculation is not far of and that this leads to the same fraction as the other answers.

## Answer by Dave Harris (score 2)

https://quant.stackexchange.com/a/44742

Let $X_i$ be the result, where $X_i=1$ implies heads and $X_i=0$ as tails.

Let $\theta_j\in\{0.5,1\}$, where $\theta_j$ is the bias for heads.

$\theta_1=.5$ and $\theta_2=1$.

$$\Pr(\theta_1|X_{1\dots{10}}=1)\propto{\left(\frac{1}{2}\right)^{10}}\frac{999}{1000}=\frac{999}{1000\times{1024}}.$$

$$\Pr(\theta_2|X_{1\dots{10}}=1)\propto{1^{10}}\frac{1}{1000}=\frac{1024}{1000\times{1024}}.$$

$$\Pr(X_{1\dots{10}}=1)=\frac{999+1024}{1000\times{1024}}$$

$$\Pr(\theta_1|X_{1\dots{10}}=1)=\frac{999}{999+1024}=\frac{999}{2023}$$

$$\Pr(\theta_2|X_{1\dots{10}}=1)=\frac{1024}{999+1024}=\frac{1024}{2023}$$

$$\Pr(X_{11}=1|X_{1\dots{10}}=1)=\sum_{j=1}^2\left[\theta_j(1-\theta_j)\right]\Pr(\theta_j|X_{1\dots{10}}=1)$$

$$\Pr(X_{11}=1|X_{1\dots{10}}=1)=\frac{1}{2}\frac{999}{2023}+1\frac{1024\times{2}}{2\times{2023}}=\frac{3047}{4046}\approx{.75}$$

I debated answering this question as it could be viewed as more appropriate for Cross Validated or Mathematics, however, I decided to do so for a couple of reasons directly related to QF.

First, quantitative finance is calculated gambling. Bayesian statistics are coherent. Frequentist statistics are incoherent. A statistic is considered coherent if a fair gamble can be created from it. It vastly exceeds the scope of your question, but if you are pricing a loan or an option then it is technically incorrect to use a Frequentist method, at least for a financial intermediary.

The second reason is that this problem is a discrete form of a real finance problem. Given an unknown parameter and a historical record, what is the probability of a future state of the world?

You need to get a very good grasp on the Bayesian prior distribution, the Bayesian posterior distribution and the Bayesian posterior predictive distribution.

## Answer by Attack68 (score 0)

https://quant.stackexchange.com/a/44736

Let (X,Y) the condition the coin is (fair, double headed):

$P(Y | 10H) = P(10H | Y) \frac{P(Y)}{P(10H)} $

where

$P(10H|Y) = 1$ $P(Y) = \frac{1}{1000} $ $P(10H) = P(10H|X)P(X) + P(10H|Y)P(Y) = 0.5^{10}\frac{999}{1000} + \frac{1}{1000} = \frac{999}{1000*1024} + \frac{1}{1000} = \frac{2023}{1000*1024}$

$P(Y | 10H) = 1 * \frac{1}{1000} * \frac{1024*1000}{2023} = \frac{1024}{2023}$ $P(X | 10H) = \frac{999}{2023}$

So the probability of the next coin toss being heads is:

$0.5 * P(X|10H) + 1 * P(Y | 10H) = 0.5*\frac{999}{2023} + 1 * \frac{1024}{2023} = \frac{3047}{4046}$