Importance Sampling for Rare Gaussian Tail Probabilities
Summary
The document presents a measure-change approach to estimating a very small standard normal tail probability. It shifts the sampling distribution toward the rare event by using a normal distribution centered at the threshold, where exceedances become much more frequent. Each sampled outcome is then weighted by the likelihood ratio between the original and shifted distributions, so the weighted average estimates the probability under the original measure.
A Monte Carlo example reports an estimate close to a separately computed reference value. This illustrates how importance sampling can make rare-event estimation practical when direct sampling would produce few or no tail observations. The example is specific to a one-dimensional Gaussian tail and does not discuss variance comparisons, optimal choices of shifted distribution, or numerical stability for extremely small probabilities; those considerations would matter when applying the technique to trading or risk models.
Key ideas
- Importance sampling shifts draws toward a rare event to obtain more informative samples.
- Likelihood-ratio weights convert estimates under the shifted measure back to the original probability.
- Centering the sampling distribution at the threshold makes the tail event much more common in the example.
- The reported simulation estimate is compared with a reference probability, but broader variance and stability analysis is absent.
Tags
Full text
# Probability in different measures
# Probability in different measures
I'm having some troubles understanding a problem.
The problem: "Show how a measure change can be used to estimate the probability for $Y > 100$ when $Y \sim \mathcal{N}(0, 1)$.
The book I'm using doesn't go through this part that well, so could anyone explain this example?
## Answer by ragoragino (score 13, accepted)
https://quant.stackexchange.com/a/36414
In order to estimate a probability of an event with small probability, you might want to try to estimate the probability for a changed random variable that allocates a larger probability mass for the event happening. So, in your case, you might want to change the original $N(0, 1)$ to $N(100, 1)$ because for the second r.v. the probability of it being higher than 100 is $\frac{1}{2}$. So, you are looking for a change of measure in form:
$$ \frac{dQ}{dP} = \frac{dN(0,1)}{dN(100,1)} = exp(-100y+5000), $$
where $Q$ is the probability measure associated with $N(0, 1)$ and $P$ is the probability measure associated with $N(100, 1)$.
And therefore for an event $A = \{Y > 100\}$ by the Radon-Nikodym theorem we can obtain: $$ Q(A) = E_{P}[exp(-100Y+5000)I(A)] \approx \frac{1}{n} \sum_{i=1}^{n} e^{-100y_i + 5000} I(y_i > 100) \\ = e^{-5000} \frac{1}{n} \sum_{i=1}^{n} e^{-100(y_i-100)} I(y_i > 100) $$
By running, for example this C++ code:
```
#include <vector>
#include <random>
#include <iostream>
int main()
{
std::vector<long double> estimation;
int simulation_length { 1000000 };
std::default_random_engine generator;
std::normal_distribution<long double> distribution(100.0, 1.0);
long double pick;
long double estimation_sum { 0.0 };
for (int i = 0; i != simulation_length; ++i)
{
pick = distribution(generator);
if (pick > 100)
{
estimation.push_back(exp(-100 * (pick - 100)));
}
}
for (double i : estimation)
{
estimation_sum += i;
}
std::cout << estimation_sum / simulation_length << std::endl;
return 0;
}
```
you get the output 0.00400264. By multiplying it by exp(-5000) you get 1.34876725857872×10^(-2174), which is very close to the true value (i.e. 1.34417907674465×10^(-2174), obtained through Mathematica)Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.