Estimating Conditional Expectations with Least Squares Monte Carlo
Summary
This note illustrates how least squares Monte Carlo can approximate a conditional expectation using simulated observations. For independent uniform variables X and Y, it forms an observable Z from a nonlinear function of both, then regresses simulated X values on basis functions of Z. The fitted regression estimates the conditional mean of X given Z at selected values, making the idea accessible outside Bermudan option valuation.
The example uses Z and its square root as regression predictors, while another answer frames conditional expectation as the function minimizing expected squared error and suggests polynomial regression when a linear form is unsuitable. The demonstration is illustrative, not a comparison of estimators: it provides no error analysis, validation, or guarantee that the chosen basis captures the true conditional relationship. Results depend on simulation size and basis choice.
Key ideas
- Conditional expectation can be viewed as the function of Z that minimizes expected squared prediction error for X.
- Least squares Monte Carlo estimates this function by simulating observations and regressing X on functions of Z.
- The example uses both Z and its square root as basis terms to represent a nonlinear relationship.
- A linear basis may be inadequate, so basis choice and validation matter for the quality of the estimate.
Tags
Full text
# Estimating conditional expectation using monte carlo and least squares regression
# Estimating conditional expectation using monte carlo and least squares regression
I'm looking to understand the problem of least squares monte carlo that is used in valuation of bermudan options, but from a simpler context.
Say I have random variables $X$ and $Y$ which are uniform [0,1] and independent. Define $Z=X^2+Y^2+XY$. Let us say I want to evaluate the expectation $E(X|Z=a)$ using Monte Carlo. Can least squares MC help in this case? If so, can anyone outline the process?
I'm trying to understand the core of the algorithm without the tedious notation one has to go through while reading papers and other articles explaining MC least squares.
## Answer by Bob Jansen (score 1, accepted)
https://quant.stackexchange.com/a/68898
The link in your comments mention section 11.6 of Numerical Methods in Economics by Kenneth Judd. I recommend giving that a read as well. It's only a few pages. Below some code that implements least squares Monte Carlo for the problem you gave:
```
set.seed(42)
fun <- function(x, y) x^2 + y^2 + x * y
N <- 1e3L
X <- runif(N)
Y <- runif(N)
Z <- fun(X, Y)
plot(Z, X)
# We can create a function of Z that gives an estimate of X:
model <- lm(X ~ Z + sqrt(Z))
print(model)
# Call:
# lm(formula = X ~ Z + sqrt(Z))
#
# Coefficients:
# (Intercept) Z sqrt(Z)
# 0.02860 0.07183 0.44900
a <- seq(0, 3, by = 0.01)
x_hat <-
model$coefficients[[1L]] +
a * model$coefficients[[2L]] + sqrt(a) * model$coefficients[[3L]]
lines(a, x_hat, col = 'blue')
```
## Answer by Kurt G. (score 1)
https://quant.stackexchange.com/a/68895
I am not sure how to outline that process without going through some notation. The definition of the conditional expectation $f(a)=E[X|Z=a]$ is that the function $f$ is such that the squared norm $E[(X-f(Z))^2]$ is minimized.
Least squares regression looks only for affine linear functions $f(a)=\beta\,a+\varepsilon$ (where $\beta,\varepsilon$ are the constants that are to be found). In your example $X$ is not a linear function of $Z$ (take a small $Y$ so see that it is more like $X\sim\sqrt{Z}$). Therefore I find it unlikely that an affine linear $f$ is a good candidate for $E[X|Z=a]\,.$ It is probably better to try Polynomial regression instead.
Python should have all the packages to simulate $X,Y,Z$ and try all sorts of regressions and compare.Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.