Deriving the Optimal Stopping Equation from a Bellman Relation
Summary
The document works through a question about deriving a continuous-time equation for the value of an investment with an option to stop. It presents a Bellman relation that compares immediate stopping value with the value of continuing, including profit flow and discounting. In the continuation region, the value follows an Ito process, and applying Ito’s lemma yields a differential equation involving the value’s time derivative, first and second derivatives with respect to the state, drift, volatility, discount rate, and profit flow.
The questioner’s algebra reveals the central issue: the profit term in a discrete-time Bellman step must represent profit accrued over the time increment, so it should scale with that increment. Dividing by the increment and taking its limit produces the continuous-time equation; the discount factor likewise contributes the discount-rate term in the limit. The document supplies the setup and attempted derivation but not a posted resolution, so the explanation is reconstructed from the displayed equations and conventions.
Key ideas
- The Bellman relation compares stopping immediately with continuing the investment.
- The continuation value is expanded using Ito’s lemma for the state process.
- Profit flow over a short interval must be scaled by the interval length.
- Taking the continuous-time limit yields the drift, diffusion, discount, and profit terms in the equation.
Tags
Full text
# Dixit & Pindyck (1993) Chapter 4, equation 13
# Dixit & Pindyck (1993) Chapter 4, equation 13
Starting with the Bellman equation for the optimal stopping problem: $$F(x,t)=max\{\Omega(x,t), \pi(x,t)+(1+\rho dt)^{-1} E[F(x+dx, t+dt)|x]\}$$ In the continuation region where the second term is the greatest they get the following equation after expanding by Ito's Lemma: $$\frac{1}{2}b^2(x,t)F_{xx}(x,t)+a(x,t)F_x(x,t)+F_t(x,t)-\rho F(x,t)+\pi(x,t)=0 $$ (13)
$F(x,t)$ is the value of an investment. $\pi(x,t)$ is the profit flow, $\rho$ is the discount rate and $\Omega(x,t)$ is the value achieved if stopping. $x$ is assumed to follow an Ito process: $$dx=a(x,t)dt+b(x,t)dz$$ I dont understand how (13) is reached. The closest i get is: $$F(x,t)=\pi(x,t)+(1+\rho dt)^{-1}E[F(x+dx, t+dt)|x]$$ Multiplying with $(1+\rho dt)$ $$F(x,t)+\rho F(x,t)dt=\pi(x,t)(1+\rho dt)+E[F(x+dx, t+dt)|x]$$ $$\rho F(x,t)dt=\pi(x,t)(1+\rho dt)+E[F(x+dx, t+dt)|x]-F(x,t)$$ $$\rho F(x,t)dt=\pi(x,t)(1+\rho dt)+E[F(x+dx, t+dt)-F(x,t)|x]$$ $$\rho F(x,t)dt=\pi(x,t)(1+\rho dt)+E[dF|x]$$ Expanding dF with Itos lemma: $$dF=[F_t(x,t)+a(x,t)F_x(x,t)+\frac{1}{2}b(x,t)^2F_{xx}(x,t)]dt+b(x,t)F_x(x,t)dz$$ Now I can continue. $$\rho F(x,t)dt=\pi(x,t)(1+\rho dt)+[F_t(x,t)+a(x,t)F_x(x,t)+\frac{1}{2}b(x,t)^2F_{xx}(x,t)]dt$$ Divide by $dt$ $$\rho F(x,t)=\pi(x,t)\frac{1+\rho dt}{dt}+F_t(x,t)+a(x,t)F_x(x,t)+\frac{1}{2}b(x,t)^2F_{xx}(x,t)$$ Then rearranging I may have. $$\frac{1}{2}b(x,t)^2F_{xx}(x,t)+a(x,t)F_x(x,t)+F_t(x,t)-\rho F(x,t)+\pi(x,t)\frac{1+\rho dt}{dt}=0$$ To have the same equation as in the book i can think of two adjustment/assumptions. $\rho dt = 0$ and $\pi(x,t)$ needs to be replaced with $\pi(x,t)dt$ in the first Bellman equation. Is this how equation (13) is achieved or am I missing something?Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.