Skip to content
All library documents

Transition Probabilities and Expectations in a Two-State Markov Chain

Article Quant Q&A · Author: fincecon
Read the original

Summary

The document shows how to calculate the future distribution of a binary state that switches between high and low values in continuous time. It represents the process as a two-state continuous-time Markov chain, with one transition rate out of each state. These rates form the entries of a generator matrix: the diagonal entries are negative exit rates, and the off-diagonal entries are transition rates to the other state.

Exponentiating the generator over the time interval gives the transition matrix. Its rows provide the probabilities of being in each state at the future time, conditional on starting in the corresponding current state. The expected future value of the process can then be calculated by weighting the high and low state values by those probabilities. The answer notes that the matrix is the identity when the interval is zero and tends toward the stationary distribution over a long horizon. The method assumes constant transition rates and the Markov property; time-varying rates would require a different transition calculation.

Key ideas

  • Represent the high and low states with a generator matrix whose off-diagonal entries are transition rates.
  • The transition matrix over a time interval is the matrix exponential of the generator multiplied by that interval.
  • Each row of the transition matrix gives future state probabilities conditional on the starting state.
  • Compute the conditional expectation by weighting each possible state value by its transition probability.
  • With constant rates, the transition probabilities converge toward a stationary distribution over long horizons.

Tags

StatisticsMulti-asset
Full text
# Continuous-time two-state Markov process


# Continuous-time two-state Markov process












$\lambda_t$ is binary with $\lambda_H$ and $\lambda_L$, with instantaneous transition probailities of $\mu_H$ and $\mu_L$.

What is $\mathbb{E}_t[\lambda_T]$, assuming $\lambda_t=\lambda_H$ or $\lambda_t=\lambda_L$?

## Answer by Wei (score 2, accepted)

https://quant.stackexchange.com/a/80773

This is a continuous-time Markov chain with rate matrix $$Q=\begin{pmatrix} -\mu_H & \mu_H\\ \mu_L & -\mu_L \end{pmatrix}.$$ The transition matrix associated with going from time $t$ to time $T$ is then (using WolframAlpha) $$P(t,T) = \exp((T-t)Q)=\frac{1}{\mu_H+\mu_L}\begin{pmatrix} \mu_He^{-(T-t)(\mu_H+\mu_L)} + \mu_L & \mu_H(1-e^{-(T-t)(\mu_H+\mu_L)})\\ \mu_L(1-e^{-(T-t)(\mu_H+\mu_L)}) & \mu_H + \mu_Le^{-(T-t)(\mu_H+\mu_L)} \end{pmatrix}.$$ The top row of this matrix gives the distribution of $\lambda_T$ given $\lambda_t=\lambda_H$. The bottom row of this matrix gives the distribution of $\lambda_T$ given $\lambda_t=\lambda_L$. You can see that it's the identity matrix when $t=T$, and as $T\to \infty$ it converges to the stationary distribution given by nbbo2 in the comments of your question.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.