Causal and Noncausal Stationary Solutions of an AR(1) Model

# Why is $Z_t$ uncorrelated with $X_{t-1}$ in $X_t=\theta X_{t-1}+Z_t$?


# Why is $Z_t$ uncorrelated with $X_{t-1}$ in $X_t=\theta X_{t-1}+Z_t$?












In a solution to the problem below, the teaching assistant solves it by calculating $\mathbb{E}[X_t^2]$ and ends up with also having to calculate $\mathbb{E}[X_{t-1}Z_t]$ after expanding the square. To do this, he states that "$\mathbb{E}[X_{t-1}Z_t]=0$ since $X_{t-1}$ is independent of $Z_t$ and $X_{t-1}$ is uncorrelated with $Z_t$".

Questions:

- Why is $X_{t-1}$ independent of $Z_t$? I don't see that we assume that the time series is causal.

- Why is $X_{t-1}$ uncorrelated with $Z_t$?

Problem:

> Let a timeseries model $X:=(X_t, t\in\mathbb{Z})$ be given by $$X_t=\phi X_{t-1}+Z_t, \quad \text{where} \quad Z_t\sim \text{WN}(0,\sigma^2)\quad \text{and} \quad |\phi|\neq 1.$$ Assume that the stochastic process satisfying this model is stationary. Compute the variance of $X.$

Note: Yes, I know that one simply can calculate $\text{Var}[X_t]$ directly in one line, but I'm trying to understand the motivations behind the instructors steps.

## Answer by Jose Avilez (score 4, accepted)

https://quant.stackexchange.com/a/66514

$E(X_{t-1}Z_t) = 0$ in the causal case $|\phi | < 1$, but not in the non-causal case $|\phi | >1$.

Causal case $(|\phi| < 1)$

In this case, the unique stationary solution to the AR(1) equation is given by $$X_t = \sum_{j=1}^\infty \phi^j Z_{t-j}$$ Thus, $$E(X_{t-1} Z_t) = E \left( Z_t \sum_{j=1}^\infty \phi^j Z_{t-1-j} \right) = \sum_{j=1}^\infty \phi^j E (Z_t Z_{t-1-j}) = 0$$ where the last equality follows from $Z_i$ and $Z_j$ being uncorrelated for $i \neq j$, and all the exchanges of integration are guaranteed by Fubini's theorem.

Non-causal case $(|\phi| > 1)$

The unique stationary solution to the AR(1) equation is $$X_t = - \sum_{j=1}^\infty \phi^{-j} Z_{t+j}$$ This equation can be arrived at by performing the recursion forward, rather than backward. Thus,

$$E(X_{t-1}Z_t) = -\sum_{j=1}^\infty \phi^{-j} E(Z_{t-1+j} Z_t) = - \frac{\sigma_Z^2}{\phi}$$

This finding is confirmed if you use the AR equations to get $$Var(X_t) = \phi^2 Var(X_{t-1}) + Var(Z_t) + 2 \phi E(X_{t-1} Z_t)$$

Using $Var(X_{t-1}) = \frac{\sigma_Z^2}{\phi^2 - 1}$ you get $E(X_{t-1} Z_t) = - \frac{\sigma_Z^2}{\phi}$.

Aside: most people restrict the study of ARMA processes to the first case for two reasons: (i) the non-causal future-dependent case is strange, and (ii) in the non-causal stationary case you may always find a white noise process $\tilde{Z}_t$ such that $X_t$ is the causal solution to $X_t = \phi^{-1} X_{t-1} + \tilde{Z}_t$. This may explain why your TA simply assumed $E(X_{t-1}Z_t) = 0$ always.

Edit: There seems to be some confusion in the comments about what the solutions to a time series equation are. I find most textbooks skim over this technical detail.

A solution to a stochastic equation $$g(X_t, Z_t) = 0$$ is a pair $(X_t, Z_t)$ such that $(Z_t)_{t \in \mathbb{Z}}$ is a white noise and for each $t \in \mathbb{Z}$, $X_t$ is a (measurable) function of the entire white noise sequence (i.e. $X_t = h((\epsilon_{k})_{k\in\mathbb{Z}})$) and $X_t$ satisfies the stochastic equation in some sense (e.g. almost surely). Note that $X_t$ can depend on the "past" noise, on the "future" noise, or the entire sequence restrictions. The solution $(X_t, Z_t)$ is said to be stationary if $X_t$ is a stationary process.

Notice that for the AR(1) equation $X_t = \phi X_{t-1} + Z_t$ we can find unique stationary solutions when $|\phi| \neq 1$; you can do this by recursion into the past if $|\phi| < 1$ or by recursion into the future (time-reversal) if $|\phi| >1$. See Example 3.1.2 and Theorems 3.1.1-3.1.3 of Brockwell and Davis for more details.

In the case $|\phi| > 1$, we may also fix a stochastic process $(X_t)_{t \in \mathbb{N}}$ such that $X_0 = x$ and for $t \geq 1$, $X_t = X_0 + \sum_{j=1}^t Z_j$. While this is indeed a solution to the AR(1) equation, it is clearly not stationary. Thus, we can remove this case, as the OP asked us to consider the solution to the AR(1) equation when $X_t$ is stationary.