Conditional and Exact AR Likelihoods for Residuals
Summary
The document explains why fitting an autoregressive model can produce fewer usable residuals than observations, and how the likelihood method affects that count. Conditional maximum likelihood starts the recursion after the model's lag order, leaving the initial residuals missing. Exact maximum likelihood accounts for the initial observations through their marginal distribution; for a stationary model, the presample values can be initialized using the sample or unconditional mean, allowing residuals from the first observation.
It also defines standardized residuals as model residuals divided by their standard deviation. When conditional variance changes over time, that scale can be estimated with a volatility model such as GARCH; otherwise, the residuals can be scaled by their unconditional volatility. The explanation gives formulas and an AR(1) likelihood comparison, but does not provide a worked dataset or discuss how initialization choices affect estimates. Its discussion assumes stationarity for the suggested presample treatment.
Key ideas
- Conditional likelihood for an AR model omits the initial observations needed to form lagged residuals.
- Exact likelihood includes a marginal term for the initial observations and can yield residuals for the full sample.
- For a stationary AR model, presample values may be initialized at the sample or unconditional mean.
- Standardized residuals divide residuals by their estimated conditional or unconditional standard deviation.
Tags
Full text
# How to obtain Standardized Residuals from a Time-Series?
# How to obtain Standardized Residuals from a Time-Series?
I have my estimates for an AR(3). To obtain the residuals I'm supposed to use $$Y_t-\hat\phi_0-\hat\phi_1Y_{t-1}-\hat\phi_2Y_{t-2}-\hat\phi_3Y_{t-3},$$
where the Y's are from the dataset. If I do this, won't I get a smaller number of residuals than observations?
I ask this, because in a book I'm reading, the authors fit an AR(3) from a dataset, and then present a plot of standardized residuals with the same numbers of observations. How is this possible?
Also, how does one obtain a standardized residual from a residual?
## Answer by Malick (score 5, accepted)
https://quant.stackexchange.com/a/26144
Usually for MLE estimation as you said we compute the residuals starting from index `number of lag+1` (p+1 for AR model) in this case we obtain Conditional MLE estimates:
$\hat{\theta} = \text{arg max} \sum_{p+1}^{T} \ln f(Y_{t}|\theta)$
where $f(Y_{t})$ is the marginal density of observation $Y_{t}$ and $\ln$ is employed to maximized the log likelihood. The first residuals are then fixed as missing values and then we get a smaller number of usable residuals than observations. (in this case you start the recursion algorithm from index P+1)
However, it is also possible to estimate your model with Exact MLE estimates:
$\hat{\theta} = \text{arg max} \sum_{p+1}^{T} \ln f(Y_{t}|\theta) + \ln f(Y_{p},...,Y_{1},\theta)$
This require to fix some pre-sample values (the $Y_{0},...Y_{-p+1}$ ) in order to be able to run the model. Given that the AR model is stationary you can fix these values to their sample mean or unconditional theoretical mean. In this case first residuals are not missing values and you obtain the same number of usable residuals than observations. (in this case you start the recursion algorithm from index 1)
Regarding standardized residuals $res_{std}$, it is simply the residuals from the model divided by the conditional standard deviation : $ res_{std}= res /\sigma_{t}$ , this require to estimate $\sigma_{t}$ via, for instance, a GARCH model. If you don't model the conditional variance part, you use : $res_{std}= res /\sigma $ where $\sigma$ is the unconditional volatility of residuals obtained during estimation.
See this nice paper for details.
EDIT
I found my answer no enough documented so this is an update. First the MLE method for a simple AR1 model is very well explained in the following page
See below,
(from : Hamilton, J. D. (1994). Time Series Analysis (1 edition). Princeton University Press. (page 122) ):
[...]
So as we see the Exact Log Likelihood [equation 5.2.9] differs from the Conditional Log likelihood [equation 5.2.27] in the sense that in the exact MLE we maximize the full likelihood (that is why we call it exact) whereas in the conditional version we truncate the likelihood by dropping the marginal : $-0.5 \log(2\pi) -0.5\log(\sigma^{2}/(1-\phi^{2}))- \frac{(y_{1}-(c/1-\phi))^2}{2\sigma^{2}/(1-\phi^2}$
This marginal replaces the expectation of $E(Y_{0}) = c+\phi Y_{-1} + \epsilon$ by the unconditonal mean $E(Y_{0}) = \frac{c}{1-\phi}$Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.