Decomposing a Random-Effects Covariance Matrix into Common and Residual Components
Summary
The document explains an algebraic rewriting of a covariance matrix formed from a shared random effect and independent residual noise. The key object is the matrix that projects onto the direction of the all-ones vector; with a vector of length T, its normalized outer product has every entry equal to 1/T. The complementary matrix projects onto directions orthogonal to that common component.
Rewriting the covariance in these two subspaces separates the eigenvalue associated with the shared component from the residual eigenvalue. This makes the inverse and determinant formulas in the question easier to understand: each subspace is scaled by its corresponding variance. The answer gives a stepwise expansion and assumes the usual dimensions and scalar-identity notation. It is a focused derivation rather than a broader treatment of maximum likelihood, and does not discuss estimation, empirical results, or extensions beyond this covariance structure.
Key ideas
- The normalized outer product of the all-ones vector projects onto the common component.
- The identity matrix minus that projection selects directions orthogonal to the common component.
- The covariance matrix acts with one scale on the common direction and another on the orthogonal directions.
- This decomposition clarifies how to obtain the matrix inverse and determinant.
Tags
Full text
# Understanding the derivation of a ML-estimator
# Understanding the derivation of a ML-estimator
I'm trying to understand the derivation of a ML-estimator and more specifically the rewriting of the covariance matrix Sigma. In this rewriting a lemma is used to show that
$$ (1) \hspace{1.4 cm}\Omega=\sigma^2_{c}\boldsymbol{1}\boldsymbol{1'} + \sigma^2_{\varepsilon}I_T=(\sigma^2_{\varepsilon}+T\sigma^2_{c})\boldsymbol{1}(\boldsymbol{1}'\boldsymbol{1})^{-1}\boldsymbol{1}'+\sigma^2_{\varepsilon}(I_T- \boldsymbol{1}(\boldsymbol{1}'\boldsymbol{1})^{-1}\boldsymbol{1}')$$ $$\Omega^{-1}=\frac{1}{\sigma^2_{\varepsilon}+T\sigma^2_{c}}\boldsymbol{1}(\boldsymbol{1}'\boldsymbol{1})^{-1}\boldsymbol{1}'+\frac{1}{\sigma^2_{\varepsilon}}(I_t-\boldsymbol{1}(\boldsymbol{1}'\boldsymbol{1})^{-1}\boldsymbol{1}')$$ $$|\Omega|=(\sigma^2_{\varepsilon}+T\sigma^2_{c})\sigma^{{2(T-1)}}_{\varepsilon}$$
The Lemma states:
Can anyone explain the second equality in (1)?
## Answer by pbr142 (score 2, accepted)
https://quant.stackexchange.com/a/10764
I agree with vanguard2k's comment: A few more details on the notation would be helpful. But, as far as I can tell, the second equality is a simple expansion.
First, $\mathbf{1}'\mathbf{1} = T$ (assuming the vectors are elements of $\mathbb{R}^T$). The expression $\mathbf{1} (\mathbf{1}'\mathbf{1})^{-1} \mathbf{1}'$ is therefore nothing else than a $T\times T$ matrix with $\frac{1}{T}$ in every element.
I assume that scalar addition to a matrix actually means that a matrix with the scalar as each element is added, i.e. that $\sigma_c^2 + \sigma_{\epsilon}^2 \mathbf{I}_T := \sigma_c^2 \mathbf{1}\mathbf{1}' + \sigma_{\epsilon}^2 \mathbf{I}_T$. Then you can write: \begin{align} \sigma_c^2 + \sigma_{\epsilon}^2 \mathbf{I}_T =&\ \sigma_c^2 T \frac{1}{T} \mathbf{1}\mathbf{1}' + \sigma_{\epsilon}^2 \mathbf{I}_T \\ =&\ T \sigma_c^2 \mathbf{1}(\mathbf{1}'\mathbf{1})^{-1} \mathbf{1}' + \sigma_{\epsilon}^2 \mathbf{I}_T + \sigma_{\epsilon}^2\mathbf{1}(\mathbf{1}'\mathbf{1})^{-1} \mathbf{1} - \sigma_{\epsilon}^2\mathbf{1}(\mathbf{1}'\mathbf{1})^{-1} \mathbf{1}' \\ =&\ (\sigma_{\epsilon}^2 + T \sigma_c^2) \mathbf{1}(\mathbf{1}'\mathbf{1})^{-1} \mathbf{1}' + \sigma_{\epsilon}^2 (\mathbf{I}_T - \mathbf{1}(\mathbf{1}'\mathbf{1})^{-1} \mathbf{1}') \end{align}Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.