Skip to content
All library documents

Visualizing the Outer-Product Expansion of a Regression Matrix

Article Quant Q&A · Author: whisperer
Read the original

Summary

The document explains how the matrix product XᵀX can be written as a sum of outer products. When the rows of X are represented as column vectors xᵢ, each term xᵢxᵢᵀ contributes the products of one row’s entries to the corresponding cells of XᵀX. Summing these contributions across rows reconstructs the full matrix.

A small two-by-two example makes the identity concrete: it forms the outer product for each row vector, then adds the resulting matrices to obtain XᵀX. This is an illustration of matrix algebra rather than a treatment of regression estimation or a trading method. The example shows the relationship directly, but does not discuss larger matrices, assumptions for regression, or numerical implementation.

Key ideas

  • Each row of X contributes one outer product to XᵀX.
  • An outer product xᵢxᵢᵀ records all pairwise products of entries in a row vector.
  • Adding the row-wise outer products reproduces XᵀX.
  • A small numeric matrix can make the summation identity easier to visualize.

Tags

Statistics
Full text
# [Notation Query ]Expressing matrix as summation over product of vectors (Coefficient of Regression)


# [Notation Query ]Expressing matrix as summation over product of vectors (Coefficient of Regression)












The coefficient of regression $\beta$ is often expressed as:

$\beta = (X^TX)^{-1}X^Ty$

I came across the notation below. Can someone help me visualize how the summation of column vectors $x_i$ is equivalent to the matrix notation?

$X^TX = \sum_{i=1}^{n}(x_ix_i^T)$

## Answer by phdstudent (score 2)

https://quant.stackexchange.com/a/39217

In econometrics it is usually simpler to use a small example. I also have a lot of issues in visualizing those notations and small examples always help me. Let's make an example:

$ X= \begin{bmatrix} 1 & 5 \\ 3 & 7 \end{bmatrix}$

Therefore: $ X^T X= \begin{bmatrix} 10 & 26 \\ 26 & 74 \end{bmatrix}$

Now two vectors: $x_1 = \begin{bmatrix} 1 \\ 5 \end{bmatrix}$ and $x_2 = \begin{bmatrix} 3 \\ 7 \end{bmatrix}$

So:

$x_1 x_1^T = \begin{bmatrix} 1 & 5\\ 5 & 25 \end{bmatrix}$ and $x_2 x_2^T = \begin{bmatrix} 9 & 21\\ 21 & 49 \end{bmatrix}$

Add both togheter: $x_1 x_1^T + x_2 x_2^T = \begin{bmatrix} 10 & 26\\ 26 & 74 \end{bmatrix}$

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.