Using a Kalman Filter to Estimate a Time-Varying Pairs Hedge Ratio

# Kalman Filter Implementation using PyKalman


# Kalman Filter Implementation using PyKalman












I am trying to apply a simple Kalman filter to pair trading. My underlying stock pair is cointegrated with no constant term. Can someone kindly advise if i am going on the right track with the Kalman filter?

```
from pykalman import KalmanFilter
import numpy as np
import pandas as pd

# Log prices of stock_1 and stock_2
stock_1_price = np.log(df_combined[f"Close_{stock_1}"])  # independent variable
stock_2_price = np.log(df_combined[f"Close_{stock_2}"])  # dependent variable

# Transition matrix for beta (1x1 identity matrix because we're only estimating beta)
transition_matrix = np.eye(1)  # 1x1 identity matrix

# Observation and transition covariances
observation_covariance = 1  # Observation noise, can be tuned or optimized
transition_covariance = np.array([[0.001]])  # Process noise covariance (small to keep beta stable)

# Initial estimates
initial_state_mean = np.array([0])  # Initial guess for beta
initial_state_covariance = np.array([[1]])  # Initial covariance

# Set up the Kalman Filter
kf = KalmanFilter(
    transition_matrices=transition_matrix,
    initial_state_mean=initial_state_mean,
    initial_state_covariance=initial_state_covariance,
    observation_covariance=observation_covariance,
    transition_covariance=transition_covariance
)

state_means = np.zeros(len(stock_2_price))  # beta over time
state_covariances = np.zeros(len(stock_2_price))

state_mean = initial_state_mean
state_covariance = initial_state_covariance

# Run the Kalman filter manually in a loop
for t in range(len(stock_2_price)):
    observation_matrix = np.array([[stock_1_price[t]]])  # stock_1_price for beta
    state_mean, state_covariance = kf.filter_update(
        state_mean,
        state_covariance,
        observation_matrix=observation_matrix,
        observation=stock_2_price[t]
    )
    state_means[t] = state_mean[0]
    state_covariances[t] = state_covariance[0, 0]

# Extract the hedge ratios (beta)
hedge_ratios = state_means  # beta values over time

# Calculate the spread using the estimated hedge ratios
kalman_spread = stock_2_price - hedge_ratios * stock_1_price
plt.plot(kalman_spread)
```
```

## Answer by Sane (score 1)

https://quant.stackexchange.com/a/80487

A few considerations:

In your code, the `observation_matrix` is a 1x1 matrix based on `stock_1_price[t]`, which is incorrect. For the Kalman filter to work correctly, you need to ensure that the `observation_matrix` and `observation` are correctly sized. In your case, since you're estimating a single `beta`, the observation matrix should be a 1-dimensional array rather than a 2D matrix.

The `kf.filter_update` method is typically used to update the state estimate in each iteration. However, this method is meant to be used within the Kalman filter's internal loop, and you should use the `kf.filter` method for the entire time series data.

You should ensure that the dimensions and initializations of `state_mean` and `state_covariance` match the expected dimensions of your state vector.