先物リターンを対象とした分割別の確率的割引ファクター

# CME Futures: Stochastic Discount-Factor Features


# CME Futures: Stochastic Discount-Factor Features

The stochastic discount-factor model learns fold-scoped latent factors from the product panel and
maps them to each declared forward-return horizon. Training rows determine the representation;
validation rows are transformed without refitting. The fitted model, fold identity, prediction
shard, and eligible validation keys are persisted together.

The notebook executes the declared SDF configurations and publishes their catalog rows. IC remains
diagnostic. The equal-weight validation backtest in `13_backtest` selects configurations.

One framing to carry through the rest: this is the most theoretically motivated model in the
case study and it is given no special standing because of that. Its rows enter the same funnel
as the linear model's, are selected on the same statistic, and can lose to a gradient-boosting
configuration that rests on no asset-pricing argument at all. A better story about why a model
should work is not evidence that it does.

Prerequisites: `03_financial_features`, `04_model_based_features`, and `05_evaluation`.

## What a stochastic discount factor is

Asset pricing has one central result worth stating plainly, because everything this notebook
does follows from it. If prices admit no arbitrage, then there exists a single random variable
- call it `m` - such that for every asset, the expected product of `m` and that asset's return
is the same constant. One object prices everything: equities, bonds, corn futures, all of it.
That object is the **stochastic discount factor**.

Its usefulness is that it converts "which assets earn more, and why" into a question about one
quantity. An asset earns a premium when its return covaries negatively with `m` - when it pays
badly in the states `m` says are expensive, which are the states investors care most about
being protected in. Under that view a risk premium is not compensation for variance; it is
compensation for failing to pay off when payment matters.

`m` is not observable. But it is a well-defined thing to estimate, and estimating it with a
neural network means not having to assume in advance what functional form it takes - which
matters, because the classical models that assume one (a market factor, a small set of
characteristics entering linearly) have a long record of being rejected on the data.

## How this differs from `10a`, on two axes rather than one

The index for this stage frames the two configurations as differing in objective. That is true
and it is not the whole difference - they also see different data, which is worth being exact
about because it changes what each is capable of finding.

**PCA sees returns only.** `run_pca_fold` takes the characteristics panel and discards it with
`del` before fitting; it is handed a matrix of product returns and nothing else.

**The SDF sees returns and characteristics.** `run_sdf_fold` passes `chars_train` through to
`_cross_section_batch(chars_train, returns=returns_train)`, and the number of instruments it
builds is derived from the characteristics' width. So carry, momentum and volatility are
inputs here in a way they are not in `10a`.

The two differences compound. PCA asks which directions explain the most variation in returns,
using returns alone. The SDF asks which combination of assets, weighted by their observable
characteristics, best explains the cross-section of returns - so it can express "products with
high carry and low volatility load on this factor" in a way PCA structurally cannot, because
PCA never sees carry.

That is why the comparison between them is informative rather than decorative. A win for the
SDF is evidence that the characteristics carry pricing information beyond what the return
covariance already encodes. A win for PCA is evidence that they do not, and that the extra
freedom bought overfitting.

```python
"""Fit the declared CME futures stochastic discount-factor population."""

import polars as pl

from case_studies.cme_futures.research_workflow import (
    ALL_LABELS,
    model_request_catalog,
    open_study,
    product_universe_table,
    resolve_model_requests,
    resolved_model_plan,
    run_official_model_catalog,
    run_resolved_model_requests,
)
```

```python
EXECUTION_TIER = "canonical"
WORKSPACE: str | None = None
PREVIEW_REDUCTIONS: dict = {}
# The population hash this run replaces, read from the registry and set by a person. A
# first population takes None; a re-run whose membership has changed is refused without
# the hash it supersedes, and the refusal names the value required.
SUPERSEDES_POPULATION: str | None = None
```

## Declared requests

Both return horizons use the named stochastic discount-factor configuration. The resolved plan
shows the eligible rows, folds, feature count, checkpoint schedule, and identity before fitting.

**This model publishes on `cuda`, declared in `setup.yaml` rather than detected.** The device
enters both `runtime` and `numerical_runtime` inside the hashed computation, so leaving it to
`preferred_latent_device()` would resolve one training identity on a GPU host and a different one
on a CPU host, both publishing under this population's name. `10a_pca` overrides the same
declaration to `cpu`, because PCA has no GPU implementation to record.

**Nothing here fails on a fit that did not converge.** The shared runner enforces convergence for
IPCA only, and this case study does not declare IPCA; for the stochastic discount factor it
records the training history and the terminal Sharpe as fold extras and checks neither. That is a
gap in shared code rather than in this notebook, so what is claimed below is that every declared
checkpoint was produced and registered - not that the objective had settled when it was.

```python
study = open_study(execution_tier=EXECUTION_TIER, workspace=WORKSPACE)
requests = model_request_catalog(
    "latent_factors",
    labels=ALL_LABELS,
    config_names=("sdf",),
)
resolved = resolve_model_requests(
    study,
    requests,
    execution_tier=EXECUTION_TIER,
    preview_reductions=PREVIEW_REDUCTIONS,
)
universe = product_universe_table()
universe
```

```python
resolved_model_plan(resolved)
```

## Execute and validate

The shared latent-factor runner fits each representation inside its training fold, persists the
fitted state, and requires the complete validation key set before publication.

### Why checkpoints are part of what gets selected

A neural fit is not one model, it is a trajectory. It passes through a sequence of states as
training proceeds, and which one is kept is a decision with the same standing as the choice of
architecture. `checkpoint_epochs` declares the epochs this configuration publishes, so each
becomes its own candidate row, and `13_backtest` selects among them on validation backtest
Sharpe like any other configuration.

Publishing them rather than picking one is the honest arrangement. Choosing the best epoch by
looking at validation performance and then reporting that model's validation performance is
selection inside the number being reported, and it does not stop being that because the choice
was made by hand. Declaring the checkpoints up front puts the choice into the same funnel and
the same trial count as everything else - which the deflated Sharpe downstream then divides by.

This is the concrete contrast with `10a`, which has one fitted state per fold and nothing to
checkpoint: its solution is closed-form, so there is no trajectory to choose a point on.

### What the fold discipline protects

Training rows determine the representation and validation rows are transformed without
refitting. For a model that reads characteristics as well as returns, that discipline covers
two channels rather than one: the factors must not be shaped by validation returns, and the
instrument weights must not be shaped by validation characteristics either. Both would produce
predictions that look ordinary and a backtest that runs.

### Why the complete key set is required

The same reason it is required in `10a`, and for the same structural cause: this is a
cross-sectional model, so a missing product does not leave one prediction absent - it changes
the estimated factors and therefore every prediction the fold publishes. An incomplete key set
is refused rather than published short.

### Where the freedom cuts against it

The SDF has far more capacity than the eigenvectors of a covariance matrix, and this panel is
thirty products. Capacity on a panel that size is as likely to fit noise as structure, and
nothing in a training loss distinguishes the two. That is what makes `10a` the baseline worth
beating rather than a formality - and what makes a narrow SDF win, if one appears, weaker
evidence than its margin suggests.

```python
if EXECUTION_TIER == "canonical":
    execution, population = run_official_model_catalog(
        study,
        requests,
        population_name="cme_futures-sdf-validation-v1",
        resolved_requests=resolved,
        supersedes=SUPERSEDES_POPULATION,
    )
else:
    if WORKSPACE is None or not PREVIEW_REDUCTIONS:
        raise ValueError("preview execution requires WORKSPACE and PREVIEW_REDUCTIONS")
    execution = run_resolved_model_requests(study, resolved)
    population = None
```

```python
catalog = execution.catalog_rows.select(
    "family",
    "label",
    "config_name",
    "checkpoint_kind",
    "checkpoint_value",
    "execution_tier",
    "complete",
    "training_hash",
    "prediction_hash",
).sort("label", "checkpoint_value")
if catalog.filter(~pl.col("complete")).height:
    raise RuntimeError("stochastic discount-factor execution returned a partial prediction")
catalog
```