سلاسل الخيارات المستمرة: افصل تسميات الصفقات عن عوائد التقييم السوقي

# Constructing Continuous Options Series


# Constructing Continuous Options Series

**Docker image**: `ml4t`

## Purpose

Demonstrate why a "constant-maturity" option series chains together different
contracts, produces phantom price jumps at every roll, and contaminates label
construction. Implement two clean alternatives: same-contract holding returns
(correct labels) and a roll-zeroed continuous reconstruction (backtestable
mark-to-market).

## Learning Objectives

- Detect contract rolls in a 30-day ATM straddle time series and quantify the
  resulting bias against non-roll-day returns.
- Compute same-contract holding returns by looking up the entry contract's
  exit price in the raw option chain.
- Build a continuous price series suitable for backtesting, and measure what the choice of
  roll-day return does to it.
- Tell apart the construction a label needs from the one a mark-to-market series needs.

## Book Reference

§2.2, "The asset-class market data landscape" - the derivatives part of it. The futures
analogue is `06_futures_continuous`; the case-study scale-up is
`case_studies/sp500_options/02_labels`.

## Prerequisites

- `07_sp500_options_eda` for option-chain structure.
- `08_options_greeks_computation` for theta and time decay.
- The AlgoSeek S&P 500 options EDA parquet at `$ML4T_DATA_PATH/sp500_options/`.

```python
"""Constructing Continuous Options Series — constant-maturity roll adjustment."""

import plotly.graph_objects as go
import polars as pl
from plotly.subplots import make_subplots

from data import load_sp500_options_eda
from utils.style import COLORS, show_plotly_with_alt
```

```python
DEMO_SYMBOL = "AAPL"
DEMO_YEAR = 2019

# Constant-maturity selection. These decide which contract the series holds on each day, so
# they belong with the other declared inputs rather than beside the code that applies them.
DTE_WINDOW = (25, 35)
TARGET_DELTA = 0.50
DELTA_TOL = 0.15
MIN_BID = 0.01
MAX_REL_SPREAD = 0.30

# Holding period for the same-contract return, in trading days.
HOLDING_PERIOD = 10
```

## 1. The Constant-Maturity Construction

A 30-day ATM straddle is a common volatility instrument: you buy (or sell) both
an ATM call and an ATM put with ~30 days to expiration. To build a daily time
series, we select the "best" straddle each day — the one closest to 30 DTE and
50-delta.

The problem: this selection is **independent** each day. The contract identity
(strike, expiration) changes whenever a new weekly/monthly option enters the
25–35 DTE window or the underlying moves enough to shift which strike is ATM.

### Loading the raw chains for one symbol

```python
raw = load_sp500_options_eda(
    symbols=[DEMO_SYMBOL],
    start_date=f"{DEMO_YEAR}-01-01",
    end_date=f"{DEMO_YEAR}-12-31",
).rename({"timestamp": "date"})

print(f"Raw {DEMO_SYMBOL} options ({DEMO_YEAR}): {len(raw):,} rows")
print(f"  Dates: {raw['date'].n_unique()}")
print(
    f"  Unique (strike, expiration): {raw.select(pl.struct('strike', 'expiration')).n_unique():,}"
)
```

### Selecting a constant-maturity straddle

For each trading day, select the ATM straddle closest to 30 DTE.
This replicates the logic in `materialize_options.py`.

### Straddle Selection Logic

For each trading day, select the ATM straddle closest to 30 DTE.

```python
def select_constant_maturity_straddle(raw_options: pl.DataFrame) -> pl.DataFrame:
    """Select each day's ATM straddle: one call and one put on the SAME strike and expiration.

    Ranking the two legs independently and joining them on the date alone does not produce a
    straddle. It produces whichever call and whichever put each sat closest to the target, and
    when those differ the result is a strangle or a diagonal - priced, tracked and rolled as
    though it were a single instrument. The legs are therefore paired on strike and expiration
    first, and the ranking is applied to the pair.
    """
    opts = raw_options.with_columns(
        pl.col("delta").abs().alias("abs_delta"),
        ((pl.col("ask") - pl.col("bid")) / pl.col("mid_price").clip(lower_bound=0.01)).alias(
            "rel_spread"
        ),
    )
    filtered = opts.filter(
        pl.col("days_to_maturity").is_between(DTE_WINDOW[0], DTE_WINDOW[1])
        & (pl.col("bid") >= MIN_BID)
        & (pl.col("rel_spread") <= MAX_REL_SPREAD)
        & (pl.col("iv_convergence") == "Converged")
        & pl.col("abs_delta").is_between(TARGET_DELTA - DELTA_TOL, TARGET_DELTA + DELTA_TOL)
    )

    key = ["date", "strike", "expiration"]
    calls = filtered.filter(pl.col("call_put") == "C").select(
        *key,
        "days_to_maturity",
        "underlying_price",
        pl.col("mid_price").alias("call_mid"),
        pl.col("bid").alias("call_bid"),
        pl.col("ask").alias("call_ask"),
        pl.col("delta").alias("call_delta"),
        pl.col("theta").alias("call_theta"),
        pl.col("abs_delta").alias("call_abs_delta"),
    )
    puts = filtered.filter(pl.col("call_put") == "P").select(
        *key,
        pl.col("mid_price").alias("put_mid"),
        pl.col("bid").alias("put_bid"),
        pl.col("ask").alias("put_ask"),
        pl.col("delta").alias("put_delta"),
        pl.col("theta").alias("put_theta"),
    )

    # An inner join on the full key is what makes this a straddle: a strike and expiration
    # survive only if BOTH legs are quoted and pass the filters there.
    pairs = calls.join(puts, on=key, how="inner").with_columns(
        (pl.col("call_mid") + pl.col("put_mid")).alias("instr_mid"),
        (pl.col("call_theta") + pl.col("put_theta")).alias("instr_theta"),
        (pl.col("call_delta") + pl.col("put_delta")).alias("instr_delta"),
        (pl.col("call_abs_delta") - TARGET_DELTA).abs().alias("_delta_gap"),
        (pl.col("days_to_maturity") - sum(DTE_WINDOW) / 2).abs().alias("_dte_gap"),
    )

    return (
        pairs.sort(["date", "_delta_gap", "_dte_gap", "strike"])
        .group_by("date")
        .first()
        .drop(["_delta_gap", "_dte_gap"])
        .sort("date")
    )
```

```python
cm_straddles = select_constant_maturity_straddle(raw)
print(f"Constant-maturity straddle series: {len(cm_straddles)} days")
print(f"  Date range: {cm_straddles['date'].min()} to {cm_straddles['date'].max()}")
cm_straddles.select(["date", "strike", "expiration", "days_to_maturity", "instr_mid"]).head(10)
```

## 2. The Roll Problem

How often does the underlying contract change?

```python
cm_straddles = cm_straddles.with_columns(
    pl.col("expiration").shift(1).alias("prev_exp"),
    pl.col("strike").shift(1).alias("prev_strike"),
    pl.col("instr_mid").shift(1).alias("prev_mid"),
)

cm_straddles = cm_straddles.with_columns(
    (pl.col("expiration") != pl.col("prev_exp")).fill_null(False).alias("expiry_moved"),
    (pl.col("strike") != pl.col("prev_strike")).fill_null(False).alias("strike_moved"),
    ((pl.col("instr_mid") - pl.col("prev_mid")) / pl.col("prev_mid")).alias("daily_return"),
).with_columns(
    (pl.col("expiry_moved") | pl.col("strike_moved")).alias("is_roll"),
    pl.when(pl.col("expiry_moved"))
    .then(pl.lit("expiry changed"))
    .when(pl.col("strike_moved"))
    .then(pl.lit("strike moved, same expiry"))
    .otherwise(pl.lit("same contract"))
    .alias("change_kind"),
)

n_rolls = cm_straddles.filter(pl.col("is_roll")).height
n_total = len(cm_straddles) - 1
print(f"Contract changes: {n_rolls} out of {n_total} days ({n_rolls / n_total:.0%})")
```

### Roll days against non-roll days

If the constant-maturity series were a genuine instrument, days when the contract changed and
days when it did not would have similar return characteristics. They do not.

**But "the contract changed" is two different events.** The selection can move to a different
strike at the same expiration, because spot drifted and a neighbouring strike is now closer
to the money. Or it can move to a different expiration, because a new listing entered the
day-count window. Only the second resets time value, so only the second should produce the
phantom step - and treating them as one category would average a large effect with a small
one and report something in between that describes neither.

```python
by_change = (
    cm_straddles.drop_nulls("daily_return")
    .group_by("change_kind")
    .agg(
        pl.len().alias("days"),
        pl.col("daily_return").mean().alias("mean_return"),
        pl.col("daily_return").median().alias("median_return"),
        (pl.col("daily_return") > 0).mean().alias("pct_positive"),
    )
    .sort("mean_return", descending=True)
)
print("Daily return by what changed in the selection:")
print(by_change)

roll_rets = cm_straddles.filter(pl.col("is_roll"))["daily_return"].drop_nulls()
nonroll_rets = cm_straddles.filter(~pl.col("is_roll"))["daily_return"].drop_nulls()

roll_summary = pl.DataFrame(
    {
        "metric": ["count", "mean_return", "median_return", "pct_positive"],
        "roll_days": [
            float(roll_rets.len()),
            roll_rets.mean(),
            roll_rets.median(),
            (roll_rets > 0).mean(),
        ],
        "non_roll_days": [
            float(nonroll_rets.len()),
            nonroll_rets.mean(),
            nonroll_rets.median(),
            (nonroll_rets > 0).mean(),
        ],
    }
)
roll_summary = roll_summary.with_columns(
    (pl.col("roll_days") - pl.col("non_roll_days")).alias("difference"),
)
roll_summary
```

The split is the result, and only one of the three groups is positive. Expiration changes
carry a large positive mean. Days when only the strike moved and days when nothing changed
are close to each other and both negative, which is ordinary time decay showing through.

The expiration result is what the mechanism predicts. Moving to a later expiration buys back
the time value decay had removed, and the series records the difference as a return no
position earned.

The strike result invites a conclusion it does not support. Moving to a neighbouring strike
at the same expiration exchanges one contract for a similar one with no time value to reset,
and the group mean duly lands beside the group mean for days when nothing changed. That is a
statement about two averages. It is not a statement about the individual days, which is
where a return series is used, and Section 5 measures those directly once the held contract's
own price is available. The answer there is not the one this table suggests.

Lumping all the changes into a single "roll day" average blends one large positive effect
with two indistinguishable negative ones and reports a figure describing none of the three.

What this table does not establish is that strike changes are harmless. Two group means can
agree while the individual days behind them are wrong in both directions, and the quantity
that matters is per-day: the gap between the return the series records and the return the
contract actually held would have earned. That gap needs the held contract's own price,
which Section 5 recovers; the comparison is made there, on the days themselves.

The futures analogue is roll yield, and the difference is one of degree that becomes a
difference in kind. A quarterly futures roll contributes a small adjustment four times a
year. This series reselects on most days, so the phantom component is not an occasional
correction to an otherwise sound return stream - it is a large part of the stream.

**A note on sign, because everything below depends on it.** Every return in this notebook is
the return of the *instrument*: today's price over yesterday's, minus one. Positive means the
straddle got more expensive.

That is the long convention, and it is used here for every series without exception, because
a price series and its returns should mean the same thing in every section. The case study
this feeds sells straddles rather than buying them. A seller's P&L over a *single* period,
before costs and for a fixed quantity, is the negation of the long return printed below.

That does not extend to a cumulative return, and the difference is not small. Negating each
period's return and compounding is a different series from negating the compounded result:
a long series of $+10\%$ then $-10\%$ compounds to $-1\%$, and so does its per-period
negation. A cumulative short return is only defined once the position's sizing is - whether
the quantity is fixed for the whole period or reset against equity each day - so this
notebook prints the long series and states the convention where a short figure is needed.

### The sawtooth

```python
dates = cm_straddles["date"].to_list()
prices = cm_straddles["instr_mid"].to_list()
dtes = cm_straddles["days_to_maturity"].to_list()
rolls = cm_straddles["is_roll"].to_list()

roll_dates = [d for d, r in zip(dates, rolls, strict=False) if r]
roll_prices = [p for p, r in zip(prices, rolls, strict=False) if r]
nonroll_dates = [d for d, r in zip(dates, rolls, strict=False) if not r]
nonroll_prices = [p for p, r in zip(prices, rolls, strict=False) if not r]
```

Both panels are built in one cell so the inline backend does not flush the figure
mid-construction and publish a half-drawn copy above the finished one.

```python
fig = make_subplots(
    rows=2,
    cols=1,
    shared_xaxes=True,
    subplot_titles=[
        f"{DEMO_SYMBOL} constant-maturity straddle mid",
        "Days to expiration",
    ],
    vertical_spacing=0.08,
)

fig.add_trace(
    go.Scatter(
        x=nonroll_dates,
        y=nonroll_prices,
        mode="markers",
        name="Same contract",
        marker=dict(size=4, color=COLORS["blue"]),
        opacity=0.7,
    ),
    row=1,
    col=1,
)
fig.add_trace(
    go.Scatter(
        x=roll_dates,
        y=roll_prices,
        mode="markers",
        name="Contract switch",
        marker=dict(size=6, color=COLORS["negative"], symbol="diamond"),
        opacity=0.9,
    ),
    row=1,
    col=1,
)
fig.add_trace(
    go.Scatter(
        x=dates,
        y=prices,
        mode="lines",
        name="Price",
        line=dict(width=1, color=COLORS["neutral"]),
        showlegend=False,
    ),
    row=1,
    col=1,
)
fig.add_trace(
    go.Scatter(
        x=dates,
        y=dtes,
        mode="lines+markers",
        name="DTE",
        marker=dict(size=3, color=COLORS["positive"]),
        line=dict(width=1),
    ),
    row=2,
    col=1,
)

fig.update_yaxes(title_text="Straddle Mid ($)", row=1, col=1)
fig.update_yaxes(title_text="DTE", row=2, col=1)
fig.update_xaxes(title_text="Date", row=2, col=1)
fig.update_layout(height=600, legend=dict(x=0.01, y=0.99))
show_plotly_with_alt(
    fig,
    "Two stacked panels sharing a date axis across the year. The upper panel plots the daily mid price of the selected straddle as a line with the points marked, distinguishing days when the contract stayed the same from days when it switched; the trace is a dense jagged band with no smooth trend, and almost every point is marked as a contract switch rather than a continuation. The lower panel plots days to expiration, which zigzags rapidly and continuously between the two edges of a narrow band, with no sustained run in either direction.",
)
```



Run lengths below come from the contract identity itself. Encoding runs of an "unchanged"
flag instead drops the first observation of every contract - identities A, B, B, B give two
unchanged flags although B is held for three days - and understates every holding period by
exactly one.

```python
_dte = cm_straddles["days_to_maturity"]
_delta_dte = (_dte - _dte.shift(1)).drop_nulls()
_held_runs = (
    cm_straddles.select("strike", "expiration")
    .with_columns(
        (
            (pl.col("strike") != pl.col("strike").shift(1))
            | (pl.col("expiration") != pl.col("expiration").shift(1))
        )
        .fill_null(True)
        .cum_sum()
        .alias("spell")
    )
    .group_by("spell")
    .agg(pl.len().alias("days"))
)
print(f"Days to expiration stays within {_dte.min()} and {_dte.max()} all year")
print(f"  it falls on {(_delta_dte < 0).sum()} days and rises on {(_delta_dte > 0).sum()}")
print(f"  longest stretch holding one contract: {_held_runs['days'].max()} days")
print(f"  mean stretch: {_held_runs['days'].mean():.2f} days")
```

The teeth of the sawtooth are much finer than the description usually attached to one. Days
to expiration never sweeps the width of the selection window and resets: the longest run on
one contract is a handful of days and the average is under two, so what the panel shows is a
rapid zigzag inside a narrow band rather than a slow decay with occasional jumps.

The direction is still mostly downward, which is decay doing its work, and the upward steps
are the expiration changes identified above. But the scale matters for what follows. A series
that resets every few days is not a lightly contaminated version of a real instrument with a
few bad days in it; the contamination is the majority of its observations, and any adjustment
applied to those days touches most of the sample.

## 3. Same-Contract Holding Returns

The correct return for a straddle trade is: enter a specific contract at mid on
day $t$, exit the **same contract** at mid on day $t + h$. This requires
looking up that specific contract in the raw option chain $h$ days later.

$$r_{same} = \frac{P_{exit}^{mid}(K, T) - P_{entry}^{mid}(K, T)}{P_{entry}^{mid}(K, T)}$$

where $(K, T)$ identifies the specific strike and expiration.

### An exit-price lookup from the raw chain

```python
def build_exit_lookup(raw_options: pl.DataFrame) -> pl.DataFrame:
    """Build a lookup table: (date, strike, expiration, call_put) → mid_price.

    Used to find the exit price of a specific contract h days after entry.
    """
    return raw_options.filter(
        pl.col("bid") >= MIN_BID,
        pl.col("iv_convergence") == "Converged",
    ).select(
        [
            "date",
            "strike",
            "expiration",
            "call_put",
            "mid_price",
            "bid",
            "ask",
            "delta",
            "theta",
            "days_to_maturity",
        ]
    )
```

```python
# Build lookup from the raw chain
lookup = build_exit_lookup(raw)
print(f"Exit price lookup: {len(lookup):,} rows")
print(
    f"  Unique contracts: {lookup.select(pl.struct('strike', 'expiration', 'call_put')).n_unique():,}"
)
```

### Computing same-contract returns

For each day's constant-maturity straddle, find the same contract's price
$h$ trading days later in the raw chain.

```python
# Get the trading calendar (ordered dates)
trading_dates = cm_straddles["date"].unique().sort().to_list()
date_to_idx = {d: i for i, d in enumerate(trading_dates)}


def get_exit_date(entry_date, h: int, cal: list):
    """Get the trading date h business days after entry_date."""
    idx = date_to_idx.get(entry_date)
    if idx is None or idx + h >= len(cal):
        return None
    return cal[idx + h]


# Build entry-exit pairs
entries = cm_straddles.select(
    [
        "date",
        "strike",
        "expiration",
        "instr_mid",
        "call_mid",
        "put_mid",
        "instr_delta",
        "is_roll",
    ]
).rename({"date": "entry_date", "instr_mid": "entry_mid"})

# Add exit dates
exit_dates = [
    get_exit_date(d, HOLDING_PERIOD, trading_dates) for d in entries["entry_date"].to_list()
]
entries = entries.with_columns(pl.Series("exit_date", exit_dates).cast(pl.Date))

# Drop entries where exit date is beyond our data
entries = entries.filter(pl.col("exit_date").is_not_null())
print(f"Entry-exit pairs to look up: {len(entries):,}")
```

```python
# Look up call exit price
call_exit = lookup.filter(pl.col("call_put") == "C").select(
    [
        pl.col("date").alias("exit_date"),
        "strike",
        "expiration",
        pl.col("mid_price").alias("call_exit_mid"),
    ]
)

# Look up put exit price
put_exit = lookup.filter(pl.col("call_put") == "P").select(
    [
        pl.col("date").alias("exit_date"),
        "strike",
        "expiration",
        pl.col("mid_price").alias("put_exit_mid"),
    ]
)

# Join: entry contract → exit prices for same (strike, expiration)
same_contract = entries.join(call_exit, on=["exit_date", "strike", "expiration"], how="left").join(
    put_exit, on=["exit_date", "strike", "expiration"], how="left"
)

# Compute same-contract straddle exit mid
same_contract = same_contract.with_columns(
    (pl.col("call_exit_mid") + pl.col("put_exit_mid")).alias("exit_mid"),
)

same_contract = same_contract.with_columns(
    ((pl.col("exit_mid") - pl.col("entry_mid")) / pl.col("entry_mid")).alias("same_contract_ret"),
)

# How many lookups succeeded?
found = same_contract.filter(pl.col("exit_mid").is_not_null()).height
print(f"Exit prices found: {found} / {len(same_contract)} ({found / len(same_contract):.1%})")
```

### Naive against same-contract

The naive return is the one a `shift` on the constant-maturity series produces. For the
comparison to isolate the construction, both sides must cover the same window: the
same-contract return runs from the selection day to `HOLDING_PERIOD` days later, so the naive
one has to as well. Entering a day later, as a label-shifted version would, means the two
series differ partly because they cover different days and partly because of the roll, and
the comparison could not separate them.

```python
naive_rets = cm_straddles.with_columns(
    pl.col("instr_mid").shift(-HOLDING_PERIOD).alias("naive_exit"),
).with_columns(
    ((pl.col("naive_exit") - pl.col("instr_mid")) / pl.col("instr_mid")).alias("naive_ret"),
)

# Align for comparison
comparison = (
    same_contract.filter(pl.col("same_contract_ret").is_not_null())
    .select(["entry_date", "same_contract_ret"])
    .join(
        naive_rets.select([pl.col("date").alias("entry_date"), "naive_ret"]).drop_nulls(),
        on="entry_date",
        how="inner",
    )
)

print(f"Paired observations: {len(comparison):,}")

naive_vs_same = pl.DataFrame(
    {
        "metric": ["mean_return", "std", "median_return", "pct_positive"],
        "naive_chained": [
            comparison["naive_ret"].mean(),
            comparison["naive_ret"].std(),
            comparison["naive_ret"].median(),
            (comparison["naive_ret"] > 0).mean(),
        ],
        "same_contract": [
            comparison["same_contract_ret"].mean(),
            comparison["same_contract_ret"].std(),
            comparison["same_contract_ret"].median(),
            (comparison["same_contract_ret"] > 0).mean(),
        ],
    }
)
corr = comparison.select(pl.corr("naive_ret", "same_contract_ret").alias("correlation")).item()
print(f"Correlation(naive, same-contract) = {corr:.3f}")
naive_vs_same
```

The correlation is the number to read, and it is low. Squaring it gives the share of variance
the naive series shares with the return an actual trade would have earned, and that share is
small: most of what the naive series moves on is not the trade.

The two are computed over the same days, on the same instrument, for the same holding period.
The only difference between them is that one follows the contract it entered and the other
follows whatever the selection rule picked up along the way. A label built from the second is
mostly a record of the selection rule.

```python
# Scatter plot: naive vs same-contract
fig = go.Figure()
fig.add_trace(
    go.Scatter(
        x=comparison["naive_ret"].to_list(),
        y=comparison["same_contract_ret"].to_list(),
        mode="markers",
        marker=dict(size=3, color=COLORS["blue"], opacity=0.4),
        name="Returns",
    )
)
fig.add_trace(
    go.Scatter(
        x=[-0.5, 0.5],
        y=[-0.5, 0.5],
        mode="lines",
        line=dict(dash="dash", color=COLORS["neutral"]),
        name="45° line",
    )
)
fig.update_layout(
    title=f"{DEMO_SYMBOL}: same-contract against naive chained returns",
    xaxis_title="Naive (chained constant-maturity)",
    yaxis_title="Same-contract holding return",
    width=600,
    height=500,
)
show_plotly_with_alt(
    fig,
    "A scatter of the same-contract holding return against the naive chained return over the same days, with a dashed forty-five degree line. The cloud is broad and only loosely oriented along that line.",
)
```

## 4. Building a Continuous Series for Backtesting

For backtesting we need a price series whose daily returns are actual P&L, with no phantom
jumps at rolls. Futures roll quarterly and suit Panama adjustment; straddles reselect on most
days, so the adjustment has to be return-based:

1. On a non-roll day, the daily return is the held contract's own move. Uncontroversial.
2. On a roll day, the naive-looking answer is to set the return to zero, on the grounds that a
   roll is a transaction rather than a return.
3. Reconstruct prices from whichever returns you chose: $P_{adj,t} = P_0 \prod (1 + r_i)$.

**Step 2 is wrong, and the section measures how wrong.** On a roll day you did hold a
position: yesterday's contract, from yesterday's close to today's. It moved with the market,
and that move is real P&L. Zeroing it does not remove the phantom jump alone - it removes the
day's genuine return along with it.

The correct return for a roll day is the *held* contract's return: yesterday's strike and
expiration, priced yesterday and today. Section 3 already built the raw-chain lookup that
makes this available, so it costs nothing but the lookup.

Both are computed below and the difference is reported. Roll costs, the bid-ask actually paid
to switch, are a separate matter and are modelled as transaction costs in Chapter 18.

```python
def held_contract_returns(cm_series: pl.DataFrame, raw_options: pl.DataFrame) -> pl.DataFrame:
    """Return each day's move of the contract actually held coming into that day.

    On a non-roll day that is the same contract the series already shows. On a roll day it is
    yesterday's strike and expiration, repriced from the raw chain at today's close.
    """
    previous = cm_series.select(
        "date",
        pl.col("date").shift(1).alias("prev_date"),
        pl.col("strike").shift(1).alias("prev_strike"),
        pl.col("expiration").shift(1).alias("prev_expiration"),
        pl.col("instr_mid").shift(1).alias("prev_mid"),
    ).drop_nulls("prev_date")

    calls = raw_options.filter(pl.col("call_put") == "C").select(
        "date", "strike", "expiration", pl.col("mid_price").alias("call_now")
    )
    puts = raw_options.filter(pl.col("call_put") == "P").select(
        "date", "strike", "expiration", pl.col("mid_price").alias("put_now")
    )

    return (
        previous.join(
            calls,
            left_on=["date", "prev_strike", "prev_expiration"],
            right_on=["date", "strike", "expiration"],
            how="left",
        )
        .join(
            puts,
            left_on=["date", "prev_strike", "prev_expiration"],
            right_on=["date", "strike", "expiration"],
            how="left",
        )
        .with_columns(
            ((pl.col("call_now") + pl.col("put_now")) / pl.col("prev_mid") - 1).alias("held_ret")
        )
        .select("date", "held_ret")
    )


def build_continuous_straddle_series(cm_series: pl.DataFrame, held: pl.DataFrame) -> pl.DataFrame:
    """Rebuild a straddle price series two ways, so the roll-day choice can be compared.

    ``zeroed_daily_ret`` sets roll days to zero. ``held_daily_ret`` uses the return of the
    contract that was actually held into the roll. They agree on every non-roll day.
    """
    df = cm_series.join(held, on="date", how="left").with_columns(
        (pl.col("instr_mid") / pl.col("instr_mid").shift(1) - 1).alias("raw_daily_ret"),
    )

    df = df.with_columns(
        pl.when(pl.col("is_roll"))
        .then(0.0)
        .otherwise(pl.col("raw_daily_ret"))
        .alias("zeroed_daily_ret"),
        pl.when(pl.col("is_roll"))
        .then(pl.col("held_ret"))
        .otherwise(pl.col("raw_daily_ret"))
        .alias("held_daily_ret"),
    )

    start_price = cm_series["instr_mid"][0]
    for source, target in [
        ("zeroed_daily_ret", "price_zeroed"),
        ("held_daily_ret", "price_held"),
    ]:
        prices = [start_price]
        for r in df[source].fill_null(0.0).to_list()[1:]:
            prices.append(prices[-1] * (1.0 + r))
        df = df.with_columns(pl.Series(target, prices))

    return df
```

```python
held = held_contract_returns(cm_straddles, raw)
adjusted = build_continuous_straddle_series(cm_straddles, held)

adj_roll = adjusted.filter(pl.col("is_roll"))
_found = adj_roll["held_ret"].drop_nulls().len()
print(f"Roll days: {adj_roll.height} of {adjusted.height}")
print(f"  held contract found in the raw chain on {_found} of them")
print(
    f"  phantom return the raw series shows on those days: {adj_roll['raw_daily_ret'].mean():+.4f}"
)
print(f"  return the held contract actually earned:          {adj_roll['held_ret'].mean():+.4f}")
```

The held contract is recoverable on every roll day, so there is no data reason to discard the
day. And the return being discarded is not noise around zero: on average the position held
into a roll made money, so zeroing those days does not remove a bias, it introduces one.

This is also where the question Section 2 left open can be answered. Splitting the roll days
by what changed showed strike-only days averaging close to the days where nothing changed,
which says the two groups have similar means and nothing more. The construction error is a
per-day quantity - what the series recorded minus what the contract held would have earned -
and it can be large on individual days while averaging to little.

```python
_error_by_kind = (
    adjusted.filter(pl.col("is_roll"))
    .drop_nulls("held_ret")
    .with_columns((pl.col("raw_daily_ret") - pl.col("held_ret")).alias("construction_error"))
    .group_by("change_kind")
    .agg(
        pl.len().alias("days"),
        pl.col("construction_error").mean().alias("mean_error"),
        pl.col("construction_error").abs().mean().alias("mean_abs_error"),
        pl.col("construction_error").abs().max().alias("worst_abs_error"),
    )
    .sort("mean_abs_error", descending=True)
)
print("Per-day construction error, raw series minus the contract actually held:")
print(_error_by_kind)
```

Strike-only days are not clean. Their mean error is several percentage points, and the error
is defined as the raw series minus the held contract, so the sign says which way each group
is wrong on average: expiration days come out positive, meaning the raw series records more
than the position earned, and strike-only days come out negative, meaning it records less.
The two biases point in opposite directions and neither is small.

The mean absolute error exceeds the absolute mean in both groups, which says the individual
days are not all wrong in their group's direction - the average is a net of errors both
ways, and the worst single day in each group is tens of percentage points. A bias that
happens to be modest in the average can still be large on the day a label is taken.

So the two group means in Section 2 agreed for a reason that has nothing to do with the
series being right on those days. A day on which the strike moved records a return that
belongs partly to a contract nobody held, and the resemblance between that group's average
and the do-nothing group's average is a coincidence of aggregation. Comparing averages of a
quantity cannot establish that the quantity is correct case by case; only the case-by-case
comparison can, and here it fails.

```python
_zeroed_total = (1 + adjusted["zeroed_daily_ret"].fill_null(0.0)).product() - 1
_held_total = (1 + adjusted["held_daily_ret"].fill_null(0.0)).product() - 1
_short_held_total = (1 - adjusted["held_daily_ret"].fill_null(0.0)).product() - 1
print(f"Cumulative return of holding the straddle long over {DEMO_YEAR}:")
print(f"  zeroing roll-day returns:         {_zeroed_total:+.1%}")
print(f"  using the held contract's return: {_held_total:+.1%}")
print(f"  the two series differ by:         {_held_total - _zeroed_total:+.1%}")
print(f"Same contracts, same days, held over {DEMO_YEAR}, sold rather than bought:")
print(f"  short series, notional reset against equity each day: {_short_held_total:+.1%}")
print(f"  what negating the long cumulative return would say:  {-_held_total:+.1%}")
```

The short figure is constructed here, by negating each day's held-contract return and
compounding the result over the same year. That is the P&L of a position whose notional is
reset against equity every day, and it has to be named because the convention decides the
answer.

Under *this* convention the short return is not the negation of the long one, and the two
printed numbers show how far from it: both lose money over the year, so a sign flip gets
the direction wrong and not merely the size. Both series reset exposure against their own
equity each day, and two paths that rebalance separately do not stay mirror images.

A different convention gives a different answer, and one of them gives exactly the
negation: holding the opposite quantity of the same contracts every day, never
rebalancing, produces the opposite dollar P&L before costs, which on the same initial
capital is the long return negated. The lesson is not that the negation is wrong in
general. It is that a cumulative short return is undefined until the sizing is stated, and
negating a compounded long return silently assumes one particular sizing that the series
above does not use.

The two roll-convention series part company by tens of percentage points of cumulative
return over a single year, from the same contracts, the same days and the same prices. Only the roll-day
convention separates them. On a series whose stated purpose is to be backtested, that is the
difference between two materially different answers about the same strategy.

Zeroing is the more conservative-looking choice and it is not the safer one. It deletes a
real return on most of the sample, because the contract changes on the majority of days, and
the deleted returns do not average to zero - so it is a one-sided subtraction repeated on the
majority of observations, not noise that cancels. Here it makes the instrument look worse
than it was.

The general form of the error is worth naming, because it is not specific to straddles. A
roll is two facts at once - the instrument changed, and the market moved - and an adjustment
that suppresses the day suppresses both. Futures get away with the crude version because
quarterly rolls make it four days a year. Here it is most of them.

```python
# Visualize raw vs adjusted price series
fig = make_subplots(
    rows=2,
    cols=1,
    shared_xaxes=True,
    subplot_titles=[
        "Constant-maturity straddle mid",
        "Reconstructed continuous price, two roll conventions",
    ],
    vertical_spacing=0.08,
)

fig.add_trace(
    go.Scatter(
        x=adjusted["date"].to_list(),
        y=adjusted["instr_mid"].to_list(),
        mode="lines",
        name="Raw",
        line=dict(color=COLORS["negative"], width=1.5),
    ),
    row=1,
    col=1,
)
fig.add_trace(
    go.Scatter(
        x=adjusted["date"].to_list(),
        y=adjusted["price_zeroed"].to_list(),
        mode="lines",
        name="Roll days zeroed",
        line=dict(color=COLORS["amber"], width=1.5, dash="dot"),
    ),
    row=2,
    col=1,
)
fig.add_trace(
    go.Scatter(
        x=adjusted["date"].to_list(),
        y=adjusted["price_held"].to_list(),
        mode="lines",
        name="Held contract's return",
        line=dict(color=COLORS["blue"], width=1.5),
    ),
    row=2,
    col=1,
)

fig.update_yaxes(title_text="Straddle Mid ($)", row=1, col=1)
fig.update_yaxes(title_text="Reconstructed price ($)", row=2, col=1)
fig.update_layout(
    height=560,
    title=f"{DEMO_SYMBOL} straddle: selected price and two reconstructions",
    legend=dict(orientation="h", yanchor="bottom", y=-0.16, x=0),
)
show_plotly_with_alt(
    fig,
    "Two stacked panels sharing a date axis. The upper panel shows the selected straddle's mid price with its repeated sawtooth. The lower panel shows two reconstructed price series that start together and separate steadily through the year, the roll-zeroed one falling further than the one built from the held contract's returns, with the gap widening throughout.",
)
```

Both reconstructions remove the sawtooth, which is what the adjustment was for, and they
separate from each other steadily over the year because they disagree on most days. The
zeroed series is the lower of the two and its gap widens monotonically: every roll day
contributes another deleted return.

The held-contract series is the one to backtest. Roll costs, meaning the bid-ask actually
crossed to switch contracts, are a separate deduction and belong with transaction costs in
Chapter 18 rather than inside the price series.

For **labels**, neither reconstruction is the right tool. A label is the return of a trade
someone could have placed and held, which is the same-contract calculation in Section 3.

## 5. Summary: Three Approaches Compared

| Approach | Mechanism | Suitable For |
|----------|-----------|--------------|
| Naive (chained) | `shift(-h)` on the constant-maturity series | Nothing. It contaminates labels |
| Same-contract | Look up the entry contract h days later | Labels, being per-trade P&L |
| Continuous, held-contract | Roll days take the held contract's return | Backtesting mark-to-market |
| Continuous, roll-zeroed | Roll days take zero | Nothing. It deletes real P&L |

Three of the four need the raw option chain; only roll-zeroing avoids it, and that is the
whole of its appeal. On a series that reselects quarterly the shortcut is defensible. On one
that reselects most days it costs the majority of the sample's return, which the section
above measures rather than argues.

The final table puts the forward return from each construction side by side over the same
days.

```python
adjusted = adjusted.with_columns(
    pl.col("instr_mid").shift(-HOLDING_PERIOD).alias("raw_exit"),
).with_columns(
    ((pl.col("raw_exit") - pl.col("instr_mid")) / pl.col("instr_mid")).alias("naive_fwd_ret"),
)

# Continuous-adjusted
adjusted = adjusted.with_columns(
    pl.col("price_held").shift(-HOLDING_PERIOD).alias("adj_exit"),
).with_columns(
    ((pl.col("adj_exit") - pl.col("price_held")) / pl.col("price_held")).alias("cont_fwd_ret"),
)

# Merge with same-contract returns
comparison_all = (
    adjusted.select([pl.col("date").alias("entry_date"), "naive_fwd_ret", "cont_fwd_ret"])
    .join(
        same_contract.select(["entry_date", "same_contract_ret"]),
        on="entry_date",
        how="left",
    )
    .drop_nulls()
)

print(f"Paired observations: {len(comparison_all):,}")

three_way = pl.DataFrame(
    {
        "metric": ["mean", "std", "corr_with_same_contract"],
        "naive": [
            comparison_all["naive_fwd_ret"].mean(),
            comparison_all["naive_fwd_ret"].std(),
            comparison_all.select(pl.corr("naive_fwd_ret", "same_contract_ret")).item(),
        ],
        "continuous": [
            comparison_all["cont_fwd_ret"].mean(),
            comparison_all["cont_fwd_ret"].std(),
            comparison_all.select(pl.corr("cont_fwd_ret", "same_contract_ret")).item(),
        ],
        "same_contract": [
            comparison_all["same_contract_ret"].mean(),
            comparison_all["same_contract_ret"].std(),
            1.0,
        ],
    }
)
three_way
```

## Key Takeaways

1. **A constant-maturity series is not an instrument.** The rule picks whichever straddle
   sits closest to the target each day, independently, so the contract identity changes on
   most days of the year. Each change resets time value and prints a price step that no position
   experienced.

2. **The roll contamination is systematic and it is large.** Roll days and non-roll days have
   means of opposite sign, and both are printed above rather than quoted here. The futures
   analogue is roll yield, but a quarterly roll contributes four such days a year and this
   series contributes most of them, which turns a correction into the bulk of the signal.
   Splitting the roll days by what changed is worth doing, and it is worth not over-reading:
   the strike-only group's mean sits beside the do-nothing group's, yet its per-day
   construction error against the held contract averages several percentage points, in the
   opposite direction from the error on expiration days. Agreement between two averages says
   nothing about whether the individual values are right.

3. **A short position's cumulative return is not the long's negation.** Per period and for a
   fixed quantity it is, which is why the notebook prints one convention and states the other.
   Compounding breaks the symmetry: over the demo year the long series and the daily-reset
   short series are both negative, so reading the seller's result off a sign flip gets the
   direction wrong, not merely the magnitude.

4. **The construction error is a per-day quantity.** What the series records minus what the
   contract actually held earned is the thing that reaches a label or a backtest, and it is
   measured day by day in Section 5 rather than inferred from group averages.

5. **Same-contract returns are the correct label.** Pricing the entry contract in the raw
   chain h days later gives the P&L of a trade someone could have placed. The correlation
   between that and the naive chained return is low enough that a label built from the naive
   series is mostly a record of the selection rule.

6. **Zeroing roll-day returns deletes real P&L.** On a roll day you held yesterday's
   contract, and it moved; that move is a return, not a transaction. The held contract is
   recoverable from the raw chain on every roll day here, and over a single year the two
   reconstructions differ by tens of percentage points of cumulative return from identical
   prices. The discarded returns do not average to zero, so the error is one-sided and it
   compounds over the majority of the sample. The continuous series should take the held
   contract's return.

7. **Roll costs are transaction costs, not returns.** The bid-ask actually crossed to switch
   contracts is execution cost, accounted in Chapter 18 rather than inside the price series.
   That is what a roll-day adjustment is for; it is not a licence to delete the day.

## Next

The S&P 500 options case study
([`02_labels`](../case_studies/sp500_options/02_labels.ipynb)) builds its labels with the
same-contract construction from Section 3, priced from the raw chain, across the full
universe and a multi-year sample. It is a label pipeline rather than a backtest, so it has no
continuous mark-to-market series and does not need one.
![notebook output](figures/p1_1.png)
![notebook output](figures/p1_2.png)
![notebook output](figures/p1_3.png)