آزمون پوشش‌های ریسک خروج زودهنگام در استراتژی‌های سهام US

# US equities panel: rules that close a position early


# US equities panel: rules that close a position early

The two notebooks before this one decided which stocks to hold and how much to put in each. Both
hold every position from one rebalancing date to the next, whatever it does in between. A **risk
overlay** is a rule that can close a position before then.

Three kinds are declared, and they differ in what they watch:

- A **stop loss** closes a position that has lost more than a fixed fraction from where it was
  opened. It watches the loss from entry.
- A **trailing stop** closes a position that has fallen more than a fixed fraction from its own
  best level. It watches the give-back, so a position that rose and then reversed is closed even
  while it is still ahead of entry.
- A **time exit** closes a position after a fixed number of bars whatever it has done. It watches
  nothing about the price, which makes it the control: it tests whether the holding period itself
  was the problem, separately from any threshold.

**An overlay only ever removes.** It cannot enter a position the strategy did not take, so it
can only cut a loss short or cut a gain short, and which of the two it does more of is exactly
what the sweep measures. Fourteen controls are declared across the three kinds, each kind swept
from its tightest declared threshold to its loosest, so the sweep says how the effect moves with
the threshold rather than whether one chosen threshold helped.

**The thing to check first is whether the overlays changed anything at all.** A control that
never fires returns the unprotected book unchanged in every digit, and so does a control that was
declared in one shape and read by the engine in another. One result cannot tell those apart: both
produce a row identical to the book it was laid on. Fourteen different rules, spanning the
tightest declared stop to the longest declared time exit, all declining to act on one book and
agreeing to the last digit would be the second and not the first, and it would read on the page
as a finding about risk control.

So the results section compares each overlay against the strategy it was laid on rather than
reporting its performance alone. **A difference proves the control acted; matching statistics
prove nothing about whether it fired** - a stop can close a position the next rebalance would
have closed anyway and leave both numbers where they were. Matching across *every* declared
setting is a reason to confirm the controls reach the engine, not a conclusion.

**Learning objectives.** By the end of this notebook you will be able to:

- Describe what each of the three kinds of overlay watches, and say which one is the control and
  why.
- Say why an overlay can only remove, and what that implies about how it can change a return
  distribution.
- Say what a result matching its unprotected baseline on two summary statistics does and does
  not establish about whether a control fired, and what evidence would settle it.
- Say why a threshold sweep is more informative than a single chosen threshold.

**Book reference**: Chapter 19, Sections 19.3 to 19.6.

**Prerequisites**: [`16_backtest`](16_backtest.ipynb) and
[`17_portfolio_management`](17_portfolio_management.ipynb) have frozen the baseline and
allocation sets this notebook draws from.

**What it writes**: one validation backtest per fixed configuration and declared control, in
`run_log/registry.db`, frozen as one named risk-overlay set per label, plus the union of the
three stages as the population validation selection is made over.
[`19_costs`](19_costs.ipynb) then charges the chosen strategy for trading.

```python
"""Generate risk overlays and freeze the official US-equities validation set."""

import json
import os
from pathlib import Path

import matplotlib.pyplot as plt
import numpy as np
import polars as pl

from case_studies.research import (
    CandidateSet,
    OfficialPopulation,
    candidate_set_supersedes,
    open_study,
    plan_backtests,
    population_supersedes,
    run_backtests,
)
from case_studies.research.strategy import strategy_warmup_periods
from case_studies.utils.backtest_loaders import load_backtest_prices_for
from case_studies.utils.sweep_config import (
    get_portfolio_risk_controls,
    get_position_risk_controls,
    get_top_n_predictions,
)
from utils.style import add_message_title, ml4t_palette, show_with_alt, zero_line
```

```python
CASE_STUDY_ID = "us_equities_panel"
BASELINE_SET_NAMES = [
    "us-equities-fwd-ret-1d-baseline-v1",
]
ALLOCATION_SET_NAMES = [
    "us-equities-fwd-ret-1d-allocation-v1",
]
VALIDATION_SET_NAME_TEMPLATE = "us-equities-{label}-validation-strategies-v1"
EXECUTION_TIER = "canonical"
POPULATION_NAME = ""
SUPERSEDES_POPULATION = ""
SUPERSEDES_SETS: dict = {}
# Empty means this run writes to the case study's own store, which is what canonical
# production execution wants. Any other value routes the run's writes there instead, at
# either tier, and is how a rehearsal at full scale is compared against the published
# result without being able to damage it.
WORKSPACE = ""
PREVIEW_LABELS = []
PREVIEW_MAX_SOURCE_ROWS = 0
PREVIEW_MAX_RISK_CONTROLS = 0
# How many parents per label the overlay grid sits on. `None` reads
# `backtest.sweep.top_n_predictions.risk_overlay`, which this case study declares as 1. The
# declaration was read unconditionally until 2026-09-20 and no parameter was bound, so a
# launcher could not move the width and only an edit to the private config copy reached it,
# which is the shape the four notebooks that do bind it were built to avoid.
TOP_N_COMBOS = None
MAX_SYMBOLS = 0
```

## 2. The strategies an overlay may be laid on

The baseline and allocation sets, opened and checked complete. Both stages are eligible: an
overlay is a rule about when to close a position, and it applies whether the position was sized
equally or by an allocator.

```python
declared_set_names = [*BASELINE_SET_NAMES, *ALLOCATION_SET_NAMES]
# Both tiers resolve the study through `open_study`. It reads the labels and features in place and
# redirects only writes, so a preview run scores the same inputs a canonical one does and cannot
# publish over it.
workspace_override = os.environ.get("ML4T_OUTPUT_DIR") or WORKSPACE
if EXECUTION_TIER == "canonical":
    if PREVIEW_LABELS or PREVIEW_MAX_SOURCE_ROWS or PREVIEW_MAX_RISK_CONTROLS or MAX_SYMBOLS:
        raise ValueError("Canonical execution cannot declare preview reductions")
    if not declared_set_names or len(declared_set_names) != len(set(declared_set_names)):
        raise ValueError("Canonical execution requires unique named strategy sets")
    study = open_study(
        CASE_STUDY_ID,
        execution_tier=EXECUTION_TIER,
        workspace=Path(workspace_override) if workspace_override else None,
    )
elif EXECUTION_TIER == "preview":
    if (
        not PREVIEW_LABELS
        or PREVIEW_MAX_SOURCE_ROWS < 1
        or PREVIEW_MAX_RISK_CONTROLS < 1
        or MAX_SYMBOLS < 1
    ):
        raise ValueError(
            "Preview execution requires labels and explicit row, risk, and symbol limits"
        )
    study = open_study(
        CASE_STUDY_ID,
        execution_tier=EXECUTION_TIER,
        workspace=Path(workspace_override or "experiments"),
    )
else:
    raise ValueError(f"Unsupported execution tier: {EXECUTION_TIER!r}")
```

## 3. Which rows can carry an overlay

Complete, validation-split, and produced under this run's tier, with a finite Sharpe. A row
failing any of those is refused rather than dropped.

```python
backtest_catalog = study.backtests.table(include_preview=True)
if EXECUTION_TIER == "canonical":
    declared_sets = tuple(CandidateSet.one(study, name=name) for name in declared_set_names)
    if any(result_set.member_kind != "backtest" for result_set in declared_sets):
        raise ValueError("Every declared input set must contain backtests")
    source_members = tuple(member for result_set in declared_sets for member in result_set.members)
    if len(source_members) != len(set(source_members)):
        raise ValueError("Declared baseline and allocation sets overlap")
    eligible = backtest_catalog.filter(pl.col("backtest_hash").is_in(source_members))
    if eligible.height != len(source_members):
        raise ValueError("The backtest catalog does not contain every declared strategy member")
else:
    eligible = (
        backtest_catalog.filter(
            (pl.col("execution_tier") == "preview")
            & pl.col("stage").is_in(["signal", "allocation"])
            & pl.col("label").is_in(PREVIEW_LABELS)
        )
        .sort("sharpe", "backtest_hash", descending=[True, False])
        .head(PREVIEW_MAX_SOURCE_ROWS)
    )

ineligible = eligible.filter(
    (pl.col("split") != "validation")
    | (pl.col("execution_tier") != EXECUTION_TIER)
    | ~pl.col("stage").is_in(["signal", "allocation"])
    | ~pl.col("complete")
    | pl.col("sharpe").is_null()
    | ~pl.col("sharpe").is_finite()
)
if eligible.is_empty() or not ineligible.is_empty():
    raise ValueError("Risk overlays require complete finite selection-eligible validation rows")
```

## 4. Fixing the strategy the overlays are applied to

One configuration per label, taken on validation Sharpe across both earlier stages. Everything
identifying the strategy is then held - the model, the checkpoint, the signal, the sizing - so
every row below differs from every other only in the overlay laid on it.

**Sweeping overlays across several strategies at once would confound the two.** A table in which
both the strategy and the control vary cannot say whether a difference came from the rule or from
the book it was applied to.


**Prices are cached by label and warmup, not once per label.** A strategy's identity digests the
price frame it was handed, and the allocator underneath each overlay needs a different amount of
history before it can decide anything - none for the ones reading only the predictions, a
volatility window for the per-stock ones, a longer lookback for the ones estimating a covariance
matrix. So each member has to receive the frame its own warmup implies, and the cache key is
what keeps that true while loading each distinct frame once.

```python
_price_cache: dict[tuple[str, int], object] = {}


def prices_for(label, warmup_periods):
    key = (str(label), int(warmup_periods))
    if key not in _price_cache:
        _price_cache[key] = load_backtest_prices_for(
            CASE_STUDY_ID,
            label,
            split="validation",
            max_symbols=MAX_SYMBOLS,
            warmup_periods=int(warmup_periods),
        )
    return _price_cache[key]


top_n = (
    TOP_N_COMBOS
    if TOP_N_COMBOS is not None
    else get_top_n_predictions(CASE_STUDY_ID, "risk_overlay")
)
if top_n < 1:
    raise ValueError("the risk overlay needs at least one parent per label")
selected_parts = []
for label in eligible.get_column("label").unique().sort().to_list():
    selected_parts.append(
        eligible.filter(pl.col("label") == label)
        .sort("sharpe", "backtest_hash", descending=[True, False])
        .head(top_n)
    )
selected_sources = pl.concat(selected_parts).sort("label", "backtest_hash")
if selected_sources.is_empty():
    raise RuntimeError("No risk-overlay source configuration was selected")

selected_sources.select(
    "label",
    "family",
    "config_name",
    "checkpoint_kind",
    "checkpoint_value",
    "stage",
    "prediction_hash",
    "backtest_hash",
    "sharpe",
)
```

## 5. The controls, and the check that they bound

Fourteen declared controls across the three kinds, planned as one backtest each against the fixed
strategy.

**Whether the overlays changed anything is checked after execution**, in the section that reports
results: each one against the strategy it was laid on, on the trade count and the Sharpe. A
difference establishes that the control acted. Matching values establish only that those two
statistics did not move, and leave every other outcome undetermined.

`SUPERSEDES_POPULATION` and `SUPERSEDES_SETS` name the generation this run replaces. A population
and a candidate set are both immutable, so a re-run that admits different members has to say
which snapshot it supersedes or the registry refuses the write. Both default to empty, which is
right for a first run and for a reader's clean clone; `population_supersedes` and
`candidate_set_supersedes` withhold a declared hash wherever offering it would be refused.

```python
risk_requests = []
for control in get_position_risk_controls(CASE_STUDY_ID):
    if control["type"] == "time_exit":
        rule = {"type": control["type"], "bars": control["bars"]}
    else:
        rule = {"type": control["type"], "threshold": control["threshold"]}
    risk_requests.append(
        {
            "name": control["name"],
            "spec": {"name": control["name"], "position_rules": [rule]},
        }
    )
for control in get_portfolio_risk_controls(CASE_STUDY_ID):
    risk_requests.append(
        {
            "name": control["name"],
            "spec": {
                "name": control["name"],
                "portfolio_limits": [{"type": control["type"], "threshold": control["threshold"]}],
            },
        }
    )
if EXECUTION_TIER == "preview":
    risk_requests = risk_requests[:PREVIEW_MAX_RISK_CONTROLS]
if not risk_requests or len({request["name"] for request in risk_requests}) != len(risk_requests):
    raise ValueError("Risk controls must be non-empty and uniquely named")

prediction_catalog = study.predictions.table(include_preview=True)
planned_requests = []
plan_rows = []
```

```python
def risk_member_records(
    label, source_row, selection, signal, allocation, risk_request, expected_hash
):
    request = {
        "label": label,
        "selection": selection,
        "signal": signal,
        "allocation": allocation,
        "risk": risk_request["spec"],
        "risk_name": risk_request["name"],
        "prediction_hash": source_row["prediction_hash"],
        "source_backtest_hash": source_row["backtest_hash"],
        "expected_hash": expected_hash,
    }
    row = {
        "label": label,
        "source_stage": source_row["stage"],
        "source_backtest_hash": source_row["backtest_hash"],
        "risk": risk_request["name"],
        "prediction_hash": source_row["prediction_hash"],
        "backtest_hash": expected_hash,
    }
    return request, row
```

```python
def plan_risk_member(label, prices, risk_request, source_row):
    selected_prediction = prediction_catalog.filter(
        pl.col("prediction_hash") == source_row["prediction_hash"]
    )
    if selected_prediction.height != 1:
        raise ValueError("A risk source must resolve one prediction catalog row")
    source_spec = json.loads(source_row["spec_json"])
    signal = dict(source_spec["strategy"]["signal"])
    allocation = source_spec["strategy"].get("allocation")
    plan = plan_backtests(
        study,
        predictions=selected_prediction,
        signal=signal,
        allocation=allocation,
        risk=risk_request["spec"],
        prices=prices,
        chapter="ch19",
    )
    if len(plan.members) != 1:
        raise RuntimeError("One risk request must plan one backtest")
    return risk_member_records(
        label,
        source_row,
        selected_prediction,
        signal,
        allocation,
        risk_request,
        plan.expected_hashes[0],
    )
```

```python
for label in selected_sources.get_column("label").unique().sort().to_list():
    for source_row in selected_sources.filter(pl.col("label") == label).iter_rows(named=True):
        for risk_request in risk_requests:
            prices = prices_for(
                label,
                # The source's allocation lives in its spec_json, not as a catalog column, so
                # source_row.get("allocation") is always None and would silently warm up 0 bars
                # for every allocation-stage source.
                strategy_warmup_periods(json.loads(source_row["spec_json"])),
            )
            request, row = plan_risk_member(label, prices, risk_request, source_row)
            planned_requests.append(request)
            plan_rows.append(row)
```

```python
planned_population = pl.DataFrame(plan_rows).sort(
    "label", "risk", "source_backtest_hash", "backtest_hash"
)
if planned_population.get_column("backtest_hash").n_unique() != planned_population.height:
    raise ValueError("The risk plan contains duplicate backtest identities")

official_population = None
if EXECUTION_TIER == "canonical":
    population_name = POPULATION_NAME or "us-equities-risk-overlay-v1"
    official_population = OfficialPopulation.create(
        study,
        name=population_name,
        supersedes=population_supersedes(
            study, name=population_name, declared=SUPERSEDES_POPULATION
        ),
        member_kind="backtest",
        members=tuple(planned_population.get_column("backtest_hash")),
    )

planned_population
```

## 6. Running them

Independent per control, so a failure costs that control and leaves the rest usable.

```python
execution_rows = []
failure_rows = []


def execute_risk_member(prices, request):
    execution = run_backtests(
        study,
        predictions=request["selection"],
        signal=request["signal"],
        allocation=request["allocation"],
        risk=request["risk"],
        prices=prices,
        chapter="ch19",
    )
    if len(execution.results) != 1 or execution.results[0].hash != request["expected_hash"]:
        raise RuntimeError("Risk execution changed its planned identity")
    return {
        "label": request["label"],
        "source_backtest_hash": request["source_backtest_hash"],
        "risk": request["risk_name"],
        "backtest_hash": execution.results[0].hash,
        "status": execution.diagnostics[0]["status"],
    }
```

```python
for label in selected_sources.get_column("label").unique().sort().to_list():
    for request in (item for item in planned_requests if item["label"] == label):
        try:
            prices = prices_for(
                label,
                strategy_warmup_periods({"strategy": {"allocation": request["allocation"]}}),
            )
            execution_rows.append(execute_risk_member(prices, request))
        except Exception as error:
            failure_rows.append(
                {
                    "label": label,
                    "source_backtest_hash": request["source_backtest_hash"],
                    "risk": request["risk_name"],
                    "backtest_hash": request["expected_hash"],
                    "error_type": type(error).__name__,
                    "error": str(error),
                }
            )
```

```python
execution_diagnostics = pl.DataFrame(
    execution_rows,
    schema={
        "label": pl.String,
        "source_backtest_hash": pl.String,
        "risk": pl.String,
        "backtest_hash": pl.String,
        "status": pl.String,
    },
)
failures = pl.DataFrame(
    failure_rows,
    schema={
        "label": pl.String,
        "source_backtest_hash": pl.String,
        "risk": pl.String,
        "backtest_hash": pl.String,
        "error_type": pl.String,
        "error": pl.String,
    },
)
if not failures.is_empty():
    raise RuntimeError(f"Risk population has {failures.height} unsuccessful members")

if official_population is not None:
    official_population.require_complete()

execution_diagnostics
```

## 7. Naming the overlay sets, and the population selection is made over

One frozen risk-overlay set per label, plus the union of all three stages - baseline, allocation
and overlay - as the population validation selection is made over.

**The union is the point.** A strategy may legitimately come from any of the three stages, so
ranking only the last of them would exclude an un-overlaid book that was better than every
overlaid one. [`19_costs`](19_costs.ipynb) derives its pool from the same stage sequence for the
same reason.

**The freeze is also the comparability check.** Nothing is declared comparable, so
`CandidateSet.create` requires every field of the protocol to be identical across the members:
two rows that measured their Sharpe on different folds are not two rankings of one thing, and
this is what refuses to freeze them together.

```python
post_sweep_catalog = study.backtests.table(include_preview=True)
completed_risk = post_sweep_catalog.filter(
    pl.col("backtest_hash").is_in(planned_population.get_column("backtest_hash"))
)
if (
    completed_risk.height != planned_population.height
    or completed_risk.filter(~pl.col("complete")).height
    or completed_risk.filter(pl.col("stage") != "risk_overlay").height
    or completed_risk.filter(pl.col("execution_tier") != EXECUTION_TIER).height
    or completed_risk.filter(pl.col("sharpe").is_null() | ~pl.col("sharpe").is_finite()).height
):
    raise RuntimeError("The risk catalog is incomplete or mis-staged")

# The rows this sweep just published carry two spec leaves that no signal or allocation row has,
# `strategy.risk.name` and `strategy.risk.position_rules`, and `study.backtests.table()` derives
# its columns from the specs present in the registry. So `backtest_catalog`, read before the sweep,
# is two columns narrower than the read above, and concatenating a frame from each raises
# ShapeError on any run where the risk stage starts empty - which is every first run. Re-deriving
# the selection-eligible rows from this read puts both sides of that concatenation on one schema by
# construction. A `how="diagonal"` concat would also pass, by filling the two columns with nulls,
# and would keep passing silently the next time the schemas diverge for a reason that matters.
eligible_post_sweep = post_sweep_catalog.filter(
    pl.col("backtest_hash").is_in(eligible.get_column("backtest_hash"))
)
if eligible_post_sweep.height != eligible.height:
    raise RuntimeError("The post-sweep catalog lost a declared strategy member")
```

**Did the control fire, and did anything move?** Those are two questions and the catalog answers
both.

`risk_triggers` counts how many times the installed control acted during the backtest, recorded
by the engine as it installs each rule. Its three states are what separate the cases nothing else
on this page can tell apart:

- **A positive count** is a control that fired. Whether that changed anything measurable is the
  second question below.
- **Zero** is a control that was installed and never reached its threshold. That is a result:
  a wide stop on a book that never drew down that far has nothing to do.
- **Null** is no control installed at all - the engine records a count for every rule it builds,
  so a null where a control was declared means the declaration did not reach the engine.

Reading it is what this stage owes, because a sweep of fourteen settings that all report the same
numbers is a finding about risk control if they were installed and a defect in the wiring if they
were not.

The second question is whether the numbers moved, and each result is compared against the
strategy it was laid on across the two axes the catalog carries: the trade count and the Sharpe.
A row that differs on either acted; a row identical on both moved neither of those two
statistics, which is weaker than "changed nothing", because two different return paths can share
a Sharpe and a trade count while differing in total return or in drawdown. It is weaker still
than "never fired" - a stop can close a position the next rebalance would have closed anyway,
replacing one exit with an earlier one and leaving the count where it was - and that is exactly
the gap the trigger count fills.

**So the movement check has three outcomes, not two.** A row is CHANGED when either comparison is
true, because one difference is enough to establish the control acted. It is UNCHANGED only when
both are false and both were comparable. Anything else is UNKNOWN: a comparison that could not be
made is not evidence of sameness, and collapsing it into one would manufacture the signature this
check exists to detect.

```python
overlay_effect = (
    completed_risk.select("label", "backtest_hash", "sharpe", "num_trades", "risk_triggers")
    .join(
        planned_population.select("backtest_hash", "risk", "source_backtest_hash"),
        on="backtest_hash",
        how="inner",
    )
    .join(
        backtest_catalog.select(
            pl.col("backtest_hash").alias("source_backtest_hash"),
            pl.col("num_trades").alias("source_num_trades"),
            pl.col("sharpe").alias("source_sharpe"),
        ),
        on="source_backtest_hash",
        how="left",
    )
    .with_columns(
        # Null on either side is unknown rather than unmoved: a source whose trade count was never
        # registered cannot answer the question, and reading it as "did not move" would
        # manufacture the very signature this check exists to detect.
        trades_moved=pl.when(pl.col("num_trades").is_null() | pl.col("source_num_trades").is_null())
        .then(None)
        .otherwise(pl.col("num_trades") != pl.col("source_num_trades")),
        sharpe_moved=pl.when(pl.col("sharpe").is_null() | pl.col("source_sharpe").is_null())
        .then(None)
        .otherwise(pl.col("sharpe") != pl.col("source_sharpe")),
    )
    .select(
        "label",
        "risk",
        "risk_triggers",
        "num_trades",
        "source_num_trades",
        "trades_moved",
        "sharpe_moved",
    )
    .sort("label", "risk")
)
_changed = overlay_effect.get_column("trades_moved").fill_null(False) | overlay_effect.get_column(
    "sharpe_moved"
).fill_null(False)
_comparable_both = (
    overlay_effect.get_column("trades_moved").is_not_null()
    & overlay_effect.get_column("sharpe_moved").is_not_null()
)
_unchanged = _comparable_both & ~_changed
n_changed, n_unchanged = int(_changed.sum()), int(_unchanged.sum())
n_unknown = overlay_effect.height - n_changed - n_unchanged
print(
    f"{n_changed} of {overlay_effect.height} overlay results differ from the strategy they were "
    f"laid on; {n_unchanged} match it on both compared statistics; {n_unknown} could not be "
    "fully compared"
)
_triggers = overlay_effect.get_column("risk_triggers")
n_fired = int((_triggers.fill_null(0) > 0).sum())
n_silent = int((_triggers == 0).sum())
n_uninstalled = int(_triggers.is_null().sum())
print(
    f"{n_fired} of {overlay_effect.height} controls fired at least once; {n_silent} were installed "
    f"and never reached their threshold; {n_uninstalled} installed no rule at all"
)
# The engine records a count for every rule it builds, so a row that declared a control and
# registered no count is one whose declaration never reached it. That is the failure C17 names,
# and it is the only one of the three states that is not a result.
if n_uninstalled:
    raise RuntimeError(
        f"{n_uninstalled} of {overlay_effect.height} overlay results registered no trigger count, "
        "so their declared control was never installed and the row is named for behaviour that "
        "did not run"
    )
if n_unchanged and not n_changed and not n_unknown:
    print(
        "  No declared control moved either the trade count or the Sharpe. Neither statistic "
        "moving is possible on a calm book, and is also what a control the engine never installed "
        "looks like, so confirm the controls reach the engine before reading it either way."
    )
print(overlay_effect)

set_rows = []
if EXECUTION_TIER == "canonical":
    for label in completed_risk.get_column("label").unique().sort().to_list():
        label_name = label.replace("_", "-")
        result_set_name = f"us-equities-{label_name}-risk-overlay-v1"
        result_set = study.backtests.freeze(
            completed_risk.filter(pl.col("label") == label),
            name=result_set_name,
            supersedes=candidate_set_supersedes(
                study, name=result_set_name, declared=SUPERSEDES_SETS.get(result_set_name, "")
            ),
        )
        set_rows.append(
            {"label": label, "set_name": result_set.name, "members": len(result_set.members)}
        )

        # The selection pool this label's holdout is chosen from: its baseline and allocation
        # members plus the risk overlays just published. No contract - one label means one
        # label_artifact, and every other protocol field being required-constant is the guard.
        validation_candidates = pl.concat(
            [
                eligible_post_sweep.filter(pl.col("label") == label),
                completed_risk.filter(pl.col("label") == label),
            ]
        ).sort("backtest_hash")
        if (
            validation_candidates.get_column("backtest_hash").n_unique()
            != validation_candidates.height
        ):
            raise ValueError(f"Selection-eligible strategy sets overlap for {label}")
        validation_set_name = VALIDATION_SET_NAME_TEMPLATE.format(label=label_name)
        validation_set = study.backtests.freeze(
            validation_candidates,
            name=validation_set_name,
            supersedes=candidate_set_supersedes(
                study,
                name=validation_set_name,
                declared=SUPERSEDES_SETS.get(validation_set_name, ""),
            ),
        )
        set_rows.append(
            {
                "label": label,
                "set_name": validation_set.name,
                "members": len(validation_set.members),
            }
        )


compatible_sets = pl.DataFrame(
    set_rows,
    schema={"label": pl.String, "set_name": pl.String, "members": pl.Int64},
)
compatible_sets
```

### What the threshold does

The sweep's argument is that a single chosen threshold says almost nothing, and the frame above
does not show a shape. Each line below traces one label's Sharpe as one kind of control is
loosened: the stop losses along their loss-from-entry threshold, the trailing stops along their
give-back threshold, and the time exits along their holding-period cap.

The dashed reference in each panel is the unprotected strategy the overlays were laid on, which
is what any of these lines has to beat. The loosest declared setting is not that reference: a 20
per cent trailing stop is a loose control, not the absence of one.

The shapes mean different things. A line that rises all the way to the loosest setting and stays
below the reference is a control this book paid for and got nothing from. One with an interior
peak has found a threshold this sample liked, which is a much weaker claim than it looks - one
sample, one strategy per label, and no correction for having looked at fourteen. A flat line
means the measured Sharpe did not move across the declared settings, which is weaker than it
sounds in two directions: two different thresholds can produce the same exits, and two different
return paths can share a Sharpe. It is not evidence that the controls never fired; the
`risk_triggers` column above is what settles that, and it is read there rather than inferred
from the shape of any line here.

```python
control_axes = {
    control["name"]: (control["type"], float(control.get("threshold", control.get("bars"))))
    for control in get_position_risk_controls(CASE_STUDY_ID)
}
# One source per label is asserted rather than assumed: `top_n` comes from the sweep
# configuration, and a label with two sources needs two reference lines.
_sources_per_label = selected_sources.group_by("label").len()
if (_sources_per_label.get_column("len") != 1).any():
    raise ValueError(
        "the overlay sweep is drawn against one source strategy per label; this run selected "
        f"{_sources_per_label.to_dicts()}"
    )
unprotected = dict(selected_sources.select("label", "sharpe").iter_rows())
sweep = (
    completed_risk.select("label", "backtest_hash", "sharpe")
    .join(planned_population.select("backtest_hash", "risk"), on="backtest_hash", how="inner")
    .with_columns(
        kind=pl.col("risk").replace_strict(
            {name: kind for name, (kind, _) in control_axes.items()}, default=None
        ),
        setting=pl.col("risk").replace_strict(
            {name: value for name, (_, value) in control_axes.items()},
            default=None,
            return_dtype=pl.Float64,
        ),
    )
    .drop_nulls(["kind", "setting"])
    .sort("kind", "setting")
)
kinds = sweep.get_column("kind").unique().sort().to_list()
sweep_labels = sweep.get_column("label").unique().sort().to_list()
axis_names = {
    "stop_loss": "Loss from entry",
    "trailing_stop": "Give-back from peak",
    "time_exit": "Bars held",
}
# `ml4t_palette` returns a list of that many colours, so it is called once and indexed.
palette = ml4t_palette(len(sweep_labels), categorical=True)

fig, axes = plt.subplots(1, len(kinds), figsize=(4.2 * len(kinds), 4), sharey=True)
axes = np.atleast_1d(axes)
for ax, kind in zip(axes, kinds, strict=True):
    for index, label in enumerate(sweep_labels):
        curve = sweep.filter((pl.col("kind") == kind) & (pl.col("label") == label)).sort("setting")
        ax.plot(
            curve.get_column("setting"),
            curve.get_column("sharpe"),
            marker="o",
            markersize=4,
            lw=1.4,
            color=palette[index],
            label=label,
        )
    for index, label in enumerate(sweep_labels):
        if label in unprotected:
            ax.axhline(
                unprotected[label],
                color=palette[index],
                lw=1.0,
                ls="--",
                alpha=0.7,
            )
    zero_line(ax)
    ax.set_xlabel(axis_names.get(kind, kind))
axes[0].set_ylabel("Validation Sharpe")
axes[-1].legend(fontsize=8, frameon=False)
add_message_title(
    axes[0],
    "Validation Sharpe against each risk control's threshold",
    subtitle=(
        "Validation Sharpe against each control's own threshold, one line per label, with the "
        "unprotected strategy dashed"
    ),
)
# The alt text counts rather than asserts: whether any line turns over is the question the sweep
# exists to answer, and a panel described as peaking when it does not is a claim the data refutes.
_peaks = (
    sweep.group_by("kind", "label")
    .agg(
        peak=pl.col("setting").sort_by("sharpe", descending=True).first(),
        low=pl.col("setting").min(),
        high=pl.col("setting").max(),
    )
    .with_columns(interior=pl.col("peak").is_between(pl.col("low"), pl.col("high"), closed="none"))
)
_n_interior = int(_peaks.get_column("interior").sum())
_above = sum(
    1
    for row in sweep.group_by("kind", "label")
    .agg(best=pl.col("sharpe").max())
    .iter_rows(named=True)
    if row["label"] in unprotected and row["best"] > unprotected[row["label"]]
)
_pairs = sweep.select("kind", "label").unique().height
show_with_alt(
    fig,
    "Line charts side by side, one panel per kind of control, sharing a vertical Sharpe axis. "
    "Every panel plots that control's own threshold on the horizontal axis with one line per "
    "label, a dashed horizontal line per label marking the unprotected strategy it was laid on, "
    "and a dashed line at zero. Counted from the underlying frame, "
    f"{_n_interior} of {_peaks.height} label-and-kind curves reach their highest Sharpe at a "
    "setting that is neither the tightest nor the loosest declared, and "
    f"{_above} of {_pairs} beat their unprotected reference at any setting.",
)
```

`22_strategy_analysis.py` reopens one of these per-label sets -
`us-equities-<label>-validation-strategies-v1` - and applies the one official rule: highest
validation backtest Sharpe with the backtest hash as deterministic tie-break, within that label.
The holdout follows from that selection with nothing in between: retrain the selected
configuration on everything up to the holdout start, predict the holdout window, and run the
same backtest configuration on those predictions.

## What to notice

**Check that the overlays moved something before reading what they did.** A result matching the
unprotected book on both compared statistics has not been shown to change anything, and matching
across every declared setting is a reason to check that the controls reached the engine rather
than a finding about risk control. The two compared columns do not settle it on their own; the
`risk_triggers` count does, and a row that declared a control and installed none stops the
notebook rather than being reported as a result. A row installed and never triggered is a
result: it says the threshold was never reached on this book.

**An overlay can only remove, so it reshapes a return distribution rather than shifting it.** It
truncates the left tail by closing losing positions early and truncates the right by closing
profitable ones early,
and which effect dominates is a property of how the strategy's returns actually arrive. A book
whose gains come from a few positions running a long way is one an early exit hurts.

**A threshold sweep says more than a chosen threshold, and it needs the unprotected book beside
it.** A control that helps at one setting and hurts either side of it has found a feature of
this sample rather than a property of the strategy. One that improves as it loosens is heading
toward the unprotected book without reaching it - the loosest declared setting is still a
control - so what says the overlay was not worth having is the reference line, not the trend.

**The time exit is the control worth reading first.** It watches no price, so where it moves the
result as much as a stop does, what the stops were doing was shortening the holding period rather
than responding to losses.

**Known limitations.** Overlays are evaluated on one fixed configuration per label, so nothing
here says whether the same control would help a different strategy. The controls act on bar
closes, so an intra-bar breach is not seen. And this remains gross of costs, while an overlay
that fires often adds trades - which is charged in [`19_costs`](19_costs.ipynb).

**Next**: [`19_costs`](19_costs.ipynb) asks how much friction the chosen strategy absorbs.
![notebook output](figures/p1_1.png)