Strategies
KitchenSink ML (QuantRocket) price/volume/breadth subset: numpy ridge forecast of next-week return, weekly long top-10 / short bottom-10 among 68 US large caps on USEQ daily bars
Outcome: Abandoned
KitchenSinkRidgeWeeklyLSUseq
Outcome Summary
KitchenSinkRidgeWeeklyLSUseq operationalized QuantRocket's TheKitchenSinkML framework without the fundamental, sector, VIX and TRIN data the platform lacks: a numpy ridge regression ranked 68 large caps weekly into a 10-long / 10-short dollar-neutral book. On a reliable sample of 3,597 closed positions it lost 7.9% with Sharpe -0.16, PF 0.973 and five of seven years negative; nearly all of its gains came in the 2022 rate-shock regime while the normal-volatility tercile lost 13.9%. The forecast's IC was -0.10, so at backtest review the analyst abandoned it before optimization (iteration 1) for negative expectancy unrelated to costs. The rejection covers only this operationalization, not the full source framework.
Hypothesis
USEQ is 8.3% of the corpus, and long_short is under-represented (21%). The design needs only OHLCV (L189) and differs in mechanism from prior USEQ pairs and contrarian-cohort runs: it is a multi-feature learned cross-sectional forecast rather than a single anomaly.
Implementation
Verification Results
Acceptable as is. In the full backtest, check that the fill timestamps of the opening legs are within one session of the closes. State the lag in the description so the analyst can account for it.
Order sequencing. Closes and trims go out at the rebalance close and fill at the next session's open. New legs and increases are sent from on_order_filled only after every working order has cleared. By the developer's own untested assumption, they therefore fill later than the hypothesis's next-open entry: at that session's close or the following open. The cause is the fixed 1.0x leverage with 1.0x gross on a margin account, which is a reasonable forced choice. Still, about half of each rebalance enters up to one session late, which shifts the holding window relative to the source's one-day lag.
Keep it, but disclose it. Alternatively, add margin_headroom to the contract's fixed set.
margin_headroom=0.95 is not in the frozen plan. It makes realised gross about 0.95x instead of the fixed gross_exposure 1.0. This is a small scale difference with no structural change: the book stays dollar-neutral at equal weights.
No change is required. Note that the effective backtest starts mid-2012 (training then needs about 520 more sessions before the first trade).
The base template's cross-leg alignment barrier makes the primary wait until all 68 same-timeframe legs have a bar at the primary timestamp. META only starts in May 2012, so nothing is processed before then; the developer states this. A single-name halt or missing bar skips that whole session for the panel. The point-in-time data is correct, but the panel can lose rows on data gaps.
Make sure the paper warmup covers at least train_years plus 1 year, or persist the resolved samples.
Restart persistence. day_count, cached_day and last_ts_ns are excluded from persistence, but last_week_idx, last_rebalance_week_idx and the EMA state are persisted. After a paper restart, the warmup replay rebuilds the panel, so this is mostly consistent. However, the training samples (resolved_samples) are rebuilt only as far as the warmup reaches. The backtest is unaffected.
The sandbox covers only about 53 sessions. Its avg_trade_return_pct of 0.447% (102 closed trades, PF 1.27) is one unoptimized draw over a short window, so it says nothing about the edge. The universe is today's 68 large caps (survivorship bias, acknowledged in the hypothesis). A pooled linear ridge on about 27 mostly rank features plausibly has a weak IC. Judge the result on full-history pooled OOS, not on this sandbox. The short leg's win rate was 0.33 against 0.71 for the long leg. In a rising window that pattern hints at market beta in the ranking, so check whether the long/short spread survives in down markets. Runtime: 232 s for the sandbox and an estimated 1092 s baseline, so optimization compute may be heavy (advisory).
Backtest Review
- Sharpe
- -0.16
- Total return
- -7.92%
- Max drawdown
- 22.46%
- Trades
- 3597
- Win rate
- 50.1%
- Profit factor
- 0.97
Implementation looks faithful to the operationalization: the book is dollar-neutral 10/10, rebalances weekly with next-open fills, peak entry leverage is about 1.01, accounting reconciles, and end_unrealized_pct is only 0.13.
The sample is reliable: 3,597 closed positions over 2018-09 to 2024-01 (META's 2012 listing plus the 1,600-bar warmup binds the start), with metrics_reliable = true.
Per-trade economics are below the floor: avg_trade_return_pct is -0.057% against a USEQ floor of +0.05% (preferred +0.10%). Commission is zero, and impact ($3.5k) plus borrow ($1.1k) total about 4.6% of traded notional in bps terms, so costs are not the cause. The gross edge is about zero or negative.
PF is 0.973, Sharpe -0.16 (95% CI -0.93 to +0.68), total return -7.9%, max drawdown 22.5%, and the drawdown took 1,393 days to recover. Five of seven calendar years are negative (2021 alone is -13%).
The executed forecast's IC is -0.103 (t = -1.68, n = 266 weekly obs), so the ridge forecast ranks slightly in the wrong direction. Net edge is -1.4 bp per name-week.
The equity curve is regime-driven: the stressed tercile returned +11.0% (almost all from the 2022 rate shock, +11.5%), while the normal tercile lost 13.9% (Sharpe -1.27). The model is a regime bet, not a stable cross-sectional forecaster.
QA's concern is borne out. The short leg's win rate is 0.458 against 0.546 for the long leg, consistent with a weak, beta-contaminated ranking rather than spread capture.
Outcome Summary
With only price, volume and breadth features, a linear pooled ridge on a small fixed large-cap universe produced no cross-sectional ranking information. A future iteration needs different information (fundamentals, a broader universe or a nonlinear estimator) rather than parameter tuning, and the profit concentrated in the 2022 rate shock shows that regime-driven PnL can look like an edge.
At backtest review the analyst chose abandon for no_edge / negative_expectancy: per-trade return was -0.057%, below the +0.05% USEQ floor, and commission was zero with small impact and borrow costs, so costs were not the cause. Optimization was skipped because tuning ridge_alpha, train_years or retrain_weeks could not reverse a near-zero, slightly negative IC.
It used the price, volume, technical and breadth features of QuantRocket's TheKitchenSinkML to train a pooled closed-form numpy ridge forecast of next-week return, then held a weekly dollar-neutral book long the top 10 and short the bottom 10 of a fixed universe of 68 US large caps on USEQ daily bars.
From 2018-09 to 2024-01 it closed 3,597 positions and lost 7.9% (Sharpe -0.16, 95% CI -0.93 to +0.68, PF 0.973) with a 22.5% max drawdown. avg_trade_return_pct was -0.057%, and the executed weekly forecast had an IC of -0.10 (t = -1.7, n = 266).
Analysis
TheKitchenSinkML
Analysis
kitchensink_ml.py
Time series / date functionality#
Backtest and paper results are hypothetical. Trading involves risk of loss.