Strategies

QuantRocket DemoMLStrategy port: pooled 25-horizon return features, numpy logistic 'beats the cross-sectional median over 22 open-to-open days' classifier, hold the top-3 probabilities, monthly equal-weight long-only rebalance at the next session open (USEQ 1-DAY, 9 Select Sector SPDRs)

Outcome: Abandoned

QuantRocketDemoMLSectorSpdrUseq

Outcome Summary

This strategy ported QuantRocket's DemoMLStrategy onto the nine Select Sector SPDRs. It used a numpy logistic 'beats the median' classifier to pick the top 3 ETFs each month. The backtest was profitable in absolute terms (+241.4%, Sharpe 0.37, average trade 1.01%, well above costs), but over the development period it grew 2.42x against 4.30x for an equal-weight basket of the same ETFs, with a larger 62.8% drawdown and a signal IC indistinguishable from noise. The analyst abandoned it at backtest review after one iteration, concluding that tuning could not create the missing selection skill, so optimization and later stages were never reached.

Strategy report

Hypothesis

Derived by the Research Lead from "DemoMLStrategy" by QuantRocket (quantrocket) https://github.com/quantrocket-codeload/moonshotml-intro/blob/2b3a8e382c2d4d7eb8da02eb8bc82fa165c684f4/intro_moonshotml/demo-ml.py: Kept from the source: the 25 return-horizon features, the 22-day open-to-open beats-the-median target, predict_proba ranking, TOP_N=3, the month-end signal carried forward, equal weights, next-open entry and open-to-open returns. Chosen by us: a numpy L2 logistic model (the source leaves the model open), an expanding walk-forward with annual refits, the 9 Select Sector SPDRs in place of QuantRocket's free stock sample, and USEQ costs. Port of QuantRocket's Apache-2.0 'DemoMLStrategy' (moonshotml-intro/demo-ml.py, credit QuantRocket LLC) onto USEQ daily bars. The source's rules are kept as written; four things are adapted and named here. (1) UNIVERSE: the source runs on 'usstock-free', QuantRocket's small free sample of US stocks. A fixed list of today's large-cap stocks would add survivorship bias, so the port uses a small universe that has no survivorship problem: the nine original Select Sector SPDR ETFs (XLB, XLE, XLF, XLI, XLK, XLP, XLU, XLV, XLY), all listed since Dec 1998 and none ever delisted. They are US-listed, Open/Close daily bars, and TOP_N=3 of 9 keeps the source's 'hold a few of a small universe' shape. (2) MODEL: the source does not define the model. MoonshotML uses whatever scikit-learn classifier the user supplies and calls predict_proba. sklearn is not importable in our sandbox, so the developer writes a pooled L2-regularised LOGISTIC REGRESSION in numpy (features z-scored on the training set, fit by IRLS or gradient descent). This is the standard probabilistic classifier, a free choice registered as a parameter. (3) WALK-FORWARD TRAINING: the source's training schedule lives outside the file, in ml_walkforward. We use an expanding window: first fit after min_train_bars=756 sessions of labelled samples, then refit every retrain_months=12 months. Each refit uses only samples whose 22-day label is fully known at the refit close. The source quirk of filling the newest 22 unlabelled rows with 0 is dropped: the port drops those rows instead, because it would otherwise train on fabricated labels. (4) COSTS: the source charges USD 0.005 per share. USEQ is commission-free with modeled spread and impact, which is about 0.5-1 bp per side on these ETFs, roughly what the source charges. Execution: the source decides on day t's close, weights.shift() enters on day t+1, and returns run open-to-open. That is exactly the platform's supported next_open_fill on a session venue: a market order decided on bar t's close fills at the next session's open, so no substitution is needed. Per-trade arithmetic: the median one-month gap in return between the best and the median sector ETF is about 2-3%. If the classifier captures even 10-15% of that spread on the names it picks, that is about 0.3-0.5% per holding-month, against a USEQ round trip of about 0.04%, more than 7x cost. Risk control route (b): signal-based exit at the monthly rebalance, unlevered, each position 1/3 of equity (100% gross, no single position above 33%). The source has no stop and none is invented. This is a long-only, fully invested sector book. A 2008- or 2020-style market drawdown hits it in full, so the predicted drawdown is set honestly near the cap. Prior art: the KitchenSink ML ridge port and the UpMinusDown momentum port are different QuantRocket files. This is the first test of the demo-ml classifier.

Premise falsified on the development period (Dec 2002 to Mar 2021). The core claim is that a logistic 'beats the cross-sectional median' classifier picks sector ETFs that outperform the rest. The classifier's top-3 monthly book returned 2.42x, while holding all nine Select Sector SPDRs equally returned 4.30x over the same window (both price-only). The book beat that basket in only 6 of 19 years, and its drawdown was larger (62.8% vs 51.5%; the registered limit was 40%). QA's concern is borne out: the selection carries no skill above the equal-weight benchmark, and the absolute profit is market beta. Fees are not the cause: average trade is 1.04% against the 0.05% USEQ floor, profit factor 1.37, Sharpe 0.34. The code implements the port correctly, so this is not an implementation defect. Tuning l2_C, top_n (2 to 5) or the label horizon cannot add the selection skill that is missing. Raising top_n toward 9 would only converge on the basket. This verdict covers the Research Lead's version of the idea (numpy logistic model, nine sector SPDRs, annual expanding refits). It is not a verdict on QuantRocket's DemoMLStrategy on its own stock universe.

Implementation

Stratmill port of QuantRocket's DemoMLStrategy onto the nine Select Sector SPDRs (USEQ daily). Every session it computes the 25 close-to-close return features per ETF. A pooled L2 logistic regression (numpy Newton/IRLS, sklearn-style C, features z-scored on the training set) is trained on whether each ETF's 22-day open-to-open forward return beats the cross-sectional median. Training is an expanding walk-forward: the first fit needs 756 sessions with fully known labels, then the model is refit every 12 calendar months. On the last trading session of each month the ETFs are ranked by predicted probability and the top 3 are held at equal weight (97% gross, 1x, long only). Orders decided at that close fill at the next session's open. The signal is the mean probability of the top-ranked names minus the median probability.

The previous refusal (capability_unsupported: next_session_open_fill) was wrong. execution_capabilities lists next_open_fill as supported on session venues: a market order decided on bar t's close fills at bar t+1's open. That matches the source's weights.shift() with open-to-open returns, so the strategy is now implemented rather than refused. The source rules are kept: the 25 LOOKBACK_WINDOWS features, FORWARD_RETURNS_WINDOW=22 with the beats-the-median open-to-open target, predict_proba ranking with TOP_N=3, the month-end signal carried forward (a new book only on the last trading day of the month, read from the platform session calendar), equal weights, and next-open fills. The adaptations named in the hypothesis are implemented: the 9 SPDR universe, the numpy L2 logistic model, expanding annual refits using only fully labelled rows (the newest unlabelled rows are dropped, not zero-filled), and USEQ costs. All nine legs are traded manually through _submit_entry_instrument and _submit_exit_position; the base entry and exit hooks are inert. Sizing is whole shares at equity_mtm x 0.97 / top_n. Risk route (b) applies: a signal exit at the monthly rebalance, unlevered, at most about 33% of equity per name. Rebalance drift tolerance 0.05: a kept name is trimmed or topped up only when its weight is more than 5 points off target. Scheduling is calendar-anchored, and per-run state is held in scalars (target_names, topup_pending, last_fit_month). The model is rebuilt from the warmup history after a restart. Optimisation parameters use the plan's exact names (l2_C, top_n, forward_returns_window); fixed values are kept and _param_bounds equals the plan's tunable domains.

Verification Results

Submit the buys on the month-end bar as well, sized from the room left after the planned exits (equity*gross - value of kept holdings). If margin checks on pending orders make that unsafe at 1x, state the t+2 entry for new names in the description.

When the month-end rebalance needs any sell (a name leaves the book or a kept name is trimmed), calculate_signal returns before buying. The incoming names are bought on the next session's close and fill at the open of the session after that (t+2), not at t+1 as the source's weights.shift() does. This is a one-session lag on part of each monthly turnover. It is small next to a 22-day hold, but the description does not disclose it.

Set a sells_pending flag and run _rebalance_sells on the next session where no orders are pending.

If orders are still pending on a month-end session, only target_names is updated. Later sessions call _buys only (no _rebalance_sells), so names that dropped out of the book are not sold until the following month-end. Under simulated full market fills this is very unlikely.

Optionally retry the fit no more than once per session month after a failure.

Until the first successful fit, _labelled_rows (a list comprehension over deque indexes) and the fit attempt run on every bar, and again on every bar if _fit keeps returning False. Measured sandbox runtime is fine (11 s), so this is a performance note only.

Nine highly correlated sector ETFs give the classifier 9 cross-sectional observations a day, and 22-day labels overlap heavily, so there are few effective independent samples per annual refit. A 25-feature logistic model on that pool may barely beat noise. The book is 97% invested long at all times, so its drawdown will track SPY (2008 or 2020 could approach the 40% cap). In the backtest review, compare against an equal-weight 9-ETF benchmark, not against cash.

Backtest Review

Sharpe: 0.37
Total return: 241.36%
Max drawdown: 62.76%
Trades: 483
Win rate: 59.6%
Profit factor: 1.40

The code matches the port. It is long-only and uses the 9 sector SPDRs. Each month it buys the top 3 by logistic probability at about one third of equity each. The book is rebalanced on the last session of the month and filled at the next open. The model is refit once a year on labels that were fully known at the time. There are 358 closed trades, all long, which fits roughly 3 names changing over 18 years.

On its own terms the result clears the cost floor: average trade 1.04% of notional against the USEQ floor of 0.05%, profit factor 1.37, Sharpe 0.34, total return +141% from Dec 2002 to Mar 2021 (development period only).

The classifier does worse than holding all 9 ETFs equally. Over the same development window (detail file, both price-only, so dividends are excluded from both), the equal-weight basket grew 4.30x (92.4k to 397.6k). The strategy grew 2.42x (100k to 242.4k). The strategy beat the basket in only 6 of 19 calendar years, and it trailed in 2004, 2007, 2009 and 2016 by 9 to 14 points.

Its drawdown is worse than the market's, not better. The strategy's max drawdown is 62.8%, against 51.5% for the equal-weight basket (1.22x). It is also far above the 40% max_drawdown_pct registered in the research contract. The longest drawdown lasted 2,412 days, and Calmar is 0.08.

Picking 3 of 9 adds concentration risk without adding return. The positive headline Sharpe is lower-volatility market beta minus a selection penalty, not an edge.

Outcome Summary

A cross-sectional selection model needs to beat the equal-weight basket of its own universe, not just make a profit, because in a fully invested long-only sector book the absolute return can be market beta alone.

The analyst abandoned it at backtest review, before optimization, because the premise was falsified: the classifier's picks did worse than holding all nine ETFs equally, beat the basket in only 6 of 19 years and had a deeper drawdown (62.8% vs 51.5%, against a registered 40% limit). Fees were not the cause, and the code was judged a correct port.

A port of QuantRocket's DemoMLStrategy: a numpy L2 logistic classifier, trained on 25 return-horizon features and refit once a year, predicts which of the nine Select Sector SPDR ETFs will beat the cross-sectional median over the next 22 open-to-open days; each month it holds the top 3 at equal weight, long-only on USEQ daily bars.

The full backtest (Dec 2002 to Oct 2026, 483 long trades) returned +241.4% with a Sharpe of 0.37, an average trade of 1.01% of notional, profit factor 1.40, beta 0.93 and a 62.8% max drawdown. On the development period (Dec 2002 to Mar 2021) the strategy grew 2.42x, while an equal-weight basket of all nine ETFs grew 4.30x, and the signal-research IC was -0.05 (t=-0.8).

Analysis

DemoMLStrategy

Analysis

demo-ml.py

Installation and Deployment

Time series / date functionality#

Backtest and paper results are hypothetical. Trading involves risk of loss.