Skip to content

View original

EthNewsSentimentShockDriftLS

Hypotheses

ETH News-Sentiment Shock Drift — Long-Short, Single-Instrument: Enter the Direction of a Sharp LLM-Scored News-Sentiment JUMP and Ride the Multi-Hour Underreaction Drift, Hard-Stopped, Time-Boxed (BINANCE ETHUSDT USD-M, 15m Bars + news_sentiment Supp, Short-Lookback, 2-Parameter)

Hypotheses

A LONG-SHORT, single-instrument, EVENT-DRIVEN directional strategy on ETHUSDT.BINANCE USD-M perp whose trigger is NOT price-derived at all — it is the LLM-scored, per-asset NEWS-SENTIMENT feed (news_sentiment supp key, 5-min buckets), a data source with ZERO prior strategies in the 2663-experiment portfolio. This is a deliberately fresh family: not OHLCV trend-confluence (0/213, killed twice), not the CME-gap-fill family (just collided/abandoned), not funding/OI gates (L70/L71), not cross-venue carry (L69), not rotation baskets (L68), not fee-graveyard microstructure, not options. ECONOMIC PREMISE: when a materially new piece of news lands, the sentiment score for that asset JUMPS sharply within a 5-15 min bucket, but spot/perp price tends to UNDERREACT to the full information content initially and then DRIFT in the sentiment direction over the following hours as the news diffuses (post-news-drift / underreaction, the sentiment analogue of post-earnings-announcement drift). We enter in the direction of a large sentiment jump and ride the drift with a hard stop and a same-session time-box, so the strategy is FLAT except in the minutes after a genuine news shock — accruing almost no fee drag and taking only high-expected-move trades. Because the move captured is a multi-hour information-diffusion drift (typically 1-3%), it clears the fee floor by a wide margin, unlike the sub-fee microstructure ideas in the graveyard. HISTORY CAVEAT (stated up front for the analyst): news_sentiment collection began 2026-08-13, so this is an explicitly SHORT-LOOKBACK design (~3 weeks); the drift edge is validated on the accrued span and the analyst judges sufficiency — the history will keep accruing. Long-short (fills the starved 14.3% bucket). Only 2 tunable parameters (sentiment-jump threshold, hard-stop ATR multiple) to resist the overfit graveyard.

Hypotheses

Iteration 3 makes the two smallest changes that fix the Layer-4 findings, leaving the signal path (which passed Layers 1-3) untouched. (1) The hard finding — the optimizer's search space contradicted the hypothesis's core overfit-resistance claim: _param_bounds declared 7 dimensions while the design promises 2, and a ~225-trial 7-D Bayesian search over the ~33 trades the ~3-week news_sentiment history supports is a guaranteed curve-fit. _param_bounds now declares only entry_pctile [0.90, 0.995] and stop_atr_mult [1.5, 5.0], and — to keep the declared-clamp rule satisfied — every other setting is now read with plain validity guards (max(2, ...), a min_baseline <= baseline consistency clamp) instead of min/max tuning ranges, so those two are the only range clamps left and both are mirrored exactly in the bounds. (2) The premise-drift finding — a purely self-referential percentile fires on ~3% of valid bars in any regime, so a dead-news window with collapsed dispersion could turn a trivial wiggle into a large z and manufacture a trade. Entry now additionally requires the RAW sentiment deviation from its baseline mean to reach min_abs_jump = 0.10 (about 0.77 sd of the real ETH feed, whose std is 0.13); a genuine shock moves sentiment ~0.3+ units and passes, quiet-regime noise does not. That floor is deliberately fixed construction, not a third tunable. Layer-3 safety was the deciding constraint on how hard to tighten: iteration 1 died at zero trades because a fixed z of 2.5 is unreachable on the sandbox's near-uniform synthetic sentiment, so I did NOT raise min_entry_z (which would risk the same failure); the absolute-jump floor is instead expressed in sentiment units, where synthetic near-uniform values deviate ~0.4 from their mean and clear 0.10 comfortably, while real quiet-window noise does not. I agree with QA's third point and state it plainly rather than arguing it: the Layer-3 sandbox numbers are not a fee/edge test for this strategy because the synthetic sentiment driver is near-random noise; real fee viability is decided at the BACKTESTING stage over the genuine ~3-week accrued span, which the analyst should judge for sufficiency (my offline read of the real feed gives ~33 trades at roughly +0.4% per trade net, per-trade SD ~1.9%, i.e. an SE of ~0.33% — suggestive, not yet statistically established).

Hypotheses

data_insufficiency_unoptimizable_short_lookback: the strategy correctly implements a fresh, non-graveyard mechanism (LLM news-sentiment shock drift) and its trades match the hypothesis, but the news_sentiment feed began only 2026-08-13, so the entire live window is ~18 trading days / 37 trades — the equity curve is flat at 100k until then. That sample is statistically unmeasured (L10) and cannot support the 3-phase optimization: walk-forward IS/OOS windows collapse to ~5-6 days and the holdout to ~3-4 days, and a ~225-trial best-of-N search over even 2 params on 37 trades curve-fits noise and dies at DSR/holdout. The headline +6.76% is outlier-carried (skew 11.4, kurtosis 692, one +4.16% day dominating) and Sharpe 0.107 has a bootstrap CI [-0.585, 0.616] straddling zero. This is not a fee death (avg_trade_return_pct 0.539% clears the floor) nor a code/premise defect — no parameter change or instrument/venue reframe helps because every news_sentiment series shares the 2026-08-13 start. The idea is not refuted, just unmeasurable today; it can be re-proposed once the feed history accrues to a multi-month span. Not worth the 2-hour optimization budget now.

Implementation

Long-short, single-instrument, event-driven strategy on ETHUSDT.BINANCE USD-M 15m bars whose trigger is the LLM-scored news_sentiment supplementary feed, not price. Each bar it computes a continuous jump z-score of the newest 5-minute sentiment bucket against its own prior ~6h baseline. It enters in the jump's direction when |z| clears the rolling entry_pctile quantile of its own recent |z| history (floored at min_entry_z) AND the raw sentiment deviation clears an absolute materiality floor (min_abs_jump), then rides the multi-hour post-news underreaction drift with a hard stop at stop_atr_mult x ATR (tested intrabar against the bar's low/high) and a 24-bar (6h) time box. Risk-first sizing: qty = equity * 0.8% / stop distance, gross notional capped at 50% x equity x leverage. If the sentiment feed is missing, cold, or staler than 30 minutes, the signal is 0.0 and the strategy stands flat — it never substitutes a price-derived proxy.

Verification Results

Verification failed (Layer 4 — QA review): - OPTIMIZATION SURFACE CONTRADICTS THE HYPOTHESIS'S CORE DESIGN. The hypothesis's central overfit-resistance claim (in the title and body: 'Only 2 tunable parameters (sentiment-jump threshold, hard-stop ATR multiple) to resist the overfit graveyard') and the code docstring ('TUNABLE SURFACE - exactly 2 parameters: entry_pctile and stop_atr_mult') both promise a 2-dimensional search. But config.parameters['_param_bounds'] declares SEVEN optimizable parameters: entry_pctile, stop_atr_mult, min_entry_z, pctile_window, pctile_min_obs, baseline_buckets, min_baseline_buckets. The optimizer treats _param_bounds as its literal search space (src/optimization/walk_forward.py:_declared_bounds -> Optuna dimensions; same in sensitivity.py / parallel_runner.py), so the 3-phase optimization will Bayesian-search 7 params, not 2. This is not cosmetic: with news_sentiment history only ~3 weeks old (collection began 2026-08-13) the real feed yields only ~33 trades (developer's own figure, per-trade SD ~1.9%, SE ~0.33%). A 7-parameter, 75-trials x 3-window (~225-trial) search over ~33 trades is a textbook curve-fit and defeats the exact 'overfit graveyard' the hypothesis was constructed to avoid. The stated structural constraint (2 params) must match what the code/config actually exposes; it does not. - The entry gate is a self-referential rolling percentile: _entry_threshold() returns the entry_pctile (0.97) quantile of the strategy's OWN recent |z| history, floored only at min_entry_z=1.0. By construction this makes roughly (1 - entry_pctile) ~= 3% of valid-z bars clear the threshold in ANY regime, so the book is not truly 'FLAT except in the minutes after a genuine news shock' as the hypothesis states - in a quiet news regime it still fires on the top ~3% of its own (small) jumps once |z| exceeds 1.0. The min_entry_z floor mitigates but does not eliminate this; a genuinely dead-news window with |z| noise above 1.0 will still manufacture trades. This is a partial premise drift, not a coding bug. - The Layer-3 sandbox result is not a valid fee/edge test and must not be read as one. avg_trade_return_pct is 0.0317% (far below the 0.15% futures floor), total_return -2.6%, Sharpe -0.67, skew 8.6, kurtosis 133.9 over 57 trades across 363 data_days - i.e. the sentiment driver in the sandbox is synthetic/near-random noise, not the real ETH feed (real news_sentiment only spans ~3 weeks, so 57 trades over 363 days cannot be the real feed). Real-data fee viability rests entirely on the developer's UNVERIFIED offline claim of +0.436% per trade on 33 trades. Code correctly reads news_sentiment and the loader (supp_loaders.load_news_sentiment) is wired, so the BACKTESTING stage WILL use real data - but only over the ~3-week accrued span.

Verification Results

Analyst should gate on trade count / significance at backtest_review over the real accrued span before spending the 3-phase optimization.

Verification Results

Short-lookback / low-trade-count power risk. The news_sentiment feed accrues only from 2026-08-13 (~3 weeks), so the developer estimates ~33 trades on the real feed — far below the ~100-trade bar for distinguishing edge from noise (per-trade SD ~1.9% vs claimed ~+0.4% mean gives SE ~0.33%). Genuine measurability concern for the analyst, but not a code defect: the hypothesis declares the short-lookback design and defers sufficiency to the analyst.

Verification Results

Treat Layer-3 metrics as plumbing/liveness only for this feed-driven design; test fee viability on the real news_sentiment span at backtesting.

Verification Results

Sandbox result is not a valid edge test. Layer-3 evaluated 206 days (39 trades) but the real feed spans ~3 weeks, so the driver was near-random synthetic sentiment. The -6.43% / PF 0.61 / avg_trade_return_pct -0.031% is the expected outcome of trading noise net of fees, not a refutation of the post-news-drift premise.

Backtest Review

Genuinely fresh family — news_sentiment feed, zero prior strategies in the 2663-experiment portfolio; not an OHLCV trend/fee-graveyard clone

Backtest Review

Trades faithfully implement the hypothesis: flat until the feed begins (2026-08-13), then ~2 event trades/day, long+short (25/12), ATR-stopped and time-boxed, 5h avg hold

Backtest Review

Not a fee death — avg_trade_return_pct 0.539% clears the 0.15% Binance USD-M floor; profit_factor 1.82; max_drawdown ~0.9%

Backtest Review

Disciplined 2-tunable surface with fixed materiality/lag guards, addressing the overfit graveyard by design

Backtest Review

Only ~18 trading days of usable history (news_sentiment began 2026-08-13); the 2019-2026 span is entirely flat until then

Backtest Review

37 trades is statistically unmeasured (L10) — far too few for a meaningful edge estimate

Backtest Review

Cannot support the 3-phase optimization: walk-forward IS/OOS windows would be ~5-6 days each and the holdout ~3-4 days — not enough to validate anything; a ~225-trial best-of-N search on 37 trades will curve-fit and die at DSR/holdout

Backtest Review

Result is outlier-carried: return_skew 11.4, return_kurtosis 692, with a single +4.16% day (2026-08-19) dominating the +6.76% total; Sharpe 0.107 with bootstrap CI [-0.585, 0.616] straddles zero

Backtest Review

No fix is actionable now — every news_sentiment series shares the 2026-08-13 start, so no instrument/venue reframe adds history

Outcome Summary

EthNewsSentimentShockDriftLS was a genuinely fresh idea — the first strategy in the 2,663-experiment portfolio to trade the LLM news-sentiment feed — entering on sharp sentiment jumps to capture the multi-hour post-news drift on ETH. Across three iterations the developer fixed the signal gate (zero-trades from a fixed z-threshold became a rolling percentile) and tightened the optimization surface to the promised two parameters with a materiality floor, producing a faithful implementation that traded profitably on the surface (PF 1.82, +6.76%, 0.9% drawdown). But the news_sentiment feed only started 2026-08-13, so the live sample was ~18 days and 37 trades, with the headline dominated by a single outlier day and a Sharpe CI straddling zero. The analyst abandoned it at backtest review as unmeasurable and unable to support optimization — not refuted, but re-proposable only once the feed history accrues to a multi-month span.

Outcome Summary

A genuinely novel, non-graveyard mechanism can still be un-promotable purely for lack of history — an event-driven edge whose feed spans only weeks cannot yield enough trades to survive optimization and holdout validation, and an outlier-carried headline over ~18 days is not evidence of edge; re-propose once the feed accrues months.

Outcome Summary

It was abandoned at the pre-optimization BACKTEST_REVIEW gate for data insufficiency: the news_sentiment feed only began 2026-08-13, so the entire live window is ~18 days / 37 trades — statistically unmeasured and too short to support the 3-phase optimization (walk-forward windows would collapse to ~5-6 days, holdout ~3-4 days), where a best-of-N search on 37 trades would curve-fit and die at DSR/holdout. It was explicitly not a fee, code, or premise defect — just unmeasurable today — and no instrument/venue reframe helps since every sentiment series shares the same start date. Optimization and analysis were not reached.

Outcome Summary

A long-short, single-instrument event-driven strategy on ETHUSDT USD-M perp (15m bars) that entered in the direction of a sharp jump in an LLM-scored news-sentiment feed (a rolling z-score plus an absolute-jump floor) to ride the multi-hour post-news underreaction drift, hard-stopped and time-boxed, staying flat except right after a genuine news shock.

Outcome Summary

Over the accrued ~18 trading days of feed history it produced 37 trades (25 long / 12 short) with profit factor 1.82, avg_trade_return_pct 0.539% (clearing the 0.15% floor), total return +6.76%, and only ~0.9% max drawdown. However the result was outlier-carried (return skew 11.4, kurtosis 692, a single +4.16% day dominating the total) and Sharpe was just 0.107 with a bootstrap CI [-0.585, 0.616] straddling zero.

Iteration History

Verification failed (Layer 3 — sandbox backtest): No trades produced Bar type used: ETHUSDT.BINANCE-15-MINUTE-LAST-EXTERNAL, Bars processed: 34852 Diagnostics: should_enter() returned a side 0 times over 34803 evaluated bars -> your ENTRY CONDITION never triggered. Loosen the entry logic / thresholds. Ensure your strategy produces trades with the given data and parameters.
Strategy report

Backtest and paper results are hypothetical. Trading involves risk of loss.