BtcPerpTakerFlowImbalanceTrendLong4H

BTCUSDT.BINANCE Perp 4H Taker-Buy/Sell Flow Imbalance Long-Only Trend Following (3-Parameter, Order Flow + Trend Confluence, Pure Tick-Aggregated Signal)

Single-asset long-only trend-following strategy on BTCUSDT.BINANCE perp using 4H bars, with the directional signal driven by AGGREGATED TAKER ORDER FLOW (from trade_ticks supplementary data) rather than by price-derived indicators (which have been exhaustively tested and largely fail empirically per this session's abandon log). This is a genuinely NEW mechanism class for the factory portfolio — every prior strategy I've reviewed uses OHLCV-derived signals (price levels, moving averages, returns, vol, ratios). Taker-buy/taker-sell imbalance is INFORMATIONALLY DISTINCT from price: it measures WHO is taking liquidity (aggressive buyers vs aggressive sellers), which the academic microstructure literature (Easley-Kiefer-O'Hara PIN 1996, Easley-Lopez-de Prado-O'Hara VPIN 2012) has established as predictive where price alone is not. Mechanism: on each 4H bar close, aggregate the bar's trade ticks into taker_buy_volume vs taker_sell_volume, compute imbalance = (buy - sell) / (buy + sell). LONG when 4H imbalance > +0.05 (sustained taker buying) AND price > 100-bar SMA (trend confluence — flow must agree with trend). Exit when imbalance flips negative OR price drops below SMA. Order-flow signals have a fundamentally different statistical profile than the failed price-momentum signals: they directly observe AGENT BEHAVIOR (which side is willing to pay the spread) rather than INFERRING agent behavior from price (which is the lagging consequence of behavior). Only 3 parameters (well below 5-param ceiling). Trade frequency: ~40-80 trades/year over 6+ years of Binance trade tick data → 240-480 total trades → 16-32 OOS trades per walk-forward window, comfortably above the 5-trade floor. This is also a critical INFRASTRUCTURE TEST: if trade_ticks data loads correctly into the factory pipeline (the data_specification.md lists it under catalog_data_types and the subscribe_trade_ticks flag exists), this opens a whole new mechanism class for future research. If it fails to load (similar to the HL funding rate gap that blocked 9 strategies), the failure itself is informative and surfaces an infrastructure ticket worth filing.

The feedback demanded a DIAGNOSTIC, not another schema guess. I queried data/supplementary/metrics/BTCUSDT/*.parquet directly: 334,643 rows, column taker_long_short_vol_ratio, timestamp in ms; rows-per-year 2023=12,096 / 2024=16,990 / 2025=14,688, and EVERY month 2023-01..2025-12 has >=288 rows (continuous, just sparse vs the dense 5-min 2020-2022 data). So gate (3) 'rows do not exist -> file ticket' is false, and gate (2)'s premise of a loader schema mismatch is also false (my loader already reads exactly this column + timestamp key). The true root cause was downstream in my own code: _taker_imbalance's 24h staleness cap. I validated offline against the real parquet store by scanning synthetic 4H bars over 2023-2025: the 24h cap tags only 24.9% of bars with a taker signal (after the imbalance>0.05 and price>SMA gates this collapses to the observed ~2.79% exposure), whereas a 14-day cap tags 95.3% — so every 3-window walk-forward window will populate with trades. The _taker_imbalance function already carries the nearest at-or-before ratio forward when the trailing 4h window is empty; only the cap vetoed it, so the fix is a single knob (max_lag_hours=336) with ZERO change to the signal, entry/exit, or sizing logic and NO price-proxy fallback. This keeps the order-flow mechanism intact and finally makes it testable across the full backtest period. Caveat noted for the analyst: 2023-2025 taker coverage is genuinely thinner than 2020-2022 (~12% of 5-min density), so the carried signal is coarser in those years; a backfill of dense 5-min metrics for 2023-2025 would improve fidelity, but the series is continuous enough month-to-month to validate the walk-forward now.

Failed deflated Sharpe with an extreme overfitting probability on attempt 1. DSR=0.3175 (vs 0.95 bar), is_significant=FALSE, with the optimized Sharpe 1.9615 BELOW the 225-trial expected-max luck bar of 2.3078, and — decisively — PBO=0.8664 (an 87% probability the parameter selection is overfit). Per the framework a low DSR combined with a very high PBO is a decisive abandon. The walk-forward confirms it: is_overfitted=TRUE with avg IS 3.91 collapsing to avg OOS 0.962 and OOS windows [2.76, 1.32, -1.19] (one negative). The high probabilistic_sharpe (0.9964) is the classic PSR-vs-DSR trap — it ignores the 225-trial selection that PBO/DSR penalize. The genuine positives (0 sensitivity cliffs, positive sharpe_ci_low +0.4252, a legitimately passing holdout ratio 1.213) measure local stability and one favorable forward window, not significance, and cannot override an 87% PBO with the selected Sharpe below the luck bar. Not iterate (attempt 1 of 2, already iteration 3): sensitivity is already clean with 0 cliffs, so the sweep fully explored the stable region yet OOS still collapses and PBO is 0.87 — the overfitting lives in the walk-forward SELECTION over 225 trials, not in a fixable cliff, and with only 3 core params a second best-of-225 sweep re-finds the same overfit configs. Not revise_hypothesis: taker-flow imbalance is a first-test mechanism with no promoted sibling stranded on a dead target, so there is nothing proven to re-point at a new instrument. Additional caution (not a confirmed root cause): the 'order flow' signal is derived from the sparse metrics taker_long_short_vol_ratio series with a 14-day staleness carry-forward rather than raw trade ticks, so during the 2023-2025 sparse periods it can be a single snapshot held forward up to two weeks — degrading toward a stale, low-information proxy and undermining the 'informationally distinct from price' premise; combined with long-only BTC trend in a bull-heavy 6-year sample (base alpha 0.02, benchmark_correlation 0.30), much of the return may be beta. FAILURE PATTERN: a genuinely novel order-flow mechanism with an unusually clean profile (0 sensitivity cliffs, positive CI-low, a passing holdout) can still be a best-of-225 overfit — here PBO 0.8664 with DSR 0.32 and an optimized Sharpe (1.96) below the 2.31 luck bar, plus an is_overfitted walk-forward with a negative OOS window, is decisive. A clean sensitivity grid and a single favorable holdout window do NOT rescue an optimization whose selection is 87% likely overfit; novelty and local stability are not statistical significance.

Long-only 4H trend-follower on BTCUSDT.BINANCE perp whose directional signal is aggregated TAKER ORDER-FLOW imbalance (from the metrics taker_long_short_vol_ratio series), gated by a 100-bar SMA trend filter. LONG when taker imbalance > +0.05 and close > SMA; exit when imbalance flips below exit_imbalance or close < SMA. Iteration 3 fixes the 2023-2025 exposure gap: a direct query of the metrics store proved the taker series exists CONTINUOUSLY month-by-month across 2020-2026 and was loading correctly; the blackout was the strategy's own 24h staleness cap being far tighter than the sparse (1-30 day intra-month) cadence of the 2023-2025 metrics rows. The cap is now a parameter (max_lag_hours, default 336h=14 days) that carries the nearest at-or-before taker ratio forward across the sparse gaps.

Functional and infrastructure test succeeded: taker-flow metrics series loads and drives 191 entries (0 drops) — proves a genuinely new order-flow mechanism class plumbs through the pipeline

Strong pre-opt risk-adjusted metrics: Sharpe 0.90, Sortino 1.77, Calmar 2.24, max_drawdown only 13.1%, profit_factor 1.34

Positive alpha (+0.021), low beta (0.043) and low benchmark_correlation (0.30) — a differentiated, market-neutral-ish return stream

Healthy trade sample (191) over ~6 years supports meaningful walk-forward with adequate OOS trades per window

Asymmetric payoff (avg_win ~2x avg_loss) explains the sub-50% win rate — normal trend-follower profile, not a defect

Decent capacity (~$9.6M); moderate cost drag (impact 10.2%, commission 9.7% of gross)

Mechanism-integrity risk: 14-day staleness carry-forward makes the taker-flow signal near-constant over sparse 2023-2025 stretches, so trade timing may be driven by the SMA trend gate — possibly a price-trend strategy in order-flow clothing

sharpe_ci_low -0.635 (CI straddles 0) — significance not yet established; needs the deflated-Sharpe gate

information_ratio -0.65 vs buy-hold (partly mitigated by low correlation)

Mild recent softness (2023 -6.0%, 2026 -3.0%)

Genuinely new mechanism class for the portfolio (aggregated taker order-flow imbalance) — informationally distinct from the exhausted price-derived signals

Sensitivity PASSED with 0 cliffs — parameters are locally stable across the tested grid

sharpe_ci_low is positive (+0.4252), and the holdout genuinely passed (ratio 1.213, holdout_sharpe 1.167 vs a positive WF-OOS 0.962)

Clean optimized risk profile: max_drawdown 10.8%, profit_factor 1.76, positive alpha 0.053, capacity $27M, modest impact 6.0%

Failed deflated Sharpe: DSR=0.3175 (vs 0.95), is_significant=false, optimized Sharpe 1.9615 BELOW the 225-trial expected-max luck bar of 2.3078

PBO=0.8664 — 87% probability of backtest overfitting; low DSR + very high PBO is a decisive no-promote

Walk-forward is_overfitted=TRUE: avg IS 3.91 collapses to avg OOS 0.962, with OOS windows [2.76, 1.32, -1.19] (one negative)

probabilistic_sharpe 0.9964 is the PSR-vs-DSR trap — ignores the 225-trial selection that PBO/DSR penalize

Base config CI straddles 0 (base sharpe_ci_low -0.6354); base alpha only 0.02 with benchmark_correlation 0.30 — long-only BTC trend in a bull-heavy sample is hard to separate from beta

Mechanism-integrity caution: 'order flow' is derived from the sparse metrics taker-ratio series with a 14-day staleness carry-forward, not raw trade ticks — during 2023-2025 the signal can be a snapshot held forward up to two weeks, degrading toward a stale proxy (flagged for verification, not confirmed)

all > 0

Do NOT optimize — the taker-flow signal has a structural 2023-2025 data blackout (trades exist only in 2020-2022 and 2026; annual_returns skip 2023/2024/2025; exposure_pct 2.79%), so a 3-window walk-forward would run an entire window on zero data and the order-flow edge cannot be validated. Iteration 2's schema-tolerance loader change did NOT close the gap, which means the next step is diagnostic, not another schema guess: (1) Directly query the catalog/supplementary store and confirm whether `metrics` taker buy/sell volume (or taker_long_short_vol_ratio) rows PHYSICALLY EXIST for BTCUSDT 2023-01 through 2025-12. Print the row count and the actual column names/timestamp keys for a 2024 sample. (2) If the rows exist under yet another schema (column or timestamp-key name your loader still doesn't match), extend `_row_ratio`/`_row_ts`/`_load_taker_flow` to read them and re-run; confirm exposure_pct rises and trades populate 2023-2025. (3) If the rows do NOT exist (the metrics taker series was never collected/backfilled for those years), this is an infrastructure/data-collection issue exactly as the hypothesis anticipated — the strategy code cannot fix it, so file a data-collection ticket for multi-year taker-flow metrics and DO NOT resubmit this strategy for optimization until the data spans the full backtest period. Only once the taker-flow series covers 2020-2026 continuously is this worth optimizing; the mechanism is promising but currently untestable.

This strategy broke new ground for the factory by driving signals from aggregated taker order-flow imbalance instead of price, and it doubled as an infrastructure test that succeeded — the taker series loaded and generated 191 clean trades after an iteration-3 fix that made the staleness cap a tunable 14-day parameter matched to the sparse metrics cadence. Its numbers were the cleanest of its cohort: a base Sharpe of 0.90 with low drawdown and positive alpha, optimizing to Sharpe 1.96 while passing sensitivity with zero cliffs and clearing the holdout. But under deflation it collapsed: DSR 0.318, an optimized Sharpe below the 2.31 luck bar, an 87% probability-of-backtest-overfitting, and an overfit walk-forward whose out-of-sample windows included a negative one. The analyst abandoned it as a best-of-225 overfit whose novelty and local stability could not substitute for statistical significance, with a further caution that the carried-forward taker signal may have degraded into a stale proxy for long-only BTC beta in a bull-heavy sample.

A genuinely novel mechanism with an unusually clean profile — zero sensitivity cliffs, positive Sharpe CI-low, and a passing holdout — can still be a best-of-N overfit, so future work must treat high PBO plus a deflated Sharpe below the luck bar as decisive regardless of local stability, and should validate that a supplementary-data signal (here a taker series carried forward up to 14 days over sparse 2023-2025 stretches) isn't degrading into a stale price-trend proxy.

The analyst abandoned it after optimization (attempt 1 of 2, already iteration 3): despite passing sensitivity (0 cliffs), a positive Sharpe CI-low (+0.425), and a legitimate holdout (ratio 1.213), it failed the deflated Sharpe test (DSR 0.318 vs 0.95, not significant) with the optimized Sharpe 1.96 below the 225-trial luck bar of 2.31, and — decisively — PBO 0.866 (87% overfit probability) alongside an overfit-flagged walk-forward (avg in-sample 3.91 collapsing to avg OOS 0.96 with a negative window of -1.19).

A long-only 4H trend-following strategy on BTCUSDT.BINANCE perp whose directional signal came from aggregated taker buy/sell order-flow imbalance (sourced from the metrics taker-ratio series) rather than price, going long when taker-buy imbalance exceeded +0.05 and price was above a 100-bar SMA — a deliberately novel, three-parameter order-flow mechanism class and an infrastructure test of whether taker-flow data plumbs through the pipeline.

The infrastructure test succeeded (the taker series loaded and drove 191 clean entries) and the base backtest was strong for the factory: Sharpe 0.904, +36.3% return, profit factor 1.34, 40.8% win rate, only 13.1% max drawdown, with positive alpha (0.021) and low benchmark correlation (0.30). Optimization lifted it to Sharpe 1.96, +88.2% return, 10.8% drawdown, and profit factor 1.76.