Strategies

Quantiacs LSTM Multiple-Features Port: AAPL Daily Next-Open-Up Regressor, Raw Output as Target Weight (AAPL.USEQ 1-DAY, 2x64 LSTMCell, LBFGS, Retrain Every 360 Bars, Fixed Source Defaults)

Outcome: Abandoned

QuantiacsLstmMultiFeatureAaplUseq

Outcome Summary

The strategy attempted a faithful NumPy port of Quantiacs’ LSTM model for daily AAPL target weights, with a declared initialization RNG deviation. Across five iterations, execution fidelity remained unresolved because the platform lacked the required next-session-open hook. The final sandbox verification processed 0 bars and failed, leaving performance unevaluated and the run abandoned before optimization or later reviews.

Strategy report

Hypothesis

Faithful port of the open-source MIT strategy 'strategy-ml_lstm_multiple_features' by Quantiacs (https://github.com/quantiacs/strategy-ml_lstm_multiple_features, commit 51f5c6d). Credit goes to the original author. Any performance the source claims is unverified; our own backtest with real costs decides. The source asset NAS:AAPL maps to AAPL.USEQ on 1-DAY bars (split-adjusted, decades of history). The source backtest starts on 2006-01-01. It uses one instrument and OHLCV only, with no supplementary data. MODEL: two stacked LSTMCell layers (input_dim=3, hidden=64) followed by Linear(64->1). Hidden and cell states start at zero at the beginning of each sequence, and the model gives an output at every step. Gates follow standard PyTorch LSTMCell order i,f,g,o with both bias_ih and bias_hh. FEATURES per bar, in this order: [ln(close), ln(open), high]. The source does NOT log the high; this quirk is kept on purpose. Missing values are forward-filled, then back-filled, then filled with 1. TARGET for bar t: 1 if open[t+1] > open[t], else 0. In a training slice the last bar has no next open, so xr.where gives it label 0; this is kept. TRAINING: use the last train_period=100 bars up to and including the current bar as one batch of size 1. Loss is MSE over the 100 per-step outputs. The optimiser is L-BFGS with lr=0.08 and PyTorch defaults (max_iter=20, max_eval=25, tolerance_grad=1e-7, tolerance_change=1e-9, history_size=100, no line search), and exactly one optimiser.step call (epochs=1). Each retrain builds a fresh seed-42 model. Retraining happens every retrain_interval=360 bars; retrain_interval_after_submit is live-only and not used here. PREDICTION (predict_each_day=False, lookback_period=155): after each retrain, the model starts from zero state 155 bars before the retrain bar and runs causally until the next retrain. The output at bar d is the target weight for bar d. POSITION: w = clip(output, -1, 1). A positive w means long w x equity; a negative w means short |w| x equity. The position is rebalanced every daily bar to whole shares, decided at the close and executed at the next available price. The source has no stop or take-profit. Leverage is 1.0. DECLARED UNAVOIDABLE DEVIATION: PyTorch cannot be imported in the sandbox. The LSTMCell forward pass, the backprop-through-time gradient and PyTorch L-BFGS are therefore re-implemented in numpy. Initial weights are drawn U(-0.125, 0.125), PyTorch's default, from numpy default_rng(42) in PyTorch parameter order. Their distribution matches, but the exact torch RNG stream does not. Everything else is line-for-line.

This is a concrete, executable strategy, ported to its exact source instrument and bar size. USEQ is under-represented (6.2%), and the strategy adds an ML mechanism family instead of another trend variant. Parameters are frozen at the source defaults because this is a faithful reproduction. The only deviation is the torch-to-numpy initialisation RNG, declared up front.

verification_loop: Verification failed (Layer 3 — sandbox backtest): execution_contract_unsupported: AAPL daily close decisions require unconditional execution at the following session open. FactoryStrategy exposes no supported next-session-open hook. The OCO stop/market-if-touched approximation misses unchanged opens and flat sessions and has been disabled. Engine support is required before this hypothesis can be evaluated faithfully. Bar type used: AAPL.USEQ-1-DAY-LAST-EXTERNAL, Bars processed: 0 Ensure your strategy produces trades with the given data and parameters.

Implementation

Quantiacs MIT AAPL daily LSTM translation with the previous numpy model, BPTT, L-BFGS, scheduled retraining and target-weight calculations preserved. Order submission now explicitly reports execution_contract_unsupported instead of submitting the inaccurate OCO approximation. This version is blocked pending engine support.

QA's critical execution finding is valid: triggers one tick above and below the prior close cannot execute unconditionally at an unchanged next-session open, and a flat session can leave both orders unfilled. Repository inspection found no supported unconditional next-session-open hook in FactoryStrategy or its backtest integration. The smallest correction replaces only the OCO submission body with an explicit unsupported-contract error, preserving the previously passing model and signal logic. No execution-case tests were run because no supported hook exists and verification belongs to the pipeline. QA's warmup warning remains an explicit prerequisite: reconstructing the preceding 360-session model can require 513 prior sessions, while the registered evaluation supplies 160. Increasing min_bars_required cannot retrieve omitted history; fixing this requires a research-approved warmup-contract revision or restoration of the scheduled model and recurrent state. The fixed plan and original retraining anchors were not changed. This is a platform-support limitation, not evidence that the hypothesis lacks an edge.

Verification Results

Verification failed (Layer 3 — sandbox backtest): execution_contract_unsupported: AAPL daily close decisions require unconditional execution at the following session open. FactoryStrategy exposes no supported next-session-open hook. The OCO stop/market-if-touched approximation misses unchanged opens and flat sessions and has been disabled. Engine support is required before this hypothesis can be evaluated faithfully. Bar type used: AAPL.USEQ-1-DAY-LAST-EXTERNAL, Bars processed: 0 Ensure your strategy produces trades with the given data and parameters.

In the sandbox the strategy behaves like a mostly-long, partly-sized AAPL holder: beta 0.415, benchmark correlation 0.94, Sharpe 0.89 and information ratio -0.76 against buy-and-hold. The model regresses a 0/1 next-open-up label, so its outputs sit mostly in (0,1). That makes it a fractional-long AAPL position rather than a directional timing signal. Check whether the full-history return beats beta x AAPL. The ledger is also a concern: the position never went flat in the sandbox (0 closed positions, 1 open, 1230 fills). So total_trades and avg_trade_return_pct are 0 and say nothing about per-trade economics. The analyst should judge on the MTM equity curve and IR vs buy-and-hold, not on the closed-trade ledger.

Analysis

Backtest gate: - backtest produced ZERO trades — the entry condition never fired on real data

Iteration History

Verification failed (Layer 4 — QA review) [class=hypothesis_mismatch]: - [edge_concern] Sandbox returns closely track AAPL exposure: benchmark correlation 0.9563, beta 0.4373 and alpha -0.0131. Full-history analysis should distinguish predictive value from passive equity exposure. Zero closed positions does not establish a defect. - [critical] The hypothesis requires retraining every 360 daily trading bars. The implementation buckets a weekday index that includes US exchange holidays, retraining after approximately 351 sessions instead. This changes training slices, model resets and predictions. The brief authorizes only the numpy/PyTorch implementation deviation; the additional schedule deviation in the developer rationale does not amend the hypothesis. (line 324) — fix: Derive a fixed retraining grid from actual USEQ sessions anchored to the source start date. Preserve exactly 360 trading bars between retrains without using a restart-relative counter. - [critical] Daily weights use the completed bar's close, open and high, then immediately submit market orders. Initial entries follow the same immediate submission path in the base template. The pinned engine documentation states market orders execute at the current simulated market price; here that is the completed bar's close. The hypothesis explicitly requires close decisions executed at the next available price. Same-close fills change the execution contract and include overnight exposure unavailable under next-session execution. (line 378) — fix: Queue the close-derived target and execute through a verified next-session price event. Add the required engine/feed support if daily bars alone cannot represent that event; delaying to the next daily close would also differ from the stated contract. - [critical] The training anchor depends on the loaded history buffer: max(idx[0], lookback_period-1, train_period-1). When a window or restart begins inside a retraining block, the model trains on a later available bar rather than reconstructing the scheduled model. Consequently, identical current timestamps can yield different training data and predictions depending on replay start. A 160-bar warmup cannot generally reconstruct a model trained earlier in a 360-bar block plus its 155-bar prediction initialization. (line 281) — fix: Load history back through the scheduled training anchor and its prediction lookback, or restore validated model and recurrent state. Do not substitute a later training anchor when history is insufficient.

Derive a fixed retraining grid from actual USEQ sessions anchored to the source start date. Preserve exactly 360 trading bars between retrains without using a restart-relative counter.

The hypothesis requires retraining every 360 daily trading bars. The implementation buckets a weekday index that includes US exchange holidays, retraining after approximately 351 sessions instead. This changes training slices, model resets and predictions. The brief authorizes only the numpy/PyTorch implementation deviation; the additional schedule deviation in the developer rationale does not amend the hypothesis.

Queue the close-derived target and execute through a verified next-session price event. Add the required engine/feed support if daily bars alone cannot represent that event; delaying to the next daily close would also differ from the stated contract.

Daily weights use the completed bar's close, open and high, then immediately submit market orders. Initial entries follow the same immediate submission path in the base template. The pinned engine documentation states market orders execute at the current simulated market price; here that is the completed bar's close. The hypothesis explicitly requires close decisions executed at the next available price. Same-close fills change the execution contract and include overnight exposure unavailable under next-session execution.

Load history back through the scheduled training anchor and its prediction lookback, or restore validated model and recurrent state. Do not substitute a later training anchor when history is insufficient.

The training anchor depends on the loaded history buffer: max(idx[0], lookback_period-1, train_period-1). When a window or restart begins inside a retraining block, the model trains on a later available bar rather than reconstructing the scheduled model. Consequently, identical current timestamps can yield different training data and predictions depending on replay start. A 160-bar warmup cannot generally reconstruct a model trained earlier in a 360-bar block plus its 155-bar prediction initialization.

Set evaluation.warmup_bars to >= 515. If short windows matter, raise the base _max_bars for this strategy (or keep its own buffer fed from on_bar) so a mid-block window can rebuild the scheduled model.

The _optimization_plan.evaluation.warmup_bars of 160 is shorter than this strategy needs. It needs the scheduled anchor (session 360*b) plus 154 earlier bars in its buffer, so up to 359+155 = 514 bars before an arbitrary window start. A reset_with_warmup window therefore stays flat until the next scheduled retrain unless an anchor lies at or after warm-up start + 154. That can be up to 360 sessions, so short validation windows will be mostly or entirely flat and underpowered. The base template also keeps only _max_bars=500 primary bars, so seeding from self._bars cannot reach anchors more than ~345 bars back even with a longer warm-up. This is acknowledged in the rationale and is faithful (no substituted anchor), and it does not affect the single long full-history run that starts in 1980.

Acceptable. Optionally track the cancelled order-list id and ignore those orders in the orders_open guard.

_cancel_stale_orders calls cancel_all_orders at the start of calculate_signal. If the cancel has not been processed by the time should_exit / _process_primary checks orders_open, that session's rebalance is skipped and re-decided on the next bar. The effect is a one-session lag on rare no-trigger days (open within one tick of the prior close and no intrabar touch). It is not a correctness break.

None required. Keep max_abs_weight and leverage fixed at 1.0 as the plan does.

This is an always-in design with no stop, rebalanced daily from get_account_equity(). Lesson L148 cautions against such designs. Here the risk is bounded: |w| <= max_abs_weight 1.0 (an explicit clamp), leverage 1.0, one instrument, so worst-case gross is 1.0x equity, within USEQ's 2x Reg-T. The source has no stop, so adding one would break the port. Sandbox max drawdown was 16.7%.

Verification failed (Layer 1.5 — hypothesis/config consistency) [class=hypothesis_mismatch]: - optimization_plan fixes leverage at 1.0

Verification failed (Layer 4 — QA review) [class=hypothesis_mismatch]: - [critical] The OCO trigger pair does not guarantee the required next-session-open execution. With previous close 100.00 and tick 0.01, triggers are 100.01 and 99.99. If the next session opens at 100.00, neither order triggers at the open. A later intraday crossing executes the rebalance at a different price; a session remaining between the triggers leaves it unfilled until cancellation. This changes the hypothesis's execution contract. The no-op flush callback does not resolve this gap. The hypothesis declares only the numpy implementation deviation, not conditional intraday execution. (line 517) — fix: Use an engine-supported next-session-open execution hook that executes the queued close decision unconditionally at the following session's open. Verify unchanged-open, gap-up, gap-down, and flat-session cases. If that hook is unavailable, report the execution contract as unsupported rather than treating the OCO approximation as faithful. - [warning] The fixed evaluation warmup of 160 bars cannot reconstruct every scheduled model. A window beginning late in a 360-session bucket needs the scheduled anchor plus its preceding 154 prediction-history bars, potentially 513 prior sessions. If those bars were not supplied, _retrain returns false, and subsequent retries cannot recover history preceding the window. The strategy stays flat until a later reconstructible anchor despite daily predictions being specified. (line 365) — fix: Supply warmup sufficient to reconstruct the preceding scheduled model, up to 513 prior sessions, or restore its model and recurrent state. Preserve the original retraining anchors.

Use an engine-supported next-session-open execution hook that executes the queued close decision unconditionally at the following session's open. Verify unchanged-open, gap-up, gap-down, and flat-session cases. If that hook is unavailable, report the execution contract as unsupported rather than treating the OCO approximation as faithful.

The OCO trigger pair does not guarantee the required next-session-open execution. With previous close 100.00 and tick 0.01, triggers are 100.01 and 99.99. If the next session opens at 100.00, neither order triggers at the open. A later intraday crossing executes the rebalance at a different price; a session remaining between the triggers leaves it unfilled until cancellation. This changes the hypothesis's execution contract. The no-op flush callback does not resolve this gap. The hypothesis declares only the numpy implementation deviation, not conditional intraday execution.

Supply warmup sufficient to reconstruct the preceding scheduled model, up to 513 prior sessions, or restore its model and recurrent state. Preserve the original retraining anchors.

The fixed evaluation warmup of 160 bars cannot reconstruct every scheduled model. A window beginning late in a 360-session bucket needs the scheduled anchor plus its preceding 154 prediction-history bars, potentially 513 prior sessions. If those bars were not supplied, _retrain returns false, and subsequent retries cannot recover history preceding the window. The strategy stays flat until a later reconstructible anchor despite daily predictions being specified.

Outcome Summary

A faithful evaluation requires engine support for next-session-open execution before this strategy can proceed.

After five iterations, verification failed because the platform lacked a supported hook for unconditional execution at the following session open. The OCO approximation was disabled because it missed unchanged opens and flat sessions.

Port Quantiacs’ two-layer LSTM strategy to AAPL daily bars, predicting next-open direction and using the clipped raw model output as a target portfolio weight.

The Layer 3 sandbox backtest processed 0 bars; no return, Sharpe, or trade metrics were reported. Optimization and later review stages were not reached.

Analysis

strategy-ml_lstm_multiple_features

Backtest Review

Sharpe: 0.51
Total return: 599.20%
Max drawdown: 36.15%
Win rate: 0.0%
Profit factor: 0.00

Backtest and paper results are hypothetical. Trading involves risk of loss.