Audit Rules and Evidence for Backtest Engine Parity
Code Machine Learning for Trading
Summary
This audit record defines how several backtesting frameworks are compared with ML4T across real strategy case studies and a synthetic stress workload. It specifies the comparison rules for ordered fills, timestamps, account-money values, quantities, and other numeric fields. It also records framework editions, immutable identifiers, support status, performance policies, and references to correctness, timing, and scale evidence. Runtime measurement is restricted to the engine call, with input loading, inference, preparation, extraction, and reporting excluded.
The records illustrate why parity claims must be scoped to each asset model and implementation. For example, VectorBT OSS is marked unsupported for the CME futures workload because it lacks native contract multipliers and margin accounting, while both VectorBT editions are unsupported for crypto perpetual funding and margin accounting. The audit’s timings are based on isolated processes and correctness-passing pairs, so they do not establish universal performance across datasets or machines. This file documents evidence and policy rather than teaching a trading signal or allocation strategy.
Key ideas
- Backtest parity depends on explicit rules for fill ordering, timestamps, and numerical tolerances.
- Runtime evidence covers engine calls and excludes data and adapter preparation as well as result handling.
- Framework support must be assessed against the asset’s accounting requirements.
- The audit limits published timing comparisons to framework and case-study pairs that pass correctness checks.
- Results from the recorded workloads do not establish a universal speed ranking.
Tags
Full text
# framework_parity_audit.json
```json
{
"audit_generated_at": "2026-09-03T11:44:14.344493+00:00",
"comparison_policy": {
"account_money_fields": [
"cash",
"commission",
"equity",
"final_value"
],
"account_money_quantum": "0.01",
"fill_order": "canonical timestamp, asset, side, quantity, price, commission",
"meaning": "account-money gaps round to zero cents; quantity gaps round to zero at 1e-5 shares or contracts; all other numeric gaps round to zero at 1e-8",
"quantity_quantum": "0.00001",
"record_numeric_quantum": "0.00000001",
"rounding": "ROUND_HALF_EVEN",
"timestamp_domain": {
"cme_futures": "session date",
"crypto_perps_funding": "exact UTC event timestamp",
"etfs": "session date",
"fx_pairs": "session date",
"us_equities_panel": "session date"
}
},
"engine_commit": "7034236519cc0a99df6ef34a21d07ec2a81fc87c",
"engine_source_sha256": "8011ecdaf90807b4fe4fb5ada518a237311774b32891179bf1fad0a0baad8a58",
"evidence": {
"correctness": "https://github.com/ml4t/backtest/blob/0c3de46b1881ce6cb9fd72b672a1dba38ebc383b/validation/REAL_STRATEGY_RESULTS.json",
"performance": "https://github.com/ml4t/backtest/blob/0c3de46b1881ce6cb9fd72b672a1dba38ebc383b/validation/REAL_STRATEGY_PERFORMANCE.json",
"source_artifacts": [
{
"path": "validation/REAL_STRATEGY_RESULTS.json",
"sha256": "d7c6460fcb5ebf9ecf74c31d06e227996c66f488de46a5beaa2145104209916b"
},
{
"path": "validation/REAL_STRATEGY_PERFORMANCE.json",
"sha256": "7276dfba4356c4ce2e6e45c256e4431a4e01cd47ca5710e9d341893286929c40"
},
{
"path": "validation/LARGE_SCALE_RESULTS.json",
"sha256": "fb3a06a7c485bfb131dfef286ff743af18fa4ba673f5f1075245d7a9123cb3f5"
}
],
"synthetic_stress": "https://github.com/ml4t/backtest/blob/0c3de46b1881ce6cb9fd72b672a1dba38ebc383b/validation/LARGE_SCALE_RESULTS.json"
},
"frameworks": {
"backtrader": {
"access": "public",
"artifact": "backtrader-1.9.78.123-py2.py3-none-any.whl",
"display_name": "Backtrader",
"environment": ".venv-backtrader",
"immutable_id": "sha256:9a07a516b0de9155539a35c56e9404d8711dd7020b3d37b30495e83e1b9d5dfd",
"license": "GPL-3.0-or-later",
"package": "backtrader",
"profile": "backtrader_strict",
"python": "3.12 audit interpreter; upstream package metadata does not declare a range",
"python_env_var": "ML4T_BACKTRADER_PYTHON",
"required_scenarios": 17,
"scenario_matrix": true,
"source": "https://pypi.org/project/backtrader/1.9.78.123/",
"unsupported_scenarios": 0,
"version": "1.9.78.123"
},
"lean": {
"access": "public",
"artifact": "quantconnect/lean@sha256:ecd62b0e418d40d1d7c0cd95e90a94e397642a21d2c8810614830c8a4e9a8f70",
"cli_immutable_id": "sha256:eaa4c08f16295b76f005e429d9ca0d0453784dc1a40c1f5cbe5e50c02a05bd7c",
"cli_version": "1.0.228",
"display_name": "LEAN",
"environment": ".venv-lean",
"immutable_id": "sha256:ecd62b0e418d40d1d7c0cd95e90a94e397642a21d2c8810614830c8a4e9a8f70",
"license": "Apache-2.0",
"package": "lean",
"platform_artifact": "linux/amd64@sha256:cbe3f26b3f16c57be836b2cf913253d434f58e010c84ed038360d26b9df88307",
"profile": "lean",
"python": "3.11",
"python_env_var": "ML4T_LEAN_PYTHON",
"required_scenarios": 0,
"scenario_matrix": false,
"source": "https://github.com/QuantConnect/Lean",
"source_commit": "278fcb3d1b815b63ccadba68d7ae54422e34b792",
"unsupported_scenarios": 0,
"version": "18001"
},
"vectorbt_oss": {
"access": "public",
"artifact": "vectorbt-1.1.0.tar.gz",
"display_name": "VectorBT OSS",
"environment": ".venv-vectorbt-oss",
"immutable_id": "sha256:67a3b41466234485af70c18d201da105f1ebb2c1d1fac079db20059e45ddc73b",
"license": "Apache-2.0 WITH Commons-Clause",
"package": "vectorbt",
"profile": "vectorbt_oss_strict",
"python": ">=3.11,<3.15",
"python_env_var": "ML4T_VECTORBT_OSS_PYTHON",
"required_scenarios": 16,
"scenario_matrix": true,
"source": "https://pypi.org/project/vectorbt/1.1.0/",
"source_commit": "259d2d89fe2e7638baf3ca76c394937cd32b656d",
"unsupported_scenarios": 1,
"version": "1.1.0"
},
"vectorbt_pro": {
"access": "licensed",
"display_name": "VectorBT Pro",
"environment": ".venv-vectorbt-pro",
"immutable_id": "git:6e18cf0aa37849cfc20848f40f1d26ecfdc771b4",
"license": "Proprietary",
"package": "vectorbtpro",
"profile": "vectorbt_strict",
"python": ">=3.11",
"python_env_var": "ML4T_VECTORBT_PRO_PYTHON",
"required_scenarios": 17,
"scenario_matrix": true,
"source": "https://github.com/polakowo/vectorbt.pro",
"source_commit": "6e18cf0aa37849cfc20848f40f1d26ecfdc771b4",
"unsupported_scenarios": 0,
"version": "2026.6.27"
},
"zipline": {
"access": "public",
"artifact": "zipline_reloaded-3.1.1.tar.gz",
"display_name": "Zipline Reloaded",
"environment": ".venv-zipline",
"immutable_id": "sha256:4a305524616f7aad836f929e5a2ba5afc7db0e238757f47eb49487d9e2457a6f",
"license": "Apache-2.0",
"package": "zipline-reloaded",
"profile": "zipline_strict",
"python": ">=3.10",
"python_env_var": "ML4T_ZIPLINE_PYTHON",
"required_scenarios": 16,
"scenario_matrix": true,
"source": "https://pypi.org/project/zipline-reloaded/3.1.1/",
"source_commit": "09885a2ebc7567d40942c891b3879dc03c745070",
"unsupported_scenarios": 1,
"version": "3.1.1"
}
},
"library_commit": "0c3de46b1881ce6cb9fd72b672a1dba38ebc383b",
"performance_policy": {
"boundary": "engine call only",
"excluded": [
"input loading",
"model inference",
"target construction",
"adapter preparation",
"result extraction",
"serialization",
"reporting"
],
"interval": "10,000-draw percentile bootstrap of the sample median",
"measured_processes": 10,
"process_isolation": true,
"publication_scope": "only correctness-passing case-study/framework pairs",
"warmup_processes": 1
},
"performance_records": [
{
"case_study": "etfs",
"framework": "vectorbt_pro",
"framework_ci95_seconds": [
0.28554552848800085,
0.2885417409997899
],
"framework_median_seconds": 0.2872375610168092,
"framework_to_ml4t_ratio": 0.6924827859284183,
"ml4t_ci95_seconds": [
0.4141692244884325,
0.41945260000647977
],
"ml4t_median_seconds": 0.4147937925008591
},
{
"case_study": "etfs",
"framework": "vectorbt_oss",
"framework_ci95_seconds": [
0.17068579950137064,
0.17357376401196234
],
"framework_median_seconds": 0.17132284450053703,
"framework_to_ml4t_ratio": 0.4124410450618539,
"ml4t_ci95_seconds": [
0.41297478799242526,
0.41699278698069975
],
"ml4t_median_seconds": 0.4153874754993012
},
{
"case_study": "etfs",
"framework": "backtrader",
"framework_ci95_seconds": [
9.30285160850326,
9.423639220010955
],
"framework_median_seconds": 9.355300096503925,
"framework_to_ml4t_ratio": 21.7615755934312,
"ml4t_ci95_seconds": [
0.42605314050160814,
0.431600633499329
],
"ml4t_median_seconds": 0.4298999425082002
},
{
"case_study": "etfs",
"framework": "zipline",
"framework_ci95_seconds": [
3.8670725979900453,
3.9098115700180642
],
"framework_median_seconds": 3.8715068249875912,
"framework_to_ml4t_ratio": 6.126579592244658,
"ml4t_ci95_seconds": [
0.6284003740001936,
0.6377887904964155
],
"ml4t_median_seconds": 0.6319197794946376
},
{
"case_study": "etfs",
"framework": "lean",
"framework_ci95_seconds": [
2.4874195760057773,
2.599079799008905
],
"framework_median_seconds": 2.564114402513951,
"framework_to_ml4t_ratio": 3.418485618498456,
"ml4t_ci95_seconds": [
0.7486901020165533,
0.7566314754949417
],
"ml4t_median_seconds": 0.750073186980444
},
{
"case_study": "cme_futures",
"framework": "vectorbt_pro",
"framework_ci95_seconds": [
0.28389174649782944,
0.2887275889952434
],
"framework_median_seconds": 0.28636373148765415,
"framework_to_ml4t_ratio": 0.6631588833815962,
"ml4t_ci95_seconds": [
0.428688713509473,
0.43454156002553646
],
"ml4t_median_seconds": 0.43181768149952404
},
{
"case_study": "cme_futures",
"framework": "backtrader",
"framework_ci95_seconds": [
2.487967457011109,
2.526018163494882
],
"framework_median_seconds": 2.494809443000122,
"framework_to_ml4t_ratio": 5.811638815571466,
"ml4t_ci95_seconds": [
0.4273611745011294,
0.4325851899920963
],
"ml4t_median_seconds": 0.4292781300027855
},
{
"case_study": "crypto_perps_funding",
"framework": "lean",
"framework_ci95_seconds": [
2.7094629269995494,
2.8912153329874855
],
"framework_median_seconds": 2.801038935489487,
"framework_to_ml4t_ratio": 4.266156698399411,
"ml4t_ci95_seconds": [
0.6548837659938727,
0.6614908079791348
],
"ml4t_median_seconds": 0.6565719764912501
},
{
"case_study": "fx_pairs",
"framework": "vectorbt_pro",
"framework_ci95_seconds": [
0.2809414380026283,
0.28584351851895917
],
"framework_median_seconds": 0.28271306499664206,
"framework_to_ml4t_ratio": 1.9393634353319569,
"ml4t_ci95_seconds": [
0.14505504499538802,
0.14667728898348287
],
"ml4t_median_seconds": 0.1457762170030037
},
{
"case_study": "fx_pairs",
"framework": "vectorbt_oss",
"framework_ci95_seconds": [
0.13754021801287308,
0.1413268009928288
],
"framework_median_seconds": 0.1387957935075974,
"framework_to_ml4t_ratio": 0.9517668155264412,
"ml4t_ci95_seconds": [
0.1447887455142336,
0.14715092899859883
],
"ml4t_median_seconds": 0.14582962049462367
},
{
"case_study": "fx_pairs",
"framework": "backtrader",
"framework_ci95_seconds": [
0.4199030729942024,
0.43193558798520826
],
"framework_median_seconds": 0.42490502599684987,
"framework_to_ml4t_ratio": 2.919662876838548,
"ml4t_ci95_seconds": [
0.14518085900635924,
0.14739416650263593
],
"ml4t_median_seconds": 0.1455322220135713
},
{
"case_study": "fx_pairs",
"framework": "lean",
"framework_ci95_seconds": [
0.8773026489943732,
0.9533085440052673
],
"framework_median_seconds": 0.9130055215064203,
"framework_to_ml4t_ratio": 5.951507252400165,
"ml4t_ci95_seconds": [
0.1521106895088451,
0.15719534500385635
],
"ml4t_median_seconds": 0.1534074450028129
},
{
"case_study": "us_equities_panel",
"framework": "vectorbt_pro",
"framework_ci95_seconds": [
0.7874248115113005,
0.8377640749968123
],
"framework_median_seconds": 0.7949095129879424,
"framework_to_ml4t_ratio": 0.03867173809347579,
"ml4t_ci95_seconds": [
20.48596853449999,
20.599761563003995
],
"ml4t_median_seconds": 20.555308661496383
},
{
"case_study": "us_equities_panel",
"framework": "vectorbt_oss",
"framework_ci95_seconds": [
16.959977449994767,
17.023569879005663
],
"framework_median_seconds": 16.9949149099848,
"framework_to_ml4t_ratio": 0.8292416699730313,
"ml4t_ci95_seconds": [
20.453040241001872,
20.533734439013642
],
"ml4t_median_seconds": 20.49452593299793
},
{
"case_study": "us_equities_panel",
"framework": "backtrader",
"framework_ci95_seconds": [
494.2269169654901,
510.27303914300865
],
"framework_median_seconds": 495.55252874150756,
"framework_to_ml4t_ratio": 23.08159898786381,
"ml4t_ci95_seconds": [
21.38583242200548,
21.54328417198849
],
"ml4t_median_seconds": 21.46959268298815
},
{
"case_study": "us_equities_panel",
"framework": "zipline",
"framework_ci95_seconds": [
106.83352753649524,
109.14979357349512
],
"framework_median_seconds": 107.60890489349549,
"framework_to_ml4t_ratio": 4.862239476470224,
"ml4t_ci95_seconds": [
22.08169161299884,
22.23382121999748
],
"ml4t_median_seconds": 22.131551811515237
},
{
"case_study": "us_equities_panel",
"framework": "lean",
"framework_ci95_seconds": [
47.33300897400477,
48.01486862101592
],
"framework_median_seconds": 47.68267791901599,
"framework_to_ml4t_ratio": 1.8083761067475683,
"ml4t_ci95_seconds": [
26.22179457700986,
26.472655913996277
],
"ml4t_median_seconds": 26.36767746548867
}
],
"protocol_scope": {
"disabled_on_both_sides": [
"transaction costs",
"position rules"
],
"shared_inputs": "frozen model-derived targets and historical market data",
"tested": [
"target sizing",
"order sequencing",
"fills",
"cash and margin",
"funding where applicable",
"valuation"
]
},
"real_strategy_records": [
{
"case_study": "etfs",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000000",
"fill_gap": "0.00000000",
"fills": 2466,
"framework": "vectorbt_pro",
"input_bundle_sha256": "01f38079ce47821a5379d3769e86f4a2170b88033108153bfc0f928698e946db",
"negative_control_detected": true,
"profile": "vectorbt_strict",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000000",
"valuation_timestamps_match": true,
"valuations": 1995
},
{
"case_study": "etfs",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000000",
"fill_gap": "0.00000000",
"fills": 2466,
"framework": "vectorbt_oss",
"input_bundle_sha256": "01f38079ce47821a5379d3769e86f4a2170b88033108153bfc0f928698e946db",
"negative_control_detected": true,
"profile": "vectorbt_oss_strict",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000000",
"valuation_timestamps_match": true,
"valuations": 1995
},
{
"case_study": "etfs",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000000",
"fill_gap": "0.00000000",
"fills": 2433,
"framework": "backtrader",
"input_bundle_sha256": "01f38079ce47821a5379d3769e86f4a2170b88033108153bfc0f928698e946db",
"negative_control_detected": true,
"profile": "backtrader_strict",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000000",
"valuation_timestamps_match": true,
"valuations": 1995
},
{
"case_study": "etfs",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000000",
"fill_gap": "0.00000000",
"fills": 2458,
"framework": "zipline",
"input_bundle_sha256": "01f38079ce47821a5379d3769e86f4a2170b88033108153bfc0f928698e946db",
"negative_control_detected": true,
"profile": "zipline_strict",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000000",
"valuation_timestamps_match": true,
"valuations": 1995
},
{
"case_study": "etfs",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000000",
"fill_gap": "0.00000000",
"fills": 2457,
"framework": "lean",
"input_bundle_sha256": "01f38079ce47821a5379d3769e86f4a2170b88033108153bfc0f928698e946db",
"negative_control_detected": true,
"profile": "lean",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000000",
"valuation_timestamps_match": true,
"valuations": 1995
},
{
"case_study": "cme_futures",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000010",
"fill_gap": "0.00000000",
"fills": 3545,
"framework": "vectorbt_pro",
"input_bundle_sha256": "c7191027f550ef1be0c9528dac08e7c2bf6a76c86d935f593efeb1ad8b628c39",
"negative_control_detected": true,
"profile": "vectorbt_futures_strict",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000007",
"valuation_timestamps_match": true,
"valuations": 1595
},
{
"case_study": "cme_futures",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000015",
"fill_gap": "0.00000000",
"fills": 3376,
"framework": "backtrader",
"input_bundle_sha256": "c7191027f550ef1be0c9528dac08e7c2bf6a76c86d935f593efeb1ad8b628c39",
"negative_control_detected": true,
"profile": "backtrader_strict",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000015",
"valuation_timestamps_match": true,
"valuations": 1595
},
{
"case_study": "crypto_perps_funding",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000000",
"fill_gap": "0.00000000",
"fills": 8408,
"framework": "lean",
"input_bundle_sha256": "2acee3c8542043266e6f9c0dfc9434c95b4469ef6fcaf0556efc569fc9721cdd",
"negative_control_detected": true,
"profile": "lean_crypto_future",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000000",
"valuation_timestamps_match": true,
"valuations": 2426
},
{
"case_study": "fx_pairs",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000000",
"fill_gap": "0.00000000",
"fills": 279,
"framework": "vectorbt_pro",
"input_bundle_sha256": "48922296cea03bfc9867a9c4e1a1d352a0bd54c0b48671e3ca8d1fdddb95d762",
"negative_control_detected": true,
"profile": "vectorbt_strict",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000000",
"valuation_timestamps_match": true,
"valuations": 2108
},
{
"case_study": "fx_pairs",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000000",
"fill_gap": "0.00000000",
"fills": 279,
"framework": "vectorbt_oss",
"input_bundle_sha256": "48922296cea03bfc9867a9c4e1a1d352a0bd54c0b48671e3ca8d1fdddb95d762",
"negative_control_detected": true,
"profile": "vectorbt_oss_strict",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000000",
"valuation_timestamps_match": true,
"valuations": 2108
},
{
"case_study": "fx_pairs",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000000",
"fill_gap": "0.00000000",
"fills": 279,
"framework": "backtrader",
"input_bundle_sha256": "48922296cea03bfc9867a9c4e1a1d352a0bd54c0b48671e3ca8d1fdddb95d762",
"negative_control_detected": true,
"profile": "backtrader_strict",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000000",
"valuation_timestamps_match": true,
"valuations": 2108
},
{
"case_study": "fx_pairs",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000000",
"fill_gap": "0.00000000",
"fills": 279,
"framework": "lean",
"input_bundle_sha256": "48922296cea03bfc9867a9c4e1a1d352a0bd54c0b48671e3ca8d1fdddb95d762",
"negative_control_detected": true,
"profile": "lean",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000000",
"valuation_timestamps_match": true,
"valuations": 2108
},
{
"case_study": "us_equities_panel",
"equity_gap": "0.00",
"equity_raw_gap": "0.00001950",
"fill_gap": "0.00000000",
"fills": 54295,
"framework": "vectorbt_pro",
"input_bundle_sha256": "02926d8a7f1c1f5b4855b2393d1763885288ef64950d5cf521ca96c61f454d88",
"negative_control_detected": true,
"profile": "vectorbt_strict",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00001880",
"valuation_timestamps_match": true,
"valuations": 4146
},
{
"case_study": "us_equities_panel",
"equity_gap": "0.00",
"equity_raw_gap": "0.00001910",
"fill_gap": "0.00000000",
"fills": 49120,
"framework": "vectorbt_oss",
"input_bundle_sha256": "02926d8a7f1c1f5b4855b2393d1763885288ef64950d5cf521ca96c61f454d88",
"negative_control_detected": true,
"profile": "vectorbt_oss_strict",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00001870",
"valuation_timestamps_match": true,
"valuations": 4146
},
{
"case_study": "us_equities_panel",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000170",
"fill_gap": "0.00000000",
"fills": 54242,
"framework": "backtrader",
"input_bundle_sha256": "02926d8a7f1c1f5b4855b2393d1763885288ef64950d5cf521ca96c61f454d88",
"negative_control_detected": true,
"profile": "backtrader_strict",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000160",
"valuation_timestamps_match": true,
"valuations": 4146
},
{
"case_study": "us_equities_panel",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000190",
"fill_gap": "0.00000000",
"fills": 54259,
"framework": "zipline",
"input_bundle_sha256": "02926d8a7f1c1f5b4855b2393d1763885288ef64950d5cf521ca96c61f454d88",
"negative_control_detected": true,
"profile": "zipline_strict",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000030",
"valuation_timestamps_match": true,
"valuations": 4027
},
{
"case_study": "us_equities_panel",
"equity_gap": "0.00",
"equity_raw_gap": "0.00000460",
"fill_gap": "0.00000000",
"fills": 54265,
"framework": "lean",
"input_bundle_sha256": "02926d8a7f1c1f5b4855b2393d1763885288ef64950d5cf521ca96c61f454d88",
"negative_control_detected": true,
"profile": "lean",
"status": "pass",
"terminal_gap": "0.00",
"terminal_raw_gap": "0.00000420",
"valuation_timestamps_match": true,
"valuations": 4027
}
],
"schema_version": 2,
"scope": {
"case_protocols": {
"fx_pairs": "USD-quoted pairs from the frozen target stream, preserving native USD valuation in every required engine"
},
"case_studies": [
"etfs",
"cme_futures",
"crypto_perps_funding",
"fx_pairs",
"us_equities_panel"
],
"frameworks": [
"vectorbt_pro",
"vectorbt_oss",
"backtrader",
"zipline",
"lean"
],
"real_strategy_equivalence_gate_passed": true,
"required_pairs": 17,
"unsupported_pairs": 8
},
"synthetic_stress": {
"assets": 250,
"bars": 1260000,
"comparison_policy": {
"account_money_quantum": "0.01",
"record_numeric_quantum": "1E-8"
},
"generated_at": "2026-09-03T08:44:03.035191+00:00",
"records": [
{
"fills": 423313,
"framework": "vectorbt_pro",
"intents": 427790,
"status": "pass",
"terminal_value": 1285886.32,
"trades": 222751
},
{
"fills": 417941,
"framework": "vectorbt_oss",
"intents": 427790,
"status": "pass",
"terminal_value": 1345348.85,
"trades": 211322
},
{
"fills": 343813,
"framework": "backtrader",
"intents": 427790,
"status": "pass",
"terminal_value": -9166273.56,
"trades": 182019
},
{
"fills": 427696,
"framework": "zipline",
"intents": 427790,
"status": "pass",
"terminal_value": 10504095.9,
"trades": 226434
},
{
"fills": 361297,
"framework": "lean",
"intents": 427790,
"status": "pass",
"terminal_value": 184538.13,
"trades": 191297
}
],
"sessions": 5040,
"synthetic": true
},
"unsupported_records": [
{
"case_study": "cme_futures",
"framework": "vectorbt_oss",
"reason": "no native futures contract multiplier and margin-account model",
"source": "https://vectorbt.dev/api/portfolio/base/",
"status": "unsupported"
},
{
"case_study": "cme_futures",
"framework": "zipline",
"reason": "the frozen workload has continuous root series but no dated contract chain or roll map",
"source": "https://zipline.ml4trading.io/api-reference.html",
"status": "unsupported"
},
{
"case_study": "cme_futures",
"framework": "lean",
"reason": "the frozen workload has continuous root series but no dated contract chain or roll map",
"source": "https://www.quantconnect.com/docs/v2/writing-algorithms/securities/asset-classes/futures",
"status": "unsupported"
},
{
"case_study": "crypto_perps_funding",
"framework": "vectorbt_pro",
"reason": "no native perpetual-futures funding and margin-account model",
"source": "https://vectorbt.pro/features/portfolio/",
"status": "unsupported"
},
{
"case_study": "crypto_perps_funding",
"framework": "vectorbt_oss",
"reason": "no native perpetual-futures funding and margin-account model",
"source": "https://vectorbt.dev/api/portfolio/base/",
"status": "unsupported"
},
{
"case_study": "crypto_perps_funding",
"framework": "backtrader",
"reason": "no native perpetual-futures funding model",
"source": "https://www.backtrader.com/docu/commission-schemes/commission-schemes/",
"status": "unsupported"
},
{
"case_study": "crypto_perps_funding",
"framework": "zipline",
"reason": "no crypto perpetual-futures asset and funding model",
"source": "https://zipline.ml4trading.io/api-reference.html",
"status": "unsupported"
},
{
"case_study": "fx_pairs",
"framework": "zipline",
"reason": "no native spot-forex asset, currency cashbook, or conversion model",
"source": "https://zipline.ml4trading.io/api-reference.html",
"status": "unsupported"
}
]
}
```Shown in full with attribution under the source's licence. Licence: MIT
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.