Live Trading Reconciliation of Orders, Fills, and Positions
Article NautilusTrader
Summary
This guide explains how a live trading system rebuilds and checks its internal order and position state against venue reports. At startup, it uses cached state when available, or creates state from venue orders and positions. During runtime, recurring checks identify discrepancies. Explicit open or flat position reports define the quantity target; a missing report does not imply a flat account. Reconciliation can generate synthetic orders and fills to align state, subject to tolerances and configured behavior.
The guide describes how order status, standalone fill, order-with-fills, and position reports affect recovery, and why snapshot timing matters when assessing possible fill corrections. It also covers execution-client attribution, incomplete venue history, delayed or duplicate reports, split strategy ownership, and instrument or precision mismatches. Its invariants include deterministic synthetic event identifiers and order-state recovery even when fills cannot be applied to position economics. These are operational rules for maintaining consistency, not evidence of trading profitability; results depend on available reports, retained events, filters, and venue history.
Key ideas
- An explicit venue position report is the target for reconciliation, while a missing report cannot establish that the account is flat.
- Cached execution events improve recovery when venue history is limited.
- Order and fill report types differ in how they rebuild missing local state.
- A late-arriving snapshot may describe an earlier state and should not automatically void a later fill.
- Synthetic reconciliation can align quantity but does not recreate missing historical profit and loss.
Tags
Full text
# Execution Reconciliation
# Execution Reconciliation
Execution reconciliation aligns the venue's actual order and position state with the system's
internal event-sourced state. Use this guide to understand startup state recovery and the
continuous checks that detect runtime discrepancies.
Unresolved live command outcomes are one source of state divergence. For how Nautilus classifies
local failures, definitive results, and unknown outcomes, see
[Command outcomes](policies.md#command-outcomes).
For the complete node lifecycle, see [Live trading](../live.md). For the available settings and
recommended values, see
[Configure a live trading node](../../how_to/configure_live_trading.md#executionengine-configuration).
## Reconciliation model
Live execution reconciles local state against venue reports. Backtesting controls both order
execution and the resulting state, so it does not need venue reconciliation.
Two scenarios:
- **Cached state exists**: report data generates missing events to align the state.
- **No cached state**: all orders and positions at the venue are generated from scratch.
:::info[Position reports are market exposure]
**An explicit position report is authoritative. During startup reconciliation,
the engine either aligns to that report within reconciliation tolerances or fails closed.**
Authoritative position reports:
- An explicit open report, including quantity and direction.
- An explicit flat report.
Not evidence of a flat position:
- A missing report.
- A null quantity.
- A venue that does not publish positions.
The fill window does not decide whether the report is authoritative. Missing reports do
not mean flat.
By default, `generate_missing_orders` is enabled. The engine generates the
orders and fills needed to align local state to the report. Disabling generation
does not allow an unresolved report through startup.
This guarantee covers reports included by the position-report and instrument
filters, with reconciliation enabled. Unresolved reports prevent actors and
strategies from starting.
:::
:::tip
Persist all execution events to the cache database. This reduces reliance on venue history
and gives reconciliation the retained order and position state needed to interpret short history
windows.
:::
### Component responsibilities
`LiveNode` owns the `ExecutionManager` and schedules recurring reconciliation. The manager tracks
activity, retries, and fill identities, interprets cached state, and prepares reconciliation events.
`ExecutionEngine` applies events to orders and positions and handles individual execution reports.
The UML diagram shows ownership and dependencies. A filled diamond denotes ownership; dashed arrows
point from a caller to a component it uses. The kernel owns the engine and shared cache; it is omitted
here to focus on reconciliation.
```mermaid
classDiagram
direction LR
namespace nautilus_live {
class LiveNode
class ExecutionManager
}
namespace nautilus_execution {
class ExecutionEngine
}
namespace nautilus_common {
class ExecutionClient {
<<interface>>
}
class Cache
}
LiveNode *-- ExecutionManager : owns
LiveNode ..> ExecutionClient : requests recurring reports
LiveNode ..> ExecutionEngine : dispatches through kernel
ExecutionManager ..> ExecutionClient : polls reports for standalone checks
ExecutionManager ..> ExecutionEngine : applies startup events
ExecutionManager ..> Cache : reads state and registers external orders
ExecutionEngine ..> ExecutionClient : routes commands and requests reports
ExecutionEngine ..> Cache : updates orders and positions
```
The live client facade shares one adapter instance between the node and engine. Pending report
requests can retain client borrows while the event loop handles other work. Instrument updates are
deferred until those borrows are released, then flushed on request completion or cancellation.
Within `nautilus-live`, the source modules divide these responsibilities as follows:
| Module | Responsibility |
| ----------------------------- | ------------------------------------------------------------------------ |
| `node/mod.rs` | Node lifecycle, event loop, and event dispatch. |
| `node/reconciliation.rs` | Recurring report tasks, deadlines, cancellation, and result handling. |
| `execution/manager.rs` | Reconciliation state, decisions, and individual reconciliation checks. |
| `execution/reconciliation.rs` | Shared types, state-independent decisions, and targeted report requests. |
The separate `nautilus_execution::reconciliation` module supplies report-to-event and arithmetic
operations shared with the execution engine.
At startup, the manager publishes raw reports, applies order and fill events, verifies historical
fill application, and then evaluates positions against the updated cache. During continuous
position checks, the node coordinates authoritative fill queries and dispatch before asking the
manager to generate synthetic events. Activity revisions detect local changes during requests or
callbacks; applying authoritative fills defers synthetic reconciliation until a fresh position report.
A fill that fails preparation or that the engine does not apply also defers synthetic
reconciliation, but only until `position_check_threshold_ms` has passed since its first failure.
Later checks skip that fill and reconcile the position synthetically. Each check stops at the first
such fill for an instrument and account, so each refused fill delays the fallback by at least one
check. If a skipped fill later applies through another path, such as the execution stream, it can
add to the synthetic correction until the next position check reconciles the difference.
The manager remains available without the `node` feature. Standalone callers can use its individual
polling methods and apply the returned events themselves. Standalone position polling directly
returns synthetic discrepancy events; the node adds the authoritative-fill recovery sequence.
### Execution-client origins
An **execution-client origin** is a write-once binding between an order and the client responsible
for its execution.
An origin is recorded:
- From an explicit client on submission, or from the final client selected after routing and venue
validation and before transport.
- When non-synthetic external orders are materialized during startup reconciliation, from the
reporting mass-status client.
- When external orders are materialized from runtime venue reports and the report's account
matches exactly one registered client that handles the instrument venue.
An origin may be absent for:
- Cache data written before resolved origins were persisted.
- External orders whose runtime report does not identify exactly one registered client by account
and instrument venue.
- Synthetic reconciliation orders.
The built-in cache backends enqueue a resolved origin for persistence before transport. Their
writes remain asynchronous, so enqueue order does not guarantee that the origin is durable before
the order reaches the client.
At startup, each client's mass status is checked against the cached origins: an order the client
reports is expected to be bound to that same client. A missing origin logs an aggregated warning
and remains compatible with existing cache data. A conflicting origin logs an aggregated
deprecation warning and reconciles for compatibility. A future release rejects the conflict as a
startup error. See the origin rows in [Startup scenarios](#startup-scenarios).
This is separate from [external order claims](#external-order-creation), which attribute
venue-sourced orders to a *strategy*. The execution-client origin records which *client* an order
belongs to.
### Reconciliation reports
The execution engine consumes four reconciliation report variants from live adapters. Each variant
has a different normal role when its matching order is absent from the cache. Explicitly bounded
history can instead use [order-only fill projection](#order-only-fill-projection).
| Variant | Purpose | Missing-order action |
| ---------------------- | ------------------------ | ---------------------------------------------------- |
| `OrderStatusReport` | Order state update. | Creates an order and infers any reported fill. |
| `FillReport` | Standalone fill. | Creates a market order, then applies fill metadata. |
| `OrderWithFills` | Order state plus fills. | Creates an order, applies fills, and infers residue. |
| `PositionStatusReport` | Venue position snapshot. | Logs the report; positions remain fill-derived. |
#### When to use each variant
Adapters choose the variant that matches the venue event:
- Use `OrderStatusReport` for order lifecycle updates when fill details arrive on a separate
stream.
- Use `FillReport` for a venue-initiated closure that has a fill but no user-level order.
Hyperliquid liquidations follow this pattern.
- Use `OrderWithFills` when one venue event contains both an order status and its fills. Binance
Futures uses this for exchange-generated ADL, liquidation, and settlement orders.
### Snapshot freshness and fill corrections
A snapshot must not undo a fill that occurs after the state it describes. Receiving a snapshot
after a stream event does not make the snapshot newer: REST requests and stream delivery can
overlap during recovery. For example, a request can observe zero filled quantity, a stream can
then deliver a fill of five units, and the older response can arrive last. Interpreting that
response as a correction would wrongly void the five units.
Snapshot corrections require a distinction between:
- **A snapshot that predates a fill**: its lower filled quantity does not establish that the fill
was voided.
- **A snapshot that covers the fill and reports a reduction**: the reduction can represent a
genuine correction, derived from retained fill history.
- **An explicit venue fill-void event**: process it under the
[OrderFillVoided contract](../events/order_fill_voided.md), including its identity, quantity,
and ordering checks. It does not depend on inferring a correction from a snapshot total.
Timestamp meaning matters when establishing coverage. The Derive adapter reports an order-update timestamp;
Betfair's `matchedDate` describes the last match, not the time of a snapshot or correction.
Response arrival time, equal timestamps, or timestamps from different clocks do not by themselves
prove that a snapshot includes a fill.
Rejecting a genuine correction as stale can leave local filled quantity and exposure overstated
until later reconciliation resolves the discrepancy. Conversely, a stale snapshot carrying a
misleadingly newer timestamp can still cause a false void if freshness checks trust that timestamp.
The execution engine applies mass-status filled-quantity decreases to retained fills even when the
snapshot contains no companion trades. It automatically skips an order snapshot when a cached fill
or fill void has a local initialization timestamp at or after collection starts
(`ExecutionMassStatus.ts_init`). This skips all changes from that order report, including status,
quantity, and price updates.
The engine still publishes the raw report and processes companion trades through normal
deduplication. It does not queue the skipped snapshot. A later snapshot can apply a genuine
correction once collection starts after the cached fill activity. The skip requires no
configuration and does not suppress explicit fill-void events.
This protection applies to runtime mass-status handling in `ExecutionEngine`. Startup reconciliation
uses `ExecutionManager`, which does not apply this timestamp boundary. For runtime protection,
adapters must capture the mass-status timestamp before collecting reports, using the same local
clock as fill events. This boundary protects against overlapping local activity; it cannot detect
venue state that is already stale when collection starts.
### Order-only fill projection
During startup reconciliation, a bounded historical fill does not change a position or the portfolio
when that instrument has no in-scope explicit position report. Omission is not a flat report. The order still
reaches the reported status and filled quantity. An explicit position report, open or flat, is the
quantity target: the engine applies the available fills, generates the difference when configured,
or leaves the position unresolved.
This projection applies only to reconciliation recovery. Raw reports remain available. Setting
`filter_position_reports` makes bounded historical fills order-only, even when the mass status
contains position reports. See [Bounded history safety](#bounded-history-safety) for the
undeclared-window exception and retained-fill rules.
### External order creation
When a report references an order that is absent from the cache, the engine creates an **external
order**. This covers venue-initiated ADL, liquidation, or settlement, orders placed by another
process, and orders not yet observed locally.
The naming distinguishes configuration intent from live ownership state:
- `external_order_instrument_ids` is the serializable strategy configuration intent. It names the
instruments whose external orders should be assigned to the strategy when it is registered.
- An **external order claim** is an active cache entry that maps one `InstrumentId` to one `StrategyId`.
The code uses `external_order_claims` for the collection of these live entries.
Live strategy registration materializes the configured instrument IDs with
`register_external_order_claims`. This operation is additive and strict: it rejects a repeated
instrument or any instrument that already has a claim, including a claim for the same strategy.
The strategy method `set_external_order_instrument_ids(...)` delegates to the cache operation
`set_external_order_claims`. This operation treats its input as the strategy's complete desired
active set. It can retain or release that strategy's existing claims and acquire unclaimed
instruments, but it cannot take a claim from another strategy. Validation covers the complete input
before changing the cache, so a conflict leaves every existing claim unchanged.
The `ExecutionManager` and `ExecutionEngine` read the same canonical claim map from the cache when
they process external reports. They assign an external order to:
- The strategy identified by the active claim for the report's instrument.
- The `EXTERNAL` strategy as a default fallback.
An active-claim update is therefore visible to both components without a coordination message. The
claim present when an external order is created determines the assignment. Existing cached orders
keep their assigned `StrategyId`; changing a claim does not reassign them.
Transferring an instrument between strategies requires the current owner to release it before the
new owner claims it. There is no atomic handoff across strategies. A report processed between the
release and acquisition has no active claim and is assigned to `EXTERNAL`. Cache resets preserve
active claims so registered routing remains configured, while retiring a strategy clears its claims.
The external order uses the report's `client_order_id` when present and otherwise derives one from
the `venue_order_id`. The engine adds the order to the cache, registers its venue order ID, and
emits the applicable `OrderAccepted`, `OrderFilled`, `OrderCanceled`, or `OrderExpired` events.
Positions then update through the normal event pipeline.
See [Claiming external orders](../strategies.md#claiming-external-orders) for strategy configuration
and runtime updates.
### Reducing external positions
A strategy can use reduce-only fills to reduce inherited `EXTERNAL` inventory under NETTING.
#### Position selection
Existing cached position links remain authoritative. Without a cached link, a reduce-only fill
uses the strategy's own open position when available. If that position is absent or closed, the
engine looks for positions that meet all of these conditions:
- Belong to `EXTERNAL` and use NETTING.
- Are open on the opposite side of the fill.
- Match the fill's instrument and account.
The engine selects a fallback only when **exactly one** position matches. The fill quantity must
not exceed that position's quantity, though the order's remaining quantity can be larger.
If no safe fallback exists, an otherwise valid fill updates the order but neither opens nor
updates a position.
#### Ownership and events
After a successful reduction, the engine links the order to the external position so subsequent
fills use the same target. The position retains `EXTERNAL` ownership:
- `OrderFilled` keeps the reducing strategy's ID and identifies the external position.
- `PositionChanged` and `PositionClosed` use the `EXTERNAL` strategy's event topic.
#### Linked reduction checks
When applying position economics, each linked reduction must match the external position's account
and reduce its open quantity without flipping or reopening it. If a fill violates these checks,
the engine rejects it **before changing the order or position**.
[Order-only fill projection](#order-only-fill-projection) bypasses these reduction checks because
it repairs order history without changing the position.
## Reconciliation configuration
Unless `reconciliation` is set to false, the live node runs startup reconciliation for each
execution client. The `reconciliation_lookback_mins` parameter controls how far back it requests
history through the execution engine. Startup enablement and polling intervals belong to the node's
`LiveExecutionEngineConfig`; the manager receives the thresholds, retry limits, filters, and lookbacks
used to make reconciliation decisions.
:::tip
Leave `reconciliation_lookback_mins` unset to use the adapter's documented default. Many adapters
request the maximum execution history the venue provides, while others use a bounded default to
match venue retention and request limits. See the integration guide for the selected venue.
:::
:::warning
A bounded history window can begin after the fill that opened a position. Some venues also filter
or drop older execution data. That does not change the position-report rule above: an explicit
report is still the quantity target.
:::
For all live trading options, see the `LiveExecutionEngineConfig`
[API reference](/docs/python-api-latest/config.html#nautilus_trader.live.LiveExecutionEngineConfig).
### Order ownership and tags
Each strategy can configure `external_order_instrument_ids` as its intent to claim venue-sourced
external orders and materialized reconciliation activity for specific instruments. Live strategy
registration materializes that intent as active claims, which the strategy can replace at runtime.
This lets a strategy resume managing open orders and positions when no cached state exists.
An external order's strategy ID and origin tag depend on its source and on whether a strategy
claims its instrument. Claimed orders and fills use the claiming strategy ID, so the strategy can
continue managing the recovered state.
| Order source | Unclaimed | Claimed |
| ----------------------------- | -------------------------------- | --------------------------------------- |
| Venue order | `EXTERNAL`, tag `VENUE` | Claiming strategy, no tag |
| Position reconciliation order | `EXTERNAL`, tag `RECONCILIATION` | Claiming strategy, tag `RECONCILIATION` |
:::tip
To detect unclaimed external orders in your strategy, check `order.strategy_id.value == "EXTERNAL"`.
Ownership does not exclude these orders from position tracking or portfolio calculations. Historical
fills still follow the [bounded history safety](#bounded-history-safety) rules when applicable.
:::
### Submission recovery
`submission_recovery_policy` defaults to `ResolveLocally`. Selecting `RetainUnresolved` enables
submission identity tracking, exhaustion diagnostics, and preservation of the submission's recovery
budget until acknowledgement. Exhausted unacknowledged submissions remain unresolved, with further
automatic per-order recovery queries stopped. Clients can also require this protection independently
of the configured policy. Polymarket enables this protection automatically.
Retained submissions also affect the [node shutdown result](../live.md#submission-recovery-at-shutdown).
Rust `LiveNode::stop` and `run`, and hosted runs, report unresolved client order IDs after teardown
when submission recovery is incomplete at the `delay_post_stop` boundary. This includes Polymarket
without an explicit opt-in. Exhaustion diagnostics and successful teardown do not establish a venue
outcome or position flatness.
#### Exhaustion diagnostic
With tracking enabled, a native `LiveNode` publishes `SubmissionRecoveryExhausted` on
`reconciliation.SubmissionRecoveryExhausted` when an unacknowledged submission reaches an existing
recovery limit. The diagnostic carries the original submission identity, recovery source, check
count, and event timestamp. It is published after processing reconciliation events, once per
tracked submission. It is not an order event and does not establish a venue outcome.
#### Recovery budget
A cancel or modify dispatched before acknowledgement shares the submission's remaining recovery
budget. Duplicate registration and missing-order bookkeeping cannot restart that budget.
Acknowledgement retires it without transferring recovery to an overlapping command.
A command dispatched after acknowledgement starts its own in-flight check at dispatch, even if its
pending event has not arrived, and uses the existing command cleanup rules. For example, a
`PARTIALLY_FILLED` order status report stops a pending cancel's timeout under both policies. That
cleanup rule does not apply to standalone fill reports. Without submission tracking
(`ResolveLocally` and no client-required retention), command dispatch starts a fresh recovery
budget, and native fill or `Triggered` events do not retire the pending command's timeout.
#### Acknowledgement
A matching `Submitted` report with a venue order ID retires the original submission's recovery.
Incoming reports and bulk or targeted query responses use the same confirmation checks. An applied
`OrderUpdated` with venue identity also confirms the submission; local updates without venue
identity do not.
The registry remembers report-only confirmation, including for cached orders not yet registered
for recovery, so duplicate submission registration cannot restart the timeout. Exhausted submission
identity also remains in the registry until venue evidence resolves it; duplicate registration and
cancel or modify dispatch cannot restart its automatic recovery. Missing-order bookkeeping does not
register an acknowledged submission again. Later commands do not produce submission-exhaustion
diagnostics.
#### Registration
`LiveNode` registers submission identity before dispatch. Direct `ExecutionManager` callers use
`register_submission` for submissions; `register_inflight` only starts command recovery and does not
infer submission identity from the cache. `LiveNode` registers client-required retention
automatically. Direct manager hosts must select `RetainUnresolved` or run `check_open_orders` with
their clients before relying on client-required retention in inflight checks.
#### Recovery limits
The existing limits and coverage checks still apply. `inflight_check_retries` counts checks: a limit
of `N` permits `N - 1` intermediate order queries before exhaustion. Missing-order checks use
`open_check_missing_retries` and require completed, matching client coverage plus a successful
targeted query with no order found before reporting exhaustion. Failed or deferred targeted queries
do not produce that diagnostic.
### Instrument availability
Adapters parse reconciliation reports using the instrument, so every instrument a report references
must already be loaded. Adapters do not fetch missing instruments from the venue during
reconciliation.
Instrument scope comes from the adapter's provider config rather than the engine.
`InstrumentProviderConfig.load_ids` decides which instruments the adapter holds, while
`reconciliation_instrument_ids` filters reports only after the adapter has produced them.
Reports for instruments outside an explicit `load_ids` scope are expected: they are dropped at debug
level, so a node scoped to one instrument stays quiet about the rest of the venue. An in-scope
instrument that does not resolve means something is wrong, whether it was named in `load_ids` or
covered by `load_all=True`, and the outcome depends on what the report describes:
- An open order or position report fails reconciliation, so the system does not start. A live
position that cannot be priced is never silently dropped.
- A closed or historical record logs a warning instead of aborting startup. When the adapter
declares a bounded history, the record also marks the report set incomplete, applying the
[bounded history safety](#bounded-history-safety) rules. Expiries routinely retire instruments
that older fills still reference.
## Startup reconciliation
All adapter execution clients follow the same reconciliation procedure, calling three methods
to produce an execution mass status:
- `generate_order_status_reports`
- `generate_fill_reports`
- `generate_position_status_reports`
```mermaid
flowchart TD
Start[Startup Reconciliation] --> Fetch[Fetch venue reports<br/>orders, fills, positions]
Fetch --> Dedup[Deduplicate reports<br/>log warnings for duplicates]
Dedup --> Orders[Order Reconciliation<br/>align order states, generate missing events]
Orders --> Fills[Fill Reconciliation<br/>verify fills, generate missing OrderFilled events]
Fills --> Pos[Position Reconciliation<br/>compare net positions per instrument]
Pos --> Match{Positions<br/>match venue?}
Match -->|Yes| Done[Reconciliation complete<br/>system ready for trading]
Match -->|No| Gen[Generate missing orders<br/>strategy: EXTERNAL, tag: RECONCILIATION]
Gen --> Recovered{In-scope reports match<br/>within tolerances?}
Recovered -->|Yes| Done
Recovered -->|No| Abort[Startup fails<br/>actors and strategies do not start]
```
These reports represent external reality. The procedure processes them in the order shown so each
position check builds on reconciled order and fill state.
### Mass-status history contract
An `ExecutionMassStatus` can declare the provenance of its historical reports:
- `lookback_start=None` means that the adapter has not declared an explicit lower time bound.
- `lookback_start=Some(timestamp)` means that historical order and fill reports exclude venue
activity before that timestamp.
- `reports_complete=true` means that every order, fill, and position source needed to interpret
the bounded history completed and all required records were mapped successfully.
An adapter can still return authoritative active orders and position reports when a historical
source fails. It marks the mass status incomplete to record the missing history. Incompleteness
does not prevent recovery from an explicit position report.
### Report deduplication
- Deduplicates order reports within the batch and logs warnings.
- Logs duplicate trade IDs as warnings for investigation.
- Counts a position report once in startup validation and continuous position checks when it
matches an earlier report apart from `report_id` and `ts_init`, and logs a warning for each
duplicate. Reports that differ in any other field, including `ts_last`, count separately.
### Order reconciliation
- Generates and applies events to move orders from cached state to current state.
- Generates external order events for unrecognized client order IDs or reports missing a client
order ID.
### Fill reconciliation
- Infers `OrderFilled` events for missing trade reports.
- Verifies fill report data consistency with tolerance-based price and commission comparisons.
- Drops a fill group that has neither an order report nor a cached order, and logs one aggregated
warning per mass status for the dropped groups. A group whose fills carry a `venue_position_id`
is handled separately: the engine builds a filled order from it when `generate_missing_orders`
is enabled, and skips it otherwise.
### Event ordering
Startup reconciliation applies the order and fill events it generates in time order:
- Fills apply in order of their event time: the venue execution time from `FillReport.ts_event`,
or the report's `ts_last` for an inferred fill.
- Each venue order's other events, such as its acceptance, apply no later than the earliest fill
that follows them for that venue order. An order whose reported acceptance time follows its own
fills, because the venue reports a last update time or the adapter uses the local time, is still
accepted before its fills apply.
- Events of different venue orders keep their own times, including a replaced order that shares
its client order ID with the order that replaced it.
Ordering changes only when events apply. Every event keeps its reported timestamp.
### Position reconciliation
- Matches the net position per account and instrument against venue position reports using
the account's quantity tolerance.
- Generates external order events when order reconciliation leaves a position that differs from
the venue.
- When `generate_missing_orders` is enabled (default: True), generates orders with strategy ID
`EXTERNAL` and tag `RECONCILIATION` to align discrepancies.
- Logs a warning when NETTING ownership is split across multiple strategies for the same account
and instrument, since venue position reports are account-level net positions.
#### Synthetic orders
Reconciliation generates synthetic MARKET order reports and fills with a known price:
- Opening from flat uses the reported `avg_px_open`.
- Increasing an existing position uses a calculated price targeting the reported entry average.
- Reducing an existing position uses the reported entry average, falling back to the cached average
when the report omits it. A reduction does not change the remaining position's entry average.
- Closing to flat uses the cached entry average.
- Reversing direction closes the cached position, then opens the reported position at its entry average.
A report that marks its average changes some of these prices; see
[reported entry averages](#reported-entry-averages).
The engine skips quantity differences that round to zero at instrument size precision. Startup
validation still checks the remaining difference against the account's quantity tolerance.
#### Replayed fills
While a position that reconciliation opened from a venue position report stays open, a fill report
for that instrument and account with an earlier `ts_event` updates its order but no position. The
report already includes those executions, so this stops a venue that resends them after a restart,
such as Interactive Brokers TWS, from doubling the position. Earlier fills still apply when no such
position is open for the instrument and account.
The protection has these known gaps:
- **Reconciliation window**: the synthetic opening fill carries the reconciliation time rather
than the report time, so a real fill that lands between the position report query and
reconciliation also stays off the position. Continuous position checks align it only when they
run with `generate_missing_orders` enabled.
- **Opening order purge**: the protection needs the synthetic opening order in the cache. When
closed-order purging is enabled, a `purge_closed_orders_buffer_mins` shorter than the venue's
replay delay can remove it first.
- **Bundled fills**: fills that arrive bundled with an order status report bypass the protection.
A fill kept off the position stays on its order only, as in
[order-only fill projection](#order-only-fill-projection), with these effects:
- **Event store restore**: restoring the cache from the event store applies the fill to a
position, so the restored position can overstate the venue quantity.
- **Fill voids**: a venue fill void for the fill is rejected, so its order stays filled.
- **Locked balance**: an open order that the fill closes keeps its locked balance until the next
order event for that instrument and account.
- **Position checks**: if the reconciled position closes before a continuous position check aligns
it, later checks stop at the fill until `position_check_threshold_ms` has passed since it first
failed to apply, then align the position synthetically.
#### Startup position validation
After applying startup reports, the live node checks each **in-scope explicit position report,
including flat reports,** against the cache. An unresolved position stops startup **before actor or
strategy `on_start`**. The error identifies the account, instrument, venue quantity, and recovery
failure.
| Position case | Cache identity | Quantity requirement |
| ------------------------------------------ | ----------------------- | ------------------------ |
| HEDGING with venue position IDs | Exact venue position ID | Exact quantity |
| NETTING with or without venue position IDs | Account and instrument | Within account tolerance |
| Net reports without position IDs | Account and instrument | Within account tolerance |
When the venue reports both long and short positions, both side totals must also match;
equal net quantities alone are insufficient. A residual difference outside the account tolerance
remains unresolved even if it rounds to zero at the instrument size precision. Differences within
the tolerance remain acceptable, including tiny residuals around zero.
For an open position with a reported `avg_px_open`, startup also checks the entry average using
the fill-adjustment relative tolerance of 0.01%. Average entry prices can fall between instrument
price ticks; the comparison does not round them to instrument price precision. NETTING reports
use quantity-weighted entry averages for each reported side. If a side spans several reports,
all contributing reports must supply an average to establish that side's price target. A report
can skip this check or widen its tolerance, as described in
[reported entry averages](#reported-entry-averages).
Matching quantity alone does not resolve a reported entry-price mismatch. When quantities already
match, position reconciliation does not generate a correction solely to change the entry average;
the remaining price mismatch fails startup. When it corrects quantity, startup still fails if the
resulting average remains outside tolerance. Synthetic recovery does not establish historical
realized PnL.
#### Reported entry averages
Two `PositionStatusReport` fields tell reconciliation how to read `avg_px_open`. Their defaults
keep the behavior described above.
- `avg_px_open_reconciliation`: `MATCH` (default) compares the average with the cached entry
average and prices synthetic fills from it. `OPENING_ONLY` marks an average computed by a
different method, such as the remaining FIFO lots, which legitimately differs from the cached
quantity-weighted average once a position has been reduced.
- `avg_px_open_precision`: the decimal places the venue kept when it rounds or truncates the
average. When unset, comparisons add no allowance.
| Use of the average | `OPENING_ONLY` | `avg_px_open_precision` set |
| --------------------------------- | ------------------------------------------------------ | ------------------------------------------------------------------------ |
| Startup entry-price check | Skipped for the side; quantity checks still apply. | Passes within 0.01% plus one unit at the coarsest precision on the side. |
| Fill adjustment match | Quantity only, so a price difference keeps real fills. | Uses the same price allowance; quantity must still match exactly. |
| Opening from flat or reversing | Uses the reported average. | Uses the reported average. |
| Increasing or reducing a position | Uses the cached average. | Uses the cached average when it already matches the report. |
For `OPENING_ONLY`, opening from flat includes the synthetic opening fill that
[fill adjustment](#fill-adjustment) adds before a partial window. A precision-marked report prices
that fill at the window fills' average when the window position has the same side as the report
and its average already matches the report, and at the default price otherwise.
One `OPENING_ONLY` report makes its whole side incomparable, because the side's weighted average
would mix methods. Pricing a change to an existing position at the cached average keeps its entry
average and books no synthetic trading PnL on a reduction. When no cached average exists, synthetic
fills fall back to the reported average.
#### Recovery prerequisites and filters
Creating a position in an empty cache from a report without order or fill history requires
`avg_px_open`. The engine does not invent an entry price.
- **Missing-order generation**: disabling `generate_missing_orders` does not bypass startup validation.
- **Report scope**: position-report and instrument filters still apply; excluded reports are not checked.
- **Reconciliation disabled**: disabling reconciliation skips the check entirely.
### Fill adjustment
The engine can analyze zero-crossings, remove closed lifecycles, and generate a synthetic fill when
the reported fills do not explain the current venue position. A declared `lookback_start` does not
skip this adjustment. Without a reported `avg_px_open`, the engine preserves the original
orders and fills instead of inventing a price for synthetic fill adjustment. Setting
`filter_position_reports` skips fill adjustment for both declared and undeclared windows.
Fill adjustment uses these terms:
- **Zero-crossing**: position quantity crosses through zero (FLAT), marking a lifecycle boundary.
- **Lifecycle**: a sequence of fills between zero-crossings representing one open-close cycle.
- **Synthetic fill**: a calculated fill report representing missing activity, priced to achieve the
correct average position.
- **Tolerance**: fill adjustment uses a relative entry-price tolerance of 0.0001 (0.01%) to absorb
minor calculation differences. Startup validation uses the same price tolerance and the account's
separate quantity tolerance. A report that sets `avg_px_open_precision` adds one unit at that
precision to the price tolerance.
When `generate_missing_orders` is disabled, the engine still processes raw venue order reports. It
filters completed lifecycles when the current lifecycle explains the venue position, but it does
not add or replace synthetic reports to align a fill window with the venue position or materialize
an order for a fill group that has no order report.
Adapters that apply a history cutoff should still declare it through the
[mass-status history contract](#mass-status-history-contract). The declaration records provenance.
It does not authorize leaving an explicit position report unmatched.
### Bounded history safety
**An explicit position report is authoritative for the position, whether open or flat.
The engine either aligns to that report within reconciliation tolerances or fails closed.** It does
not replay the bounded window to decide whether the report is authoritative. It applies the available
orders and fills, and by default generates the orders and fills required to reach the report. Disabling
`generate_missing_orders` does not allow a mismatch to stand.
A bounded fill for an instrument with no in-scope explicit position report does not open, close, or change a
position, and it does not update portfolio economics. The order still reaches the reported status
and filled quantity. Raw reconciliation reports remain available.
For compatibility, a mass status without a declared `lookback_start` can still apply historical
fills when there is no position report. This exception does not treat a missing report as flat.
Retained-fill deduplication and projection of older lifecycles continue to apply in both paths.
Reports with an explicit `venue_position_id` follow the position-specific reconciliation path and
do not require NETTING lifecycle inference.
### Failure handling
- An adapter can preserve successful report legs after an individual source failure. Explicitly
bounded mass statuses must mark the result incomplete. Incompleteness does not veto an explicit
position report. A bounded fill with no explicit position report does not change a position.
- Fill reports arriving before order status reports are deferred until order state is available.
#### Commission failures
An adapter fill commission that cannot be calculated or represented fails the report request under
the [adapter contract](../../developer_guide/adapters.md#commission-failure-handling). The adapter does
not drop that fill or replace its commission with zero or a generic formula. Startup stops before
applying that client's mass status.
When the engine asks the responsible execution client to calculate an inferred-fill commission, a
failure defers the inferred quantity and dependent terminal transition until a later reconciliation
cycle succeeds. Valid explicit fills from the same report set can still apply. For an external order,
the engine resolves the commission before adding the order to the cache or publishing its initial
event, so a failure defers the entire external order. An unavailable responsible execution client
has the same fail-closed result.
An inferred-fill commission failure while applying an otherwise successful mass status does not
by itself stop startup. Startup still fails if an in-scope explicit position report remains unresolved.
Otherwise, the unresolved work remains pending for a later reconciliation cycle.
If startup reconciliation fails for any other reason, the system logs an error and does not start.
## Runtime checks
Continuous reconciliation starts after startup reconciliation completes. It:
- Monitors in-flight orders for delays exceeding a configured threshold.
- Reconciles open orders with the venue at configured intervals.
- Checks position status with the venue at configured intervals.
- Audits internal *own* order books against the venue's public books.
The `reconciliation_startup_delay_secs` parameter adds a further delay *after* startup
reconciliation completes, giving the system time to stabilize. [Runtime scenarios](#runtime-scenarios)
lists the outcome of each check.
### In-flight checks
After exhausting the configured retries, the in-flight checker rejects a timed-out `SUBMITTED`
order with reason `INFLIGHT_TIMEOUT` and resolves a timed-out `PENDING_CANCEL` or `PENDING_UPDATE`
order to `CANCELED`, unless the order has never been accepted and submission retention applies.
`SubmissionRecoveryPolicy::RetainUnresolved` retains those orders, and Polymarket requires retention
regardless of the configured policy. Retention stops automatic per-order recovery queries while
allowing later venue evidence to resolve the order. See [Submission recovery](#submission-recovery)
for the retention rules.
[Terminal reconciliation provenance](policies.md#terminal-reconciliation-provenance) distinguishes
local policy resolutions from venue-reported outcomes.
### Order consistency checks
The table below applies when cache state differs from venue state. A missing open-order report does
not by itself prove a pending modify or cancel outcome, so these checks leave those states unresolved
until another check can determine the venue state.
:::info[Full-history checks]
The *Not found* rows apply only in full-history mode (`open_check_open_only=False`);
open-only mode is the default. Submission retention also applies here: an unacknowledged
`SUBMITTED` order remains unresolved when the policy or its client requires retention.
:::
| Cache status | Venue status | Resolution | Rationale |
| ------------------ | ------------ | ------------ | ------------------------------------------------------------------- |
| `SUBMITTED` | *Not found* | `REJECTED` | Order never confirmed by venue (e.g., lost during network error). |
| `ACCEPTED` | *Not found* | `REJECTED` | Order doesn't exist at venue, likely was never successfully placed. |
| `ACCEPTED` | `CANCELED` | `CANCELED` | Venue canceled the order (user action or venue-initiated). |
| `ACCEPTED` | `EXPIRED` | `EXPIRED` | Order reached GTD expiration at venue. |
| `ACCEPTED` | `REJECTED` | `REJECTED` | Venue rejected after initial acceptance (rare but possible). |
| `PENDING_UPDATE` | *Not found* | *Unresolved* | Modification outcome remains unknown. |
| `PENDING_CANCEL` | *Not found* | *Unresolved* | Cancellation outcome remains unknown. |
| `PARTIALLY_FILLED` | `CANCELED` | `CANCELED` | Order canceled at venue with fills preserved. |
| `PARTIALLY_FILLED` | *Not found* | `CANCELED` | Order doesn't exist but had fills (reconciles fill history). |
#### Open-only mode
Venue "open orders" endpoints exclude closed orders by design, making it impossible to distinguish
missing orders from recently closed ones. Pending cancel/update orders remain unresolved when a
missing-order check cannot prove the final venue state.
#### Recent order protection
The engine skips reconciliation for orders whose last event falls within the
`open_check_threshold_ms` window. This prevents false positives from race conditions where the
venue is still processing.
#### Targeted query safeguard
Before applying a terminal "not found" resolution, the engine issues a single-order query to the
venue. This catches false negatives from bulk query limitations or timing delays.
#### Completed orders
`FILLED` orders that are "not found" at the venue are silently ignored. Venues commonly drop
completed orders from their query results.
### Position checks
Position checks use separate retry counters per instrument and account. A successful position
match clears the counter, while repeated unresolved discrepancies stop active reconciliation for
that pair until the discrepancy clears.
If a venue position query fails, the engine skips cached positions for that venue during the cycle
instead of treating missing reports as flat.
### Retry coordination
The in-flight loop increments its own per-order retry count against `inflight_check_retries` and
mirrors that value into missing-order tracking. The open-order loop increments the missing-order
count against `open_check_missing_retries`. Each loop applies its own limit; neither setting
automatically overrides the other.
When the open-order loop exhausts retries, the engine issues one targeted
`GenerateOrderStatusReport` probe before applying a terminal state or leaving an ambiguous pending
cancel/update unresolved. If the venue returns the order, reconciliation proceeds and missing-order
tracking clears. If a pending state remains unresolved, the engine also resets the in-flight count
before checking again after the configured threshold.
When `RetainUnresolved` or the registered client requires retention, a tracked submission that has
not been acknowledged keeps its original in-flight budget instead of resetting it. Repeated
missing-order checks cannot postpone exhaustion of that budget; exhaustion retains the unresolved
submission without a synthetic terminal event.
### Single-order query throttling
The engine caps single-order queries per cycle via `max_single_order_queries_per_cycle`. Remaining
orders are deferred to the next cycle. `single_order_query_delay_ms` spaces out consecutive queries
to avoid rate limits. This handles bulk query failures across hundreds of orders without
overwhelming the venue API.
## Common reconciliation scenarios
The tables below cover startup reconciliation (mass status), runtime checks (in-flight order
checks, open-order polls, own-books audits), fill adjustment, and bounded history.
### Startup scenarios
| Scenario | Description | System behavior |
| -------------------------------------- | ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| **Order state discrepancy** | Local state differs from venue (e.g., local `SUBMITTED`, venue `REJECTED`). | Updates local order to match venue state, emits missing events. |
| **Missed fills** | Complete venue history contains a fill the engine missed. | Generates the missing `OrderFilled` event and applies its economics. |
| **Multiple fills** | A complete, coherent report set contains several fills for an order. | Reconstructs the reported fill history in event order. |
| **Incomplete bounded history** | A required order, fill, or position source failed or could not be mapped. | Aligns an explicit position report, or leaves it unresolved. Fills with no report stay order-only. |
| **Ambiguous bounded lifecycle** | The bounded reports do not prove one coherent NETTING position transition. | An explicit position report is still the quantity target. Fills with no report stay order-only. |
| **External orders** | Orders exist on venue but not in local cache. | Creates unclaimed orders with strategy ID `EXTERNAL` and tag `VENUE`. |
| **Missing client origin** | A cached order in the mass status has no recorded execution-client origin. | Logs one aggregated warning with a count and sample IDs; reconciles against the reporting client. |
| **Conflicting client origin** | A cached order's origin differs from the client that supplied the report. | Logs one aggregated deprecation warning; reconciliation proceeds during the compatibility period. |
| **Partially filled then canceled** | Order partially filled then canceled by venue. | Updates state to `CANCELED`, preserves fill history. |
| **Different fill data** | Venue reports different fill price/commission than cached. | Preserves cached data, logs discrepancies. |
| **Filtered orders** | Orders marked for filtering via config. | Skips based on `filtered_client_order_ids` or instrument filters. |
| **Unresolved instrument** | A report references an in-scope instrument the adapter has not loaded. | Fails startup for open order and position reports; warns and marks bounded history incomplete otherwise. |
| **Fill commission failure** | An adapter cannot represent a required fill commission while building reports. | Fails mass-status generation and stops startup before applying that client's reports. |
| **Inferred-fill commission failure** | The responsible execution client cannot calculate a required commission. | Defers inferred work; an external order remains absent, while valid explicit fills can still apply. |
| **Duplicate order reports** | Multiple orders share the same identifier. | Deduplicates with warning logged. |
| **Position quantity mismatch (long)** | Internal long position differs from venue (e.g., 100 vs 150). | Generates BUY LIMIT with calculated price when `generate_missing_orders=True`. |
| **Position quantity mismatch (short)** | Internal short position differs from venue (e.g., -100 vs -150). | Generates SELL LIMIT with calculated price when `generate_missing_orders=True`. |
| **Position reduction** | Venue position smaller than internal (e.g., internal 150 long, venue 100 long). | Generates opposite-side LIMIT order with calculated price. |
| **Position side flip** | Internal position opposite of venue (e.g., internal 100 long, venue 50 short). | Generates LIMIT order to close internal and open external position. |
| **Internal reconciliation orders** | Orders generated to align position discrepancies. | Tags `RECONCILIATION`; uses a claim when configured, otherwise `EXTERNAL`. |
### Runtime scenarios
| Scenario | Description | System behavior |
| ----------------------------------- | --------------------------------------------------------- | ----------------------------------------------- |
| **In-flight submit timeout** | `SUBMITTED` remains unconfirmed beyond retry exhaustion. | Resolves to `REJECTED` with `INFLIGHT_TIMEOUT`. |
| **In-flight cancel/update timeout** | `PENDING_CANCEL` or `PENDING_UPDATE` exceeds the retries. | Resolves to `CANCELED` through reconciliation. |
| **Open orders check discrepancy** | Periodic poll detects a venue state change. | Confirms status and applies transitions. |
| **Position check discrepancy** | Periodic poll detects a position mismatch. | Generates reconciliation events when eligible. |
| **Commission construction failure** | A required fill commission cannot be represented. | Defers the affected work to a later cycle. |
| **Own books audit mismatch** | Own order books diverge from venue public books. | Audits and logs inconsistencies. |
The in-flight timeout rows do not apply to an unacknowledged order when the policy or its client
requires submission retention; that order stays unresolved. See [In-flight checks](#in-flight-checks).
### Fill adjustment scenarios
These scenarios apply whether or not the mass status declares a `lookback_start`:
| Scenario | Description | System behavior |
| ----------------------------------------- | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Complete lifecycle** | All fills from opening to current state are captured. | No adjustment. |
| **Incomplete single lifecycle** | Reports miss opening fills, with no zero-crossings. | Adds a synthetic opening fill with calculated price. |
| **Multiple lifecycles, current matches** | Zero-crossings separate earlier and current lifecycles. | Filters out old lifecycles and retains the current one. An order whose fills span the crossing keeps its earlier fills, projected onto order state only, so its filled quantity is not inferred. |
| **Multiple lifecycles, current mismatch** | The current lifecycle differs from the venue position. | Generates a synthetic position fill and preserves reported orders for replay deduplication, projecting their earlier fills onto order state only. |
| **Flat position** | The venue reports flat regardless of fill history. | Makes no adjustment. |
| **No fills** | The report set contains no fills. | Returns the empty fill set. |
### Bounded history scenarios
| Position report | Historical reports | System behavior |
| ----------------- | --------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| **Explicit open** | Complete, incomplete, or ambiguous bounded history. | Applies available reports and attempts recovery to the reported quantity and entry average; unresolved differences block startup. |
| **Explicit flat** | Complete, incomplete, or ambiguous bounded history. | Applies available reports and closes residual exposure when generation is enabled; unresolved quantity differences block startup. |
| **Missing** | Bounded history, with or without a cached predecessor. | Recovers order state only; fills do not change positions or portfolio economics. |
| **Filtered** | Position-report or instrument filters exclude the report. | Position-report filtering makes bounded fills order-only; instrument filtering excludes both orders and positions. |
## Common reconciliation issues
:::tip
For persistent issues, inspect the venue reports and cached ownership before dropping state or
flattening an account.
:::
### Missing trade reports
Some venues filter out older trades. Increase `reconciliation_lookback_mins` or persist all events
locally. Explicitly bounded adapters mark incomplete history so unsupported fills do not change
positions or portfolio economics.
### Position mismatches
External orders that predate the lookback window cause position drift. Increase the window, restore
retained state, or let an authoritative position report reconcile Shown in full with attribution under the source's licence. Licence: LGPL-3.0
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.