Reconstructing Best Quotes Across Crypto Data File Boundaries

Summary

The document asks how to reconstruct a cryptocurrency order book from daily files of snapshots and level-two updates, then record each change to the best bid and offer (BBO). Its central concern is that treating the first snapshot of a new day as the first BBO may miss changes that occurred after the previous day’s last event. It also considers carrying the prior day’s final update into the next day’s processing so the top of book can be compared across the boundary.

No solution is provided, and the discussion gives no data or implementation details. The question highlights a key dependency for parallel processing: a day’s initial book state must be consistent with the event sequence and snapshots used to reconstruct that day. A first snapshot can establish the state at its own timestamp, but it cannot reveal intervening top-of-book changes. Any BBO history built from such files is therefore limited by event coverage and snapshot timing; independent daily processing needs a way to establish the correct starting state and avoid either missing or duplicating boundary events.

Key ideas

• Daily file boundaries can interrupt a continuous order-book event sequence.
• A new day’s first snapshot may not reveal BBO changes since the prior day’s last event.
• Parallel reconstruction requires a valid starting book state for each day.
• The document raises the boundary problem but does not give a resolution.

Tags

# How are BBOs are recorded in continuously-open markets across date boundaries?


# How are BBOs are recorded in continuously-open markets across date boundaries?












I have text files containing level 2 order book events (updates and snapshots) for a cryptocurrency trading pair on a crypto exchange for multiple consecutive dates.

I want to reconstruct the order book sequentially and with that, record the bbo (timestamp, levels, etc.) when the top of the book changes. I also want to do this reconstruction/recording in a parallel way where each process is working through the order book/bbos for that day.

The thing that I get hung up on is how do we handle the bbo tracking when the days change over?

Do we take the first snapshot of the day and list that as our first bbo for the day?

If we did this, aren’t we implicitly assuming that there was no change in the top of the order book between the time immediately after the last update on day 1 and the first update on day 2? which does not have to be true. (?)

Or do we include the last update of a day as the first line of the file of the next day so we can see if we should record or not based on if the top of the book changed?

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.