Handling Missing Event Times in Crypto Market Data
Summary
The document describes a timestamp problem in cryptocurrency spot market data: some level 2 order book updates have provider receipt times but no exchange event times. The data also includes trade messages, for which the author expects event identifiers may help establish order, while book updates lack such identifiers. Receipt times are recorded at finer precision than the millisecond event times, and messages may be duplicated or arrive in a different order from their event times.
The author considers assigning missing event times by rounding receipt times down to the nearest millisecond, partly to support later time-bar construction without explicitly modeling latency. Interpolation between surrounding observations is another possibility, but the author worries it could introduce look-ahead bias. The document presents these as open questions rather than validated methods: it provides no tests, comparisons, or final recommendation. Its main lesson is that receipt order and timestamp precision alone do not establish exchange event chronology, so any imputation choice needs to account for latency, ordering uncertainty, and downstream analysis.
Key ideas
- Some order book updates have receipt timestamps but lack exchange event timestamps.
- Receipt times may not preserve exchange event order when messages are delayed or reordered.
- Rounding receipt times to milliseconds is proposed as a simple imputation, not validated as unbiased.
- Interpolating missing timestamps may introduce look-ahead bias depending on how surrounding events are used.
- The document leaves timestamp imputation unresolved and gives no empirical comparison of methods.
Tags
Full text
# Common Methods to Impute Event Time if We Only Have Received Time? # Common Methods to Impute Event Time if We Only Have Received Time? Is there a common way in the literature/in practice to impute an event (trade, order book update, etc.) time from the local observation time and/or from surrounding event times? I have streams of both level 2 order book messages and trade messages for a spot trading pair on a cryptocurrency exchange. I am not completely sure, but I think that the messages for each are ordered in exactly the way the stream/aggregated streams were recorded by the data provider meaning potential duplicates, out of order received_time/event time, etc. Each message has the unix time for when it was received by the data provider (received_time) as well as the unix time for when the event (level 2 order book change or trade) occurred (event_time). The level 2 messages do not include an upwardly-incrementing event ID to assist in chronological ordering while the trades do. received_time is measured in nanoseconds and the event_time is measured in milliseconds. My problem is that for some of the level 2 messages, there are missing event_times and I only have the received_times. This could also be the case for trade messages, but I haven't gotten there yet. If possible, I would prefer to not build a model for the latency at this time and just want to analyze the events in the order that they hopefully occurred on the exchange. An approach that comes to mind is that I could just take the received_time for that event and round it down to the nearest millisecond so things would hopefully, correctly, and without bias be accounted for if we constructed time-aggregated bars down the road. I think that this would align with my goal of not wanting to construct a latency model at this time, but feel that there are some glaring problems with this approach that I'm not thinking of. I am also a bit wary of interpolation methods due to the potential(?) for them to create look-ahead/other bias, but am still open to those.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.