Validating Limit Order Books Against Feeds and Known Event Scenarios
Summary
The document discusses domain-specific ways to validate an order book reconstructed from NASDAQ ITCH messages. It recommends canned scenarios that exercise basic book behavior and edge cases, including checking that an execution message reports the trade price specified by the event rather than simply the resting order’s price. Large data runs may expose failures, but the response emphasizes checking event handling and state as well.
Comparing the reconstructed best quotes with a consolidated top-of-book feed can provide an external check, although timestamps may not align exactly. Such a comparison does not establish that orders are placed correctly within a price level or that internal book dynamics are accurate. The response also recommends injecting missing modify, delete, or execution messages and checking gap detection, stale state, and crossed or locked markets. Sequence numbers can reveal that events are missing but cannot identify the missing content, so these checks cannot by themselves guarantee a correct reconstruction.
Key ideas
- Use crafted event sequences to check core order book behavior and edge cases.
- Verify reported trade prices against the execution message’s specified price.
- Compare top-of-book prices and sizes with an external feed while allowing for timestamp differences.
- Inject missing messages to examine gap detection and the handling of stale or inconsistent book state.
- A matching top of book does not validate order placement within each price level.
Tags
Full text
# What techniques are used for testing order book implementations? # What techniques are used for testing order book implementations? I am finishing the implementation of a limit order book for modeling NASDAQ. The order book works off of the ITCH feed. My question is what techniques are typically used for testing order books. I am considering two methods primarily. - Build a canned set of messages to test the functionality and test possible edge cases. - Push large amounts of data through it and see if anything breaks. I am also curious if there are any thoughts on validating the state of the book at any moment in time. I was thinking of using the L1 feed to at least validate that the top of book quotes are the correct prices and sizes. Note: Please don't suggest things like unit testing. I am specifically interested in techniques specific to order books. ## Answer by Louis Marascio (score 9) https://quant.stackexchange.com/a/2100 Validating tops against the consolidated is a good method. Obviously the time stamps won't match up, but the event stream should. Bear in mind that this won't tell you much about whether you're getting the inner dynamics of the book correct (for example, did the newly inserted order go into the right spot within a given price level). You should build canned test scenarios for the basics of the book. For example, if you're order book reports trades, make sure the trade price is correctly reported by an Order Execute At is received (i.e., the price in the message should be reported, not the price of the resting order). You should also validate your handling of data feed gaps. What happens if you miss a Modify, or a Delete, or an Order Ex. How do you detect this. Sequence numbers can tell you that you missed something, but you don't know what you missed. Inject known errors into your data stream and verify your book manages it properly. Look for crossed/locked markets that result of missed messages and stale order state, inspect by hand and see if you did the right thing.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.