Sources of Historical Order Book Data and Data-Format Checks
Summary
The document compares sources for historical limit order book data in US equities and cryptocurrency. It identifies LOBSTER as a widely used research source for US equities, and mentions exchange-provided Binance futures files, Crypto-Lake, Tardis, and CryptoHFTData for crypto data. Access terms and coverage vary, with some sources offering sample or free data and others requiring paid plans.
It also explains why format matters when using order book history: snapshots, incremental updates, raw exchange messages, and normalized updates are not interchangeable. Researchers should check which form a provider supplies and whether it supports accurate reconstruction of the book at a historical time. The discussion is a collection of recommendations rather than a systematic comparison of data quality, coverage, or costs. One recommendation includes an affiliated provider disclosure, so its claims should be assessed accordingly.
Key ideas
- LOBSTER is a commonly used source for historical US equity order book research.
- Crypto providers differ in coverage, access methods, and pricing.
- Check whether historical data contains snapshots, incremental changes, raw messages, or normalized updates.
- Data format affects whether a historical order book can be reconstructed accurately.
Tags
Full text
# Where can I get order book data? # Where can I get order book data? In my experience trading in asian markets, I have found a few resources that conduct calculations on order books. They are able to provide the size and real time movements on specific types of orders classified based on their size. I can then see on the dashboard how much individual investors are buying, how much institutions are buying, etc. I was wondering if there are similar platforms that can provide this data in US stock markets. Thanks in advance. ## Answer by Julie Taylor (score 2, accepted) https://quant.stackexchange.com/a/69748 Generally good quality data is not free. In academia, LOBSTER data is frequently used for orderbook modeling/research. You can check out their sample files to examine their data quality. Data description is provided here and if satisfied they have a subscription option. ## Answer by Lefty (score 2) https://quant.stackexchange.com/a/74148 If anyone is interested in cryptocurrency data, here are some options: - Binance provides free, but on-request, futures orderbook data in csv files. This is probably the best free resource currently, as orderbook data are almost exclusively paid and rather expensive. - Crypto-Lake.com provides cheap orderbook data via S3/Python api. They have some small free dataset too. - Tardis.dev provides data plans with better data coverage, but much higher price tag. They have Python/NodeJS/HTTP api. ## Answer by Mate Hegedus (score 0) https://quant.stackexchange.com/a/85619 For US equities, LOBSTER is one of the better-known sources for historical limit order book data, especially in academic/research contexts. For crypto order book data, another option is CryptoHFTData: https://www.cryptohftdata.com/ Disclosure: I run CryptoHFTData, so this is an affiliated mention. It provides historical crypto order book data, trades, funding rates, and liquidations across multiple exchanges, mainly for backtesting, market microstructure research, and execution/slippage analysis. The best thing about it is that it is free. For any provider, I would check whether the data includes full snapshots, deltas, raw exchange messages, or normalized updates, since this affects whether you can correctly reconstruct the book at a given historical point.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.