Skip to content
All library documents

Storage Options for High-Frequency Tick Data

Article Quant Q&A · Author: pezpezpez
Read the original

Summary

The document collects brief suggestions in response to a firm seeking an off-the-shelf system for storing and querying high-frequency tick data. It names kdb+ as a prominent option, while noting that its programming language may be difficult for some users. Other suggestions include TeaFiles, a simple file format described as fast but likely to require users to build supporting functionality, and HDF5, characterized by one contributor as a NoSQL database with Python integration. A further analytics product is mentioned without explanation.

These are personal recommendations rather than a structured benchmark or deployment guide. The exchange provides no comparative measurements, workload details, pricing, scaling tests, or discussion of data ingestion and query patterns. Its practical lesson is that storage choice involves tradeoffs among performance, language or tooling fit, and how much infrastructure a team must supply itself; readers would need current evidence against their own requirements before choosing.

Key ideas

  • kdb+ is presented as a prominent system for high-frequency market data.
  • TeaFiles is suggested as a fast, simple format that may require users to build supporting tools.
  • HDF5 is offered as an alternative with Python integration.
  • The recommendations are anecdotal and do not include performance comparisons or workload specifications.

Tags

High-frequency tradingMarket microstructure
Full text
# Databases for storing and querying high frequency tick-level data?


# Databases for storing and querying high frequency tick-level data?












My firm is looking for an out-of-the-box database system to store and query high-frequency tick data. What are the best options? It seems that kdb+ is the market leader in this field.

## Answer by Bob Jansen (score 2)

https://quant.stackexchange.com/a/11350

An alternative is the TeaFiles file format. It's simple and boasts a high performance but I believe you'll have to reinvent some wheels.

## Answer by Onyxx (score 1)

https://quant.stackexchange.com/a/11351

kdb+ seems to be the leader but their programming language is a pain really. Personally I use a HDF5. It is a No-SQL database. It integrates very nicely with python.

I have been very happy with it so far.

## Answer by Daniel (score 0)

https://quant.stackexchange.com/a/11361

velocity analytics might be a choice.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.