Choosing a Database for Real-Time Tick Aggregation

# Which database to choose for storing and aggregating finance data?


# Which database to choose for storing and aggregating finance data?












I'm planing to store stock market data in realtime and aggregate ticks for draw volume based cluster graph. Something like this:

Every tick (or second) data will be grouped by period (1,5,10 minutes; 1,4,24 hours), type (buy, sell) and price; calculated sum of volumes. Result will be something like this:

```
[
  {timestamp: "2016/01/30 15:04:00", period: "1m", price: 123.45, buy: 2345, sell: 1998},
  {timestamp: "2016/01/30 15:04:00", period: "1m", price: 123.46, buy: 3111, sell: 1040},
  {timestamp: "2016/01/30 15:05:00", period: "1m", price: 123.46, buy: 1421, sell: 3475},
  {timestamp: "2016/01/30 15:05:00", period: "1m", price: 123.47, buy: 6056, sell: 9138},
]
```

For delivery ticks from stocks to db I will use nats (https://github.com/nats-io/gnatsd). Which database I can use for store and aggregate in realtime?

## Answer by JOHN (score 5)

https://quant.stackexchange.com/a/23212

check this out Arctic. It's a Man AHL developed Mango DB for store their financial time series. Claimed to be really good. But i haven't try myself.

## Answer by user25064 (score 2)

https://quant.stackexchange.com/a/24817

As @Nicholas said in a comment KX/KDB+ is popular in finance for this purpose. Direct message passing and local aggregation on the machine may be the best method in this case IMO.

## Answer by Martin Seeler (score 2)

https://quant.stackexchange.com/a/27790

There are many different databases out there, all specialized for different use cases. The main parts you should consider are:

- Using a time series database, since they can handle timestamped data (e.g. ticks) more efficient than any SQL solution can by using bucketing and other methods.

- Using a database with a good query language, for example to aggregate multiple values, calculating highs and lows etc.

Therefore, the best solution in my opinion currently out there would be InfluxDB. Not only because the easy API for inserting and querying data, but also because of the whole stack of InfluxData.

## Answer by relativeview (score 1)

https://quant.stackexchange.com/a/27783

Apache Cassandra would be a good fit. It's a partitioned row store, where rows are organized into table using a partition key.

It is common use case to store time series data, you could simply use ticker and a period as partition key. Cassandra is optimized for writes and it's easily scaled, but you need at least 3 servers for it to run.