Skip to content
All library documents

Sources and Limitations of Free Tick and Order Book Data

Article Quant Q&A · Author: nurxg

Summary

The document surveys sources that may provide free or sample high-frequency market data for research and training. Suggestions include foreign exchange tick histories, exchange sample files, equity order book data, and reconstructed limit order books. Some sources provide raw message feeds that must be interpreted to rebuild trades or book state; others offer downloadable files or API access. Several responses note that exchange samples can provide a useful short example, while specialist vendors may offer broader coverage for a fee.

The main caveat is that availability, quality, and coverage vary. Reported issues include missing days and sparse observations in some nominally fine-grained series, and raw ITCH data requires reconstruction. Free data for major futures markets is generally difficult to obtain because exchange data sales are an important revenue source. The post is a historical collection of suggestions, not a current verification of provider availability, licensing, or suitability for a particular backtest.

Key ideas

  • Free high-frequency data is more readily found for some FX products and sample datasets than for major futures.
  • Exchange sample feeds may require message parsing and order book reconstruction.
  • Some historical FX tick sources are reported to have missing days or sparse intervals.
  • Data quality and completeness should be checked before using a source for research.
  • The listed sources are historical suggestions and may have changed in availability or terms.

Tags

Full text
# Training set of tick-by-tick data?


# Training set of tick-by-tick data?












I'm looking to find a free source of tick by tick data (<1sec) for training purposes. It doesn't need to be longer than a day, and I don't care what instrument, or exchange, or time it is. I just want real numbers with a high frequency.

Is such a data set freely available?

## Answer by Dirk Eddelbuettel (score 15)

https://quant.stackexchange.com/a/423

You could try some of tier-2 exchanges which often give data away for free to drum up interest and hence trading volume in their product.

You will not get SP500, 10-year note, Crude Oil, Gold, ... futures for free on a tick-by-tick basis simply ... because with per-trade fees down, sales of data are the main revenue for exchanges as sternly noted in the media coverage of this week's exchange mergers.

## Answer by Terco (score 13)

https://quant.stackexchange.com/a/427

You have a couple of years of free tick by tick FX data in http://www.dukascopy.com/swiss/english/data_feed/historical/ . The easiest way to get the data is to write scripts to download all the files but if you are only playing around I guess you can get a few of them by hand.

## Answer by Meh (score 7)

https://quant.stackexchange.com/a/435

You can get 1.5 years of dense milisecond tagged forex tick data from here: http://www.truefx.com/?page=downloads (free account required)

There are a few problems with it - some missing days and a couple of days with bad data.

EUR/USD has an average of 200,000 ticks/day for January 2011.

The Dukascopy ticks mentioned in another answer are also good. Another good forex tick data source can be a demo account for FXCM, dbFX or MB Trading. But you need to use the respective APIs to retrieve them.

You can also get free equity market order book data from Trading Physics - http://www.tradingphysics.com/Feeds/DownloadHistoricalITCH.aspx. A free account is required and you can retrieve a limited number of data files per week.

## Answer by NPE (score 3)

https://quant.stackexchange.com/a/445

NYSE has a bunch of sample data on their FTP site: ftp://sampledata:datacap@ftp2.nyxdata.com/custom/

For NYSE's data products that I've worked with, the sample set tends to be a day's worth of data for their entire universe.

## Answer by GH85 (score 3)

https://quant.stackexchange.com/a/9359

Disclosure: I am part of lobster.wiwi.hu-berlin.de which providers data for academic research.

NASDAQ's Historical TotalView-ITCH sample file:

On NASDAQ's public ftp you can find sample files of NASDAQ's Historical TotalView-ITCH. The files contain message data and have to be interpreted to get tick-by-tick data. I.e. you need to reconstruct the book.

Limit order book data reconstructed from NASDAQ's Historical TotalView-ITCH:

In LOBSTER data samples you can find reconstructed data for several tickers traded on NASDAQ with book levels of 1,5,10,30,50.

Similar data is available from TradingPhysics.

## Answer by Contango (score 2)

https://quant.stackexchange.com/a/1220

IQfeed + QCollector will give you 180 days of tick data for pretty much any symbol on the US market. Not free, but definitely good quality (and not too expensive).

## Answer by martino bernasconi (score 1)

https://quant.stackexchange.com/a/47541

There is this https://github.com/martinobdl/ITCH tool you can install and use to get full depth order book data straight from exchange data (NASDAQ) to get trade by trade data (nanosecond scale).

You need the raw data from NASDAQ, you can buy it or download some free samples from their public ftp: ftp://emi.nasdaq.com/ITCH/.

More details in the github repo.

(I'm one of the programmers of the tool)

## Answer by smith378 (score 0)

https://quant.stackexchange.com/a/18477

Heres a link to free tick data for all S&P 500 stocks for the last 40 days. http://alphamodus.com/alphamodus/mod.asp?modid=1048. Hope this helps!

## Answer by Tomas Pazur (score 0)

https://quant.stackexchange.com/a/18606

On ratedata.gaincapital.com you can find about 150 GB of FX tick data. The data is split into zipped weekly .csv files. You can write a script downloading, unpacking and merging the .csv files as i did. Works fine.

## Answer by Judo (score 0)

https://quant.stackexchange.com/a/49818

Dukascopy data for intervals < 10 mins is not great in my experience, the 1-second data only is closer to 5-second as most 1-second timeslots are blank. It is hard to find well priced tick data, but FirstRate Data SPX has 1 second SPX intervals going back a couple of years.

## Answer by lonelysloth (score 0)

https://quant.stackexchange.com/a/70826

If its only a small amount of data, any provider with samples available should be fine. Have a look at data samples on Portara. they have both Trades and Level 1 tick data.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.