Sources and Handling for Historical High-Frequency Forex Tick Data
Summary
The document lists potential sources for historical foreign exchange tick data, including TrueFX, Dukascopy, TickStory Lite, and HistData.com. It presents Dukascopy as an option for downloading hourly data files, either through a website or direct scripted retrieval. Access may require registration, and the website approach involves substantial manual effort; the availability and cost of other providers may vary.
It describes Dukascopy’s files as compressed binary records containing relative time, ask and bid prices, and ask and bid volumes. Users must decompress and parse the format before converting it into a more convenient time series for analysis. The post offers a practical lead and a description of the data structure, but it does not compare source accuracy, quote conventions, coverage consistency, or licensing. Researchers should verify current access conditions and data suitability before relying on a source.
Key ideas
- Several providers are suggested as sources of historical foreign exchange tick data.
- Dukascopy data can be retrieved in hourly files through its website or scripted downloads.
- The binary files contain timestamps, bid and ask prices, and corresponding volumes.
- The compressed format must be decoded and converted for convenient time-series analysis.
- Provider coverage, access, cost, and data quality should be checked for the intended use.
Tags
Full text
# Where can someone get free (or very cheap) high frequency tick forex data?
# Where can someone get free (or very cheap) high frequency tick forex data?
I am currently working on a large data set (approx 80 million data points over 10 years). I would like another set of data that has one currency in common. Eg, I have EUR/USD and would like USD/CNY or EUR/AUD etc. Doesn't need to be over the full 10 years, 1 year would be more than sufficient.
I found a few places online that sell this data, but the cheapest I could find was approx 60 Euros which is a lot for a student.
Is there anywhere someone can get data like this cheaper?
## Answer by rupweb (score 3, accepted)
https://quant.stackexchange.com/a/34961
You might get something from Integral's True FX
## Answer by Tony (score 8)
https://quant.stackexchange.com/a/35005
Dukascopy offers historical tick data. Through their historical data website you can download what you want, but registration is required, and lots of manual clicking.
However if you are comfortable with scripting, you can directly download the tick data yourself. The URL pattern is `http://www.dukascopy.com/datafeed/{currency}/{year}/{month}/{day}/{hour}h_ticks.bi5`, so for example `http://www.dukascopy.com/datafeed/AUDCAD/2017/00/01/23h_ticks.bi5` gets you ticks for AUDCAD from 1 January 2017, 23:00-23:59:59.999 UTC.
Note that the months are zero-based (I don't know why), so Jan-Dec is 00-11 (two digits). Every hour is present as a file, even if the market is closed.
The file format is an LZMA-compressed binary packed file, so you will need to decompress it to CSV or other format according to your need. Each tick is 20 bytes, five four-byte fields:
- (long) the relative time from the hour, in milliseconds
- (long) the ask price, in points
- (long) the bid price, in points
- (float) the ask volume
- (float) the bid volume
In python, I use a `struct.unpack('>LLLff', bytes)` to extract the fields.
The data format is very space efficient for transfer and storage, but not efficient for processing, so I convert it to a time series for my storage.
I based my code on this guide, and there's plenty of GitHub repositories to help.
## Answer by Joel Alcedo (score 3)
https://quant.stackexchange.com/a/34985
Take a look as well at TickStory Lite.
IIRC, you should be able to pull tick level FX transactions back to 2003 for free. I used this several months ago, so not sure if anything has changed.
## Answer by nbbo2 (score 1)
https://quant.stackexchange.com/a/55435
Take a look also at HistData.Com, they have both 1 minute data (which I use) as well as Tick Data. It is free or very inexpensive depending on the method of downloading you choose.Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.