Why Candlestick Data Cannot Reconstruct Realistic Tick History

# Generate tick data from candlestick


# Generate tick data from candlestick












Is there software (or Python / R / ... scripts) to generate (pseudo) tick data from candlestick data.

I have candlestick data (CSV format) from monthly timeframe (MN) to minute timeframe (M1) but time range can be different. Filenames are :

- `SYMBOL1.csv` for M1 timeframe (1 minute)

- `SYMBOL5.csv` for M5 timeframe (5 minutes)

- `SYMBOL15.csv` for M15 timeframe (15 minutes)

- `SYMBOL30.csv` for M30 timeframe (30 minutes)

- `SYMBOL60.csv` for H1 timeframe (1 hour)

- `SYMBOL240.csv` for H4 timeframe (4 hours)

- `SYMBOL1440.csv` for D1 timeframe (1 day)

- `SYMBOL10080.csv` for W1 timeframe (1 week)

- `SYMBOL43200.csv` for MN timeframe (1 month)

I would like to feed software or script with every csv file. I would like to give start datetime and end datetime and software will output a csv file with ticks data.

Ticks data will be generated from the longest timeframe to the shortest timeframe (if data exists). For example the software will output ticks data from M1 candlestick timeframe if data exists for the requested time interval. If there is no data in M1 timeframe, software will try to find data in M5 timeframe to generate ticks.

I understand that the generated ticks will be generated using interpolation (so they won't be exacts)

Such ideas are implemented in Metatrader Strategy Tester http://www.metatrader5.com/en/terminal/help/tester/tester_using/tick_generation

```
$ head _FRA401.csv 
2010.11.16,08:01,3818.0,3820.0,3817.5,3820.0,41
2010.11.16,08:02,3820.0,3823.0,3801.0,3823.0,38
2010.11.16,08:03,3823.0,3825.0,3823.0,3823.5,28

$ tail _FRA401.csv 
2012.11.01,18:32,3477.0,3477.0,3474.0,3474.5,37
2012.11.01,18:33,3474.5,3476.0,3474.5,3475.5,25
2012.11.01,18:34,3475.5,3476.0,3471.5,3472.5,62

$ head _FRA4043200.csv 
2010.11.01,00:00,3818.0,3906.0,3589.5,3620.0,168480
2010.12.01,00:00,3629.0,3940.5,3618.5,3853.5,227760
2011.01.01,00:00,3848.0,4081.5,3605.5,4019.5,266725

$ tail _FRA4043200.csv 
2012.02.01,00:00,3310.5,3490.5,3305.5,3456.5,514738
2012.03.01,00:00,3441.0,3593.0,3343.0,3420.5,353738
2012.04.01,00:00,3428.0,3475.5,2974.5,3128.5,247351
```

## Answer by Darren Cook (score 4, accepted)

https://quant.stackexchange.com/a/4491

We've done this before (*), but it is trivial: one tick for open, one tick for high, one tick for low, one tick for close. For tick timestamps, open is 08:01:00, close is 08:01:59.999, high is 08:01:20 and low is 08:01:40. (Or if that gives strange results, open is 08:01:00.001 and close is 08:02:00.000; it depends how your system makes bars.)

If you want volume to match too, then create N-4 ticks at the mean of open and close, and space out the ticks evenly throughout the minute. You could interpolate a straight line from open to close if you want; you could also set all N-4 ticks equal to high, etc. All choices are fabrications, so it doesn't matter which you choose. I like to fix at the mean, as anyone looking at it will instantly know it is artificial data.

*: This was with a system where we could only patch outages and backfill historical data with tick data; that system would then make the intraday bars. I.e. we couldn't patch intraday bars directly.

If you want it to look like real ticks, because your trading algorithm is only designed to work from the ticks, then you start with straight line interpolation from open to close, then jitter them. Then filter to make sure none exceed high and low, and then choose two ticks to be the high and low and alter their values.

Another variation, specific to R, is to put in the open/close ticks, and use NA for all the others. Then choose 2 ticks randomly to be high and low. Then use R's na.approx or na.spline function. Then jitter. Then filter for high and low again.

If doing this, I strongly suggest you make multiple data sets, each with a different random seed, and run your backtest on each data set. If different data sets get different results, your algorithms are sensitive to the random noise you've introduced. (I.e. your results are meaningless and you need to go and buy real tick data!)

## Answer by Matt Wolf (score 5)

https://quant.stackexchange.com/a/4462

You said:"I understand that the generated ticks will be generated using interpolation (so they won't be exacts)".

You are very optimistic, they will not only be far away from being exact, they (the tick data) will be completely removed from reality, the only parameters known for the tick data will be boundary conditions, such as open high low close. You make a completely wild guess about how many ticks each bar contains and where the market traded/quoted at which point in time within the limits of the candle open and close times and candle high and low. Of course you can do that and Python is most likely the best tool to get the job done quickly. (I recommend http://pandas.pydata.org/)

However, I highly question the usefulness of this approach. You obviously want to use the tick data for some sort of analysis or back test. However, the results will be completely random if you attempt to peruse such tick data for higher frequency analysis/back tests. If you are not interested in such high frequency testing then you should just stick to your (assumed) accurate compressed time series (1minute bars or what have you).

In summary I question the usefulness of generating higher frequency time series from lower frequency time series in general. The opposite obviously makes a lot more sense because you know exactly how 1-hour compressed data points look like from generating those from higher frequency time series such as 1-minute compressed data points.

Just my 2 cents, feel free to elaborate what you want to use the generated tick data for, maybe I miss something here.

## Answer by BlueTrin (score 1)

https://quant.stackexchange.com/a/4460

I do not know of anything that is already made and works out of the box, but I would recommend writing a small script in Python using csv.reader, this would probably be as fast as trying to plug something on your csv, especially if your format is a bit exotic ?