Handling Missing Minute-Bar Data in Trading Series

Summary

The document raises a preprocessing question about a minute-bar time series with a small fraction of missing values, where timestamps exist for every minute between the endpoints. It considers replacing missing observations with a rolling mean or median, or using forward fill or backfill, and asks whether the missing share is small enough to avoid adverse effects.

No answer or recommended imputation method is included, so the document does not establish that any approach is appropriate. For trading data, the choice depends on the field being filled and how the missingness arose: carrying prices forward, interpolating, or using rolling statistics can alter returns and downstream signals. The prompt provides no details about the data values, missing intervals, or intended analysis, which limits conclusions about bias and suitability.

Key ideas

• The question concerns missing values in minute-bar data with a complete timestamp grid.
• Candidate approaches include rolling means, rolling medians, forward fill, and backfill.
• The document gives no recommendation or evidence that a low missing rate prevents bias.
• Imputation effects depend on which bar field is missing and how the gaps arose.

Tags

# Using a rolling mean or median to fill missing values


# Using a rolling mean or median to fill missing values












I have some 1-minute bar data. The first datapoint has time t0 and the last one t1. 99.5% of the data between the first timestamp and the last timestamp is there and 0.5% is missing (NaN values). I can confirm that there is a data point for every single minute between t0 and t1.

As a data pre-processing step, for consistency, rather than dropping the NaN values, I was wondering if I can substitute them with a rolling mean or median. Because there are very few missing values (0.5%) this shouldn't cause any ill effects. Would you do this? Would you use the median or mean or just forward/front fill or back fill?

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.