Choosing Float64 Precision for Historical Price Archives
Summary
The discussion considers whether float64 is sufficiently precise for storing historical equity prices in an HDF5 archive, compared with a fixed-point representation that separates a decimal significand and exponent. It explains that floating-point precision is relative to a number’s scale, so leading zeros do not consume the same precision as they would in a fixed-width decimal representation. Float64 also offers convenient NaN handling and works readily with common numerical tools.
The answers say market prices generally require far fewer significant digits than float64 provides, and argue that rounding near its sixteenth or seventeenth decimal digit is unlikely to matter for prices with around six to eight meaningful digits. One contributor favors fixed-precision decimal storage in some database settings, but later describes conversion friction when processing with NumPy and Pandas; another says values above roughly ten digits add little practical value for market prices. These are practical judgments rather than a formal error analysis. The appropriate representation still depends on downstream calculations and whether exact decimal reconstruction is required in unusual cases.
Key ideas
- Float64 stores precision relative to a number’s scale, so leading zeros do not use significant-digit capacity in the same way as fixed precision.
- Float64 provides convenient NaN support and integrates readily with common numerical processing tools.
- The answers judge typical market prices to need substantially fewer significant digits than float64 can represent.
- Fixed-point storage can be considered when exact decimal representation is required, though conversions may complicate analysis workflows.
Tags
Full text
# float64 to store price data: is precision sufficient?
# float64 to store price data: is precision sufficient?
I am looking to store equity price data in a hdf5 table. The use will be purely as a historical archive, not as day-to-day data source.
Options
- One option would be to store base10 significand and exponent separately as e.g. uint64 and uint8. The downside is that it is fairly awkward to handle especially as int do not come out of the box with NaN handling for missing values.
- The other option would be to use float64 which is easier to handle and has NaN support built-in.
My question: Does float64 have sufficient precision to store price data? What is the experience of the number of significant digits required for a price archive?
Note: float64 seems to have 15-17 "significant decimal digits" precision. Not sure whether this means "significant digits" or whether this only refers to the decimal digits.
## Answer by rhaskett (score 5, accepted)
https://quant.stackexchange.com/a/15497
As background, Floating point precision is a way of storing numbers such that the precision is relative to the largest digit. For instance, the number $0.00123$ stored in fixed precision needs 6 digits of precision (3 zeros and the 3 non-zero numbers). However, this same number stored as floating point precision $1.23 \cdot 10^{-3}$ needs only 3 significant decimal digits to store. Floating point is generally a more efficient way to store numbers that have many different orders of magnitude but more importantly they are stored in a form where the computer can do efficient basic calculations like multiplication and in your case is much easier to work with than option 1.
Even the deepest markets (treasuries, currencies) need only 6-7 digits of floating precision to store price data. There are some important limitations meaning the prices stored won't always be exact but maybe approximated at the 16th or 17th digit. If the prices are used in calculations later (we are on quant finance after all) it would be shocking if for numbers with 6-7 digits of precision rounding to the 16th/17th digit mattered at all. On the corner case that this approximation matters you can look into option 3 which is fixed point storage.
## Answer by comte (score 1)
https://quant.stackexchange.com/a/29581
As a practical viewpoint: having habits from float32-by-default time, I designed my db with Numeric type (aka Decimal, ie fixed-precision). In this case, it was important. In most case, I took a maximum 8 digits precision.
But now with float64 + Numpy (on which Pandas is based) not handling Decimal but float64, I'm converting my db schema to float64 (ie double on postgresql). Moving back-and-forth from float64 (for Numpy processing) and Decimal (for storage) is just too much pain...
Being also a market operator, I hardly see any value-added above 10 digits, markets bid/ask being what they are.
## Answer by MattR (score 0)
https://quant.stackexchange.com/a/15499
In my case, it really depends if you are using any software at all. I usually get my Discount Factors with a 15 decimals precision. So im always storing at least 15 digits for DF, and yields up to 7-8 as rhaskett said.
For equities, i guess it's pretty much the same as for yields, or even less. Nevertheless i store any return, price or ratio with a 7-8 digit precision.Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.