Preparing Downloaded Intraday Stock Data for Model Consistency

Summary

This guide explains how to obtain and read monthly partitioned stock bar data at four intraday frequencies, then align the local compressed tables with corresponding cloud tables used for prediction. It describes the Feather file layout, recommends loading only needed months and columns, and outlines how to map local data fields to the cloud representation before feature construction.

The key preparation steps are to convert local price and amount values from cents to currency units and change the local OHLC missing sentinel to a missing value before scaling. The guide also recommends grouping by the shared numeric instrument identifier, limiting order book features to the three levels present locally, and avoiding fields that exist only in the cloud table, such as the prior close and deeper book levels. These details support consistent local and cloud feature calculations, but the document gives no modeling results or evidence that any particular feature set predicts returns. Researchers still need to validate data quality, temporal alignment, and their own feature pipeline.

Key ideas

• The downloadable data is divided into monthly Feather files for four bar frequencies.
• Local compressed values for prices and traded amount must be scaled from cents to currency units.
• Convert the local OHLC missing-value sentinel before applying the scale conversion.
• Use the shared instrument identifier and limit comparable order book features to the first three levels.
• Avoid relying on prior close or deeper book levels when matching local features to cloud predictions.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.