BigQuant Data Sources, Feature Extraction, and Return Labeling

Summary

This Q&A explains several BigQuant modeling interfaces. It distinguishes a DataFrame, the tabular data type used by pandas, from a DataSource, a wrapper for data access and transfer between modules that can hide whether storage is in memory, local files, or distributed files. It also clarifies that cached functions return an Outputs object, while M.data is typically a DataSource. For feature extraction, the general interface uses factors already in the platform library; the user-defined interface supports custom factors. Feature transforms alter numeric inputs, and integer conversion can represent categorical values as numbers. The document says parse_general_features had been removed in the then-current platform version.

The labeler example derives a supervised-learning target from returns, scales and clips it, and shifts the range upward to make labels nonnegative. Holding period, entry and exit prices, and an optional benchmark govern the return calculation; relative or volatility-adjusted returns are also suggested. The note points to platform documentation for details and does not specify the transformations themselves, explain every labeler parameter, or provide evidence that any labeling choice improves a model.

Key ideas

• A DataSource abstracts storage and passes data between platform modules, while a DataFrame is a pandas data type.
• Cached functions return an Outputs object, whereas M.data is typically a DataSource.
• General feature extraction uses factors already in the platform library, while user feature extraction supports custom factors.
• Numeric transforms and integer encoding can prepare features, including categorical values, for machine learning.
• Return labels can be scaled, clipped, shifted to nonnegative values, and configured by holding period, trade prices, and benchmark.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.