Chen-Pelger-Zhu Firm Characteristics for Asset Pricing
Summary
This document describes a reproducible US equities benchmark drawn from the Chen, Pelger, and Zhu asset-pricing replication archive. It contains roughly 1.2 million monthly stock observations from 1967 through 2016, with 46 firm characteristics, macroeconomic indicators, forward returns, and predefined training, validation, and test partitions. Firm identifiers are anonymous and persistent within each released tensor, but the separate identifier namespaces prevent linking firms across tensors.
The dataset supports machine-learning studies of asset pricing where consistent benchmark data matters more than interpreting individual securities. Its published chronological partitions assign 1967–1986 to training, 1987–1991 to validation, and 1992–2016 to testing. The document outlines downloading, conversion from the authors’ tensor files to Parquet, and loading the full panel or individual splits, including an option to include macro data. This is a dataset description rather than an empirical result: performance depends on the models and evaluation methods applied, and the anonymized identities limit cross-split firm-level analysis.
Key ideas
- The benchmark contains monthly stock observations, firm characteristics, macro indicators, and forward returns from 1967 through 2016.
- Its predefined chronological splits separate training, validation, and test periods.
- Anonymous firm identifiers persist within each tensor but cannot be linked across tensor namespaces.
- The dataset is used for reproducible linear and nonlinear asset-pricing experiments.
- The converted Parquet files preserve identifiers recovered from the original tensor data.
Tags
Full text
# Firm Characteristics (Chen-Pelger-Zhu 2020)
# Firm Characteristics (Chen-Pelger-Zhu 2020)
Anonymized panel of ~1.2M stock-month observations with 46 firm
characteristics and forward returns, spanning 1967-2016. Built from the
replication archive of Chen, Pelger, and Zhu (2020), *Deep Learning in
Asset Pricing*. Used throughout the book for ML-based asset-pricing
examples where a standard, reproducible benchmark matters more than
symbol-level interpretation.
## Dataset
- **Source**: GitHub replication repo
(https://github.com/jasonzy121/Deep_Learning_Asset_Pricing), which
itself ships the published dataset via Google Drive
- **Coverage**: 1967-01 to 2016-12, monthly observations, ~1.2M rows
- **Features**: 46 firm characteristics (accounting ratios, price-based
measures, momentum variants), 178 macro indicators, forward returns
- **Size on disk**: ~1.1 GB raw CSV; converted to ~500 MB parquet
- **Access**: Public, no API key required
- **Canonical schema**: `symbol` (anonymous integer id), `timestamp`
(monthly Date), 46 characteristic columns, `ret`, `split`
- **Identity scope**: `symbol` is persistent within each tensor released by
the authors. Separate numeric namespaces prevent accidental linking across
tensors because the archive does not publish a cross-split map.
## Pre-defined Splits
The dataset ships with deterministic train/valid/test splits aligned to
the authors' released tensors:
| Split | Period |
| ----- | --------- |
| train | 1967-1986 |
| valid | 1987-1991 |
| test | 1992-2016 |
## Download
This is the largest free dataset in the book: ~1.5 GB pulled from the
authors' Google Drive folder (RetChar.csv alone is ~1.1 GB), followed by
a tensor-to-Parquet conversion. The tensor conversion recovers persistent
anonymous firm identifiers that the flattened CSV omits. How long it takes
depends on your bandwidth and disk speed. Per-file progress is printed as it
runs. A single command downloads and converts; no separate `--convert` pass
is needed.
```bash
# Download the ~1.5 GB folder and convert to parquet in one step
uv run python data/equities/firm_characteristics/download.py
# Verify what's already on disk, do not refetch
uv run python data/equities/firm_characteristics/download.py --check
# Force a re-download even if files exist
uv run python data/equities/firm_characteristics/download.py --force
# Re-run only the tensor-to-Parquet conversion (files already downloaded)
uv run python data/equities/firm_characteristics/download.py --convert
```
It is also fetched automatically as part of `data/download_all.py`. To
skip it there (e.g. on a metered or space-constrained connection), pass
`--skip-firm-characteristics`.
Output layout under `$ML4T_DATA_PATH/equities/firm_characteristics/`:
```
firm_characteristics_all.parquet # full panel
firm_characteristics_train.parquet # 1967-1986
firm_characteristics_valid.parquet # 1987-1991
firm_characteristics_test.parquet # 1992-2016
dl_asset_pricing/ # raw CSV + NPZ staging
RetChar.csv
Macro.csv
char/Char_{train,valid,test}.npz
macro/macro_{train,valid,test}.npz
RF/RF_{train,valid,test}_normalized_task_1.npz
```
The staging directory is kept so experiments that need the pre-split
NPZ arrays (the original Chen-Pelger-Zhu format) can read them
directly.
## Loading
```python
from data import load_firm_characteristics
df = load_firm_characteristics() # full panel
df = load_firm_characteristics(split="train")
df = load_firm_characteristics(split="test", include_macro=True)
```
If the canonical parquets are missing, the loader raises
`DataNotFoundError` pointing at the download command.
## Consumers
- Chapters 10-16 - standard benchmark for linear and nonlinear
asset-pricing models.
- `case_studies/us_firm_characteristics/` - Chen-Pelger-Zhu replication
pipeline (CV, GBM, latent factor, deep models).Shown in full with attribution under the source's licence. Licence: MIT
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.