Building an Auditable SEC Companyfacts Table Without Overwriting Records

Summary

This article describes a data-engineering workflow for turning SEC XBRL companyfacts data into an auditable long-form table. Because a tag can have observations across units, periods, filings, accessions, and frames, the author advises preserving each observation’s source fields before applying research-specific selection rules. The proposed table retains entity, taxonomy, tag, unit, period, value, filing, and frame details. Validation checks the expected data structure and required fields, removes exact duplicate rows, and fails when records with the same identity key have conflicting values.

The article reports an offline reproduction using a frozen Apple dataset, including row and tag counts, hashes, and a successful exit status. These figures describe that snapshot only. The workflow separates retrieval from local conversion and recommends recording refreshes and stopping publication when structural or conflict checks fail. It does not choose an accounting interpretation or demonstrate a trading strategy; selecting comparable financial values remains a separate, documented research decision.

Key ideas

• SEC companyfacts may contain multiple observations for one tag because of differences in units, periods, filings, and frames.
• An audit table should preserve each observation’s source and period metadata before business-level filtering.
• Exact duplicates can be counted and removed, while conflicting values under the same identity key should cause a failure.
• The reported Apple run documents one frozen snapshot and does not establish results for other entities or later downloads.
• Researchers must define and retain separate rules for selecting financially comparable observations.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.