XBRL from the same filing as the 10-K HTML

A statement filing is two datasets that share an accession and almost nothing else.

1. The exhibit you read — HTML or iXBRL in a browser, which we turn into markdown (Turning SEC filings into markdown).
2. The tagged facts — taxonomy concepts, periods, units, decimals, and dimension axes, which live in the IDEA XBRL zip EDGAR attaches to 10-K / 10-Q / 20-F / 40-F (and /A amendments).

A revenue line in the HTML can be a different concept, a restated prior period, or a segment breakdown that never appears as a single “Revenue” cell. If you grep the markdown for “Net sales” you are not reading XBRL. If you sum every fact for us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax you will double-count segments.

This is how we extract the second dataset without pretending it is the first.

Where the facts actually areCopy Anchor Link

On the live path the SGML submission includes a document whose description is the IDEA XBRL package (a .zip). That zip is not markdown-chunked. Graphics and the HTML 10-K continue on the text path; the zip goes to an XBRL processor that emits an OIM-style fact list, then a flat row per fact.

Inline XBRL in the HTML exhibit is ignored for facts. The HTML is a presentation. The zip is the instance + taxonomy the processor can hash into (concept, period, unit, dimensions). When the zip is missing (a late exhibit, a form we do not treat as a statement filing), we still have markdown and we have no facts for that accession. We do not scrape numbers out of Item 8.

Coverage is statement forms only: 10-K, 10-Q, 20-F, 40-F, including /A. 8-K, 13F, ownership, and N-PORT have other structured extractors. They are not this fact table. Japan EDINET iXBRL is a different corpus and is not merged here.

What a fact is after flattenCopy Anchor Link

Each tagged number becomes a row with, among other fields:

Duration vs instant is why XBRL Search takes period_type = duration, instant, or all. Asking for instants on an income concept is a valid empty set, not a bug.

We also keep text facts and footnote links in sibling tables. A DEI company-name string is not a float in value. If you only query the numeric fact table you will wonder where the entity name went.

dimensions_hash exists so “no axes” and “these three axes” are cheap to group. The public API still returns the dimension map, not the hash.

OIM, not “parse the iXBRL HTML”Copy Anchor Link

The processor emits XBRL Open Information Model JSON (facts as objects with value, decimals, and a dimensions map). Concept, entity, period, and unit often live inside that map, with or without an xbrl: prefix depending on the processor version. We normalize both shapes into one struct: concept, period_*, unit, leftover axes as dimensions.

Anything that looks like concept / entity / period / unit / language / noteId is not a segment axis. If you naively dump dimensions into the public map you will ship xbrl:concept as if it were srt:ProductOrServiceAxis.

Taxonomy family is inferred from namespace URIs (fasb.org/us-gaap, xbrl.sec.gov/dei, IFRS, SRT, …), including a year when the URI has one. Company extension taxonomies (/cikNNNN) are marked as extensions, not as us-gaap. New SEC families should not require a code change if the URI matches the registry pattern.

DEI (Document and Entity Information) is lifted to filing-level metadata: DocumentPeriodEndDate, EntityCentralIndexKey, AmendmentFlag, entity common name. Those facts still exist in the fact list; the copy on the filing row is so Search can filter “this 10-K’s period end” without joining DEI concepts every time.

Periods in the wild include 9999-12-31 and 0001-01-01. We clamp or drop dates a columnar store will reject rather than failing the accession. A single poison fact must not kill 8,000 good ones.

Footnotes are a link table (fact → note id → text), not concatenated into value_str. Analysts who want the number and the hedge need both; stuffing the hedge into the numeric column breaks decimals.

Consolidated is a missing axis, not a flag the filer setsCopy Anchor Link

Filers tag operating segments, product lines, restatement axes, weighted-average vs period-end shares. The consolidated total is the fact whose dimension map is empty. There is no reliable consolidated=true attribute on the instance that you can trust across taxonomies.

So consolidated=true (the default on Concepts, Search, and Filing Facts) keeps empty-dimensions facts. false includes the breakdowns. If you need both, you asked for false and you must group by dimensions yourself.

This is the mistake everyone makes once: they filter concept = Revenue…, sum value, and report 3× revenue. The segments were in the zip.

Units, decimals, and restatementsCopy Anchor Link

A fact without a unit is not always wrong — pure ratios and some flags have none — but a revenue fact with unit = shares is a filer or extension bug you should not “fix” in ingest. We store what the instance said. Your aggregator should group by (concept, unit, dimensions) or you will add dollars to share counts.

decimals is an exponent, not “number of digits after the point.” INF means the value is exact as stored. A finite -6 means the number is accurate to the million. If you round-trip through JSON floats and then compare two 10-Ks, you invented a restatement. Keep the string when the processor cannot make a float (value_str).

Restatements show up as the same concept + period on a later accession (often a 10-K/A or the next year’s 10-K with a restated prior column). We do not collapse those into one “latest truth” row at ingest. Search will return both unless you filter accession or form. That is intentional: “what did they tag in this zip?” is a different question from “what is the company’s current history?” Standardized financials are the second question; this pipeline is the first.

Company extension concepts (aapl:SomethingTheFilerMadeUp) are first-class. Concepts inventory includes them. If you only whitelist us-gaap:* you will under-read every 10-K that matters.

Concepts are not labelsCopy Anchor Link

Display labels (“Net sales”, “Total net revenue”) change by taxonomy version and by filer extension. The id you pass to Search is the prefixed concept. XBRL Concepts is the per-issuer inventory of tags that filer actually used in recent statement filings. Call it first. A substring q= on Search is resolved against that inventory — it is not full-text on the 10-K HTML.

Search defaults to five years of period_end unless you pass accession or period_start. Filing Facts (XBRL Filing Facts) is one accession and has no year default: you asked for that zip, you get that zip.

Issuers without a SEC CIK are 404. There is nothing in this pipeline for them.

Same accession, two idsCopy Anchor Link

Each fact still carries a slim Document so you can open the exhibit you just tagged:

• filing.document_id — pass as id to Document Reference / Contents / Filing Facts
• source_document_url — SEC HTML exhibit (integer CIK in the path)
• source_filing_url — EDGAR index
• backlink_url — our viewer

Markdown chunks and XBRL facts do not share a primary key. The join is accession + (optionally) filename. Standardized Financials cite a statement filing (filing_match) so you can jump to the 10-K; they do not invent a us-gaap: tag for revenue. Mapping warehouse lines to concepts is your problem if you need both. We refused to guess that map on ingest — filers do not use one revenue tag.

Processor notes that matter if you rebuild thisCopy Anchor Link

• Parse the zip, not the iXBRL HTML, as the instance. HTML ingest is a presentation path.
• Flatten to one row per fact with a stable fact_id. Dedup on (accession, fact_id) or you will double-insert on correction.
• Clamp nonsense dates. XBRL instances contain 9999s and 0001s. A warehouse will reject them; drop or sentinel rather than failing the whole accession.
• Keep all_ciks from the SGML header. Co-registrants are real.
• Amendments (/A) are new accessions. Do not overwrite the original accession’s facts unless you are explicitly modeling restatement. We store both; Search will return both unless you filter form_type.
• Do not markdown-chunk the zip. You will index taxonomy schemas.

On correction of the live SGML (see the markdown article), structured facts for that accession are rewritten with the same “replace this accession” rule as chunks.

When to use whichCopy Anchor Link

RelatedCopy Anchor Link