pankaj9296/document-accuracy-aggregation-examples
Document Accuracy Aggregation Examples Two error distributions can have the same 99% field accuracy and radically different document error rates: 1% versus 50%. This educational dataset provides reproducible, synthetic examples for understanding document AI evaluation. It helps developers test metric aggregation and explain why field accuracy alone does not describe the proportion of complete documents that need correction. How we calculated these results… See the full description on the dataset page: https://huggingface.co/datasets/pankaj9296/document-accuracy-aggregation-examples.
Document Accuracy Aggregation Examples
Two error distributions can have the same 99% field accuracy and radically different document error rates: 1% versus 50%.
This educational dataset provides reproducible, synthetic examples for understanding document AI evaluation. It helps developers test metric aggregation and explain why field accuracy alone does not describe the proportion of complete documents that need correction.
How we calculated these results
DigiParser constructed 1,000 document records with 50 required fields per record. Each scenario has 500 incorrect fields out of 50,000, giving 99% field accuracy. The concentrated scenario puts all 500 incorrect fields in 10 documents. The spread scenario puts one incorrect field in each of 500 documents. A document is counted as having an error when at least one required field is incorrect.
These rows describe constructed error counts, not document images or extraction measurements. They demonstrate how error distribution changes aggregate document results. They do not evaluate a particular extraction system.
The second file contains 18 calculated probability scenarios. Each assumes a constant field accuracy and statistically independent errors across the required fields. Under that assumption, the probability of a completely correct document equals field accuracy multiplied by itself once for each required field. At 99% field accuracy and 100 fields, that probability is about 36.6%. Correlated errors can give different results; the constructed file shows why.
Files and schema
constructed-document-errors.csv contains 1,000 rows:
independent-field-model.csv contains 18 rows:
The Hub split name train is a file-loading convention. These examples are intended for metric education and checks, rather than training an extraction model.
Reproduce and check
Run python3 verify.py from this directory. It uses Python's standard library and checks all row counts, field totals, error counts and probability calculations. There is no customer data and no external API call.
Source and attribution
Original analysis published 23 September 2026. Package prepared 8 October 2026 by DigiParser.
Source: The Document AI Accuracy Trap.
The source article permits reuse of its example datasets with attribution and a link to the article. Preserve the synthetic-data description and model assumptions when reusing the results. This package retains those existing terms.
Suggested citation: DigiParser (2026), Document Accuracy Aggregation Examples, synthetic examples accompanying The Document AI Accuracy Trap.