amazon-agi/RealKIE-FCC-Verified
RealKIE-FCC-Verified It is a test set with single and multi-page invoices sourced from the Federal Communications Commission (FCC) to evaluate key information extraction (KIE) performance. Task Extract information from the document in JSON format given the corresponding JSON schema. It contains 75 documents, with: a) image_files: Each document has multiple pages b) json_schema: A common JSON schema requiring extraction of specified information including line… See the full description on the dataset page: https://huggingface.co/datasets/amazon-agi/RealKIE-FCC-Verified.
RealKIE-FCC-Verified
It is a test set with single and multi-page invoices sourced from the Federal Communications Commission (FCC) to evaluate key information extraction (KIE) performance.
Task
Extract information from the document in JSON format given the corresponding JSON schema. It contains 75 documents, with:
a) image_files: Each document has multiple pages
b) json_schema: A common JSON schema requiring extraction of specified information including line items in the invoices
c) json_response: This is the expected output from the model
Modifications to the original dataset
This release resolves two issues present in the original evaluation set (https://arxiv.org/abs/2403.20101):
a) Line Item Grouping: Fields previously treated as independent entries are now grouped within each individual line item, aligning the annotations with real-world invoice structure.
b) Annotation Corrections: Erroneous values in the original annotations have been corrected to ensure data accuracy.
These corrections are applied to json_schema and json_response.
Note: We haven't edited the OCR (text) field, and instead use the original version.
License
Document Source: FCC https://publicfiles.fcc.gov/
Document License: Fair use under 17 USC 107.
License: CC-BY-NC 4.0