Team Ai
Apppublic

documentExtractionag051/ExtractDocument

sourceHugging Faceupdated 9mo agoView on Hugging Face
0likes
pp_handler.cpython-313.pyc80 linesDownload Raw Back to __pycache__
1�

2Y�_i��
�3�SSKrSSKrSSKrSSKrSSKJrJr SSKJr \R"\RSS9 \R"\5r
SrSrSrSS	jrSS\S\S
\S\S\S\4SjjrSr\S:Xa4Sr\"\S5S9 gg)�N)�	open_json�create_text_rects)�do_custom_post_processingz"%(levelname)s:%(name)s:%(message)s)�level�formatc�\�SU;a[U5$SU;a[U5$[S5e)z�6Create validation data structure from input JSON.7 8Supports both simplified schema and original engine output format.9 10Args:11    input_data (dict): Input JSON data12 13Returns:14    tuple: (validation_data, ocr_elems, page_dimensions)15�	ocrBlocks�16engineDatazHInvalid input format. Expected 'ocrBlocks' or 'engineData' in input JSON)�_parse_simplified_schema�_parse_engine_output�17ValueError)�18input_datas �bC:\Users\poona\OneDrive\Documents\nextlifyai\Document_Automation\Document_Extraction\pp_handler.py�!create_validation_data_from_inputrs6���j� �'�19�3�3�	��	#�#�J�/�/��c�d�d�c���0nURS/5HGnURS[U5S-5nURSS5URSS54X'MI URS/5Vs/sH(nURSS5URSS5S.PM* sn0S/0S	.n[URS20/55n[R	S[U5S[U5S
35 XEU4$s snf)z�21Parse simplified input schema.22 23Expected format:24{25    "pages": [{"pageNum": 1, "width": 2479, "height": 3508, ...}],26    "ocrBlocks": [{"pageNum": 1, "blockType": "LINE", "text": "...", "geometry": {...}}, ...]27}28�pages�pageNum��widthr�height�rr�table�r�fields�tablesr	zParsed simplified schema: � pages, �
 OCR elements)�get�lenr�logger�info)r�page_dimensions�page�page_num�validation_data�	ocr_elemss      rrr.s���O����w��+���8�8�I�s�?�';�a�'?�@��%)�X�X�g�q�%9�4�8�8�H�a�;P�$Q��!�,�#���w��3�29�3���h�h�w��*�d�h�h�x��6K�L�3�30���B�-�
�O�"�*�.�.��b�"A�B�I�31�K�K�,�S��-A�,B�(�3�y�>�JZ�Zg�h�i���6�6��32s�4/C9c33�\�URS05RS05RS/5n[U5VVs0sH-up#US-URSS5URSS54_M/ nnnUVs/sH(nURSS5URSS5S.PM* sn0S	/0S34.nURS05RS05RS/5n[U5n[R	S
[U5S[U5S35 XhU4$s snnfs snf)z)35Parse original DA engine output format.36r37�imagePreprocessingResultrrrrrrrr�	ocrResult�blockszParsed engine output: rr)r�	enumeraterr!r"r )	rr�idx�pr#r$r&�38ocr_blocksr's	         rrrPs<��	���|�R�(�	��
'��	,�	��W�b�	�39� ��&��&�F�C�	�a��!�%�%���#�Q�U�U�8�Q�%7�8�8�&����40����h�h�w��*�d�h�h�x��6K�L��41���B�-�
�O�����b�1�5�5�k�2�F�J�J�8�UW�X�J�!�*�-�I�42�K�K�(��_�)=�(>�h�s�9�~�FV�Vc�d�e���6�6��+��43s�4D#�</D)c��USnURS5nURS5nURS5nURS5n[R"U5nU(d[R	SU35 gUH�nURS5(dM[RRX(5n	[U	5n44[U455up�n
[R	S	[U5S4635 [U47UUUUUUS9n[SU35 U(d.[R"U	5 [R	S
U	35 [R"U5s $ g![ aWn[R#SUSU35 SSKnUR'5 U(d[R"W	5 SnA gSnAff=f)a048Execute post-processing using the input JSON.49 50Args:51    input_dict (dict):52        - 'input_dir' (str): Directory containing input JSON.53        - 'column_headers' (optional): Dict of column field names to header text.54        - 'anchor_column' (optional): Column name for row detection.55        - 'anchor_pattern' (optional): Regex pattern for anchor validation.56        - 'advanced_options' (optional): Dict with special logic options.57    standalone (bool): If True, do not delete input files.58 59Returns:60    str: Final validation_data JSON (stringified).61�	input_dir�column_headers�
anchor_column�anchor_pattern�advanced_optionszNo files found in directory: Nz.jsonzCreated z OCR TextRect objects)r2r3r4r5zvalidation_data: zRemoved processed file: zError processing file z: r)r�os�listdirr!r"�endswith�path�joinrrr r�print�remove�json�dumps�	Exception�error�	traceback�	print_exc)�62input_dict�63standaloner1r2r3r4r5�files�filer9rr&r'r#�erAs                r�post_process_datarHss��� �;�'�I� �^�^�$4�5�N��N�N�?�3�M��^�^�$4�5�N�!�~�~�&8�9���J�J�y�!�E�����3�I�;�?�@����(	��=�=��)�)���7�7�<�<�	�0�D�"�4��J�;\�\f�:g�7�O���K�K�(�3�y�>�"2�2G�H�I�8����-�+�-�!1��O�
�%�o�%6�7�8���	�	�$�����6�t�f�=�>��:�:�o�.�.�?�V���	��L�L�1�$��r�!��=�>�����!���	�	�$����	�s �E%�CE%�%64G�/AG�GTr1rDr2r3r4r5c�"�[UUUUUS.US9$)a�65Wrapper for running post-processing directly from a directory.66 67Args:68    input_dir (str): Folder path containing input JSON files.69    standalone (bool): If True, keeps input files.70    column_headers (dict): Optional column headers mapping.71    anchor_column (str): Optional anchor column name.72    anchor_pattern (str): Optional anchor pattern regex.73    advanced_options (dict): Optional advanced options for special logic.74 75Returns:76    str: JSON string of validation_data or None.77)r1r2r3r4r5�rD)rH)r1rDr2r3r4r5s      r�run_post_processingrK�s*��"��(�&�(�,���
�rc��[R"SS9nURS[SSS9 URSSS	S789 UR	5$)zParse command line arguments.z(Run the table extraction post-processor.)�descriptionz--input_dirTz&Directory containing input JSON files.)�type�required�helpz--standalone�79store_truez.Enable standalone mode (preserve input files).)�actionrP)�argparse�ArgumentParser�add_argument�str�80parse_args)�parsers rrWrW�se��
�
$�
$�>��F�����
��
5�	�������
=�������r�__main__z^C:\Users\...\Documents\...\Customer_Issue1\CS\CS-35329\ceva_post_processor\ceva_post_processorrJ)F)TNNNN)rSr=�loggingr6�utils.pp_utilsrr�custom_processorr�basicConfig�INFO�	getLogger�__name__r!rrrrHrV�bool�dictrKrWr1�rr�<module>rds������	�7�6����'�,�,�/S�T�	�	�	�8�	$���e�.7�D 7�FH�V<@�IM�LP��3��D��'+��CF��'*��EI��4�(�z��y�I��	�d�3�r