Team Ai
Apppublic

delsj/function-gemma

sourceHugging Faceupdated 10mo agoView on Hugging Face
0likes
imports.cpython-312.pyc161 linesDownload Raw Back to __pycache__
1�

2LIi�.����ddlZddlmZddlZddlmZmZddlm	Z						dde3ezde4de5dzde6dzd	e7dzd8edzdeddfd
�Z					dde9ezde10de11dzde12dzd	e13dzd14edzdeddfd�Z
y)�N)�Path)�deploy�utils)�
SQLiteStorage�csv_path�project�name�space_id�15dataset_id�private�force�returnc���tj|�rtd|�d���t|�}|j	�std|����t
j|�}|jrtd��tj|jj��}|j|��}d}	|jD]}16|17j�dk(s�|18}	n|	�td��|�|j}g}g}g}
g}|jD]:}||	k(r�	|d	k(r�	t
j ||d19��|j#|��<|j'�D]�\}}i}|D]4}||}t)t
j*|��s�'t-|�||<�6|s�D|j#|�|j#t/||	��d	|jvr?t)t
j*|d	��r|
j#t1|d	����|
j#d���|rtj2|||||
�
�t5dt7|��d|�d|�d|�d�	�t5ddj9|dj;�����tj<||�\}}|�!|t>j@d<t5d|���|�tjB|�ytEjF|||��tEjH|��tEjJ|||��t5dtDjLjO|�����y#tt$f$rY��cwxYw)a�	20    Imports a CSV file into a Trackio project. The CSV file must contain a `"step"`21    column, may optionally contain a `"timestamp"` column, and any other columns will be22    treated as metrics. It should also include a header row with the column names.23 24    TODO: call init() and return a Run object so that the user can continue to log metrics to it.25 26    Args:27        csv_path (`str` or `Path`):28            The str or Path to the CSV file to import.29        project (`str`):30            The name of the project to import the CSV file into. Must not be an existing31            project.32        name (`str`, *optional*):33            The name of the Run to import the CSV file into. If not provided, a default34            name will be generated.35        name (`str`, *optional*):36            The name of the run (if not provided, a default name will be generated).37        space_id (`str`, *optional*):38            If provided, the project will be logged to a Hugging Face Space instead of a39            local directory. Should be a complete Space name like `"username/reponame"`40            or `"orgname/reponame"`, or just `"reponame"` in which case the Space will41            be created in the currently-logged-in Hugging Face user's namespace. If the42            Space does not exist, it will be created. If the Space already exists, the43            project will be logged to it.44        dataset_id (`str`, *optional*):45            If provided, a persistent Hugging Face Dataset will be created and the46            metrics will be synced to it every 5 minutes. Should be a complete Dataset47            name like `"username/datasetname"` or `"orgname/datasetname"`, or just48            `"datasetname"` in which case the Dataset will be created in the49            currently-logged-in Hugging Face user's namespace. If the Dataset does not50            exist, it will be created. If the Dataset already exists, the project will51            be appended to it. If not provided, the metrics will be logged to a local52            SQLite database, unless a `space_id` is provided, in which case a Dataset53            will be automatically created with the same name as the Space but with the54            `"_dataset"` suffix.55        private (`bool`, *optional*):56            Whether to make the Space private. If None (default), the repo will be57            public unless the organization's default is private. This value is ignored58            if the repo already exists.59    �	Project 'z:' already exists. Cannot import CSV into existing project.zCSV file not found: zCSV file is empty)�columnsN�stepz/CSV file must contain a 'step' or 'Step' column�	timestamp�raise)�errors��r�run�metrics_list�steps�60timestamps�* Imported z rows from z into project '�61' as run '�'z* Metrics found: �, r�TRACKIO_DATASET_ID�:* Trackio metrics will be synced to Hugging Face Dataset: )r62rr�r63)rr64r
�* View dashboard by going to: )(r�get_runs�65ValueErrorr�exists�FileNotFoundError�pd�read_csv�emptyr�simplify_column_namesr�tolist�rename�lower�stem�66to_numeric�append�	TypeError�iterrows�bool�notna�float�int�str�bulk_log�print�len�join�keys� preprocess_space_and_dataset_ids�os�environ�print_dashboard_instructionsr�create_space_if_not_exists�wait_until_space_exists�upload_db_to_space�	SPACE_URL�format)rrr	r67rrr
�df�column_mapping�step_column�colrrr�numeric_columns�column�_�row�metrics�values                    �:/usr/local/lib/python3.12/dist-packages/trackio/imports.py�68import_csvrR69sK��d���g�&����y� Z�[�70�	71��H�~�H��?�?���"6�x�j� A�B�B�	���X�	�B�	�x�x��,�-�-��0�0����1B�1B�1D�E�N�	���>��	*�B��K��z�z���9�9�;�&� ��K���72���J�K�K��|��}�}���L��E��J��O��*�*���[� ���[� ��	��M�M�"�V�*�W�5��"�"�6�*���+�+�-���3���%�F���K�E��B�H�H�U�O�$�"'��,����&�73�����(��L�L��S��-�.�/��b�j�j�(�T�"�(�(�3�{�;K�2L�-M��!�!�#�c�+�&6�"7�8��!�!�"�%� � ������%��!�	74�75�76�c�,�'�(��H�:�_�W�I�U_�`d�_e�ef�g��77��d�i�i��Q��(<�(<�(>�?�@�78A�B� �A�A�(�J�W��H�j���+5��79�80�'�(�
�J�:�,�W�X���
�*�*�7�3��)�)��*�g�	81�	�&�&��9��!�!�'�H�E�R�
�,�V�-=�-=�-D�-D�h�-D�-W�,X�Y�	82��]�I�&�	��	�s�"+M#�#M6�5M6�log_dirc83���	ddlm}tj|�rtd|�d���t
|�}|j�std|����|t|�dh��}	|	j}84|85jrtd	|����d}g}|86jd�D�]�\}
}	|
d87k(rd}n|
}|r|�d|��}|jrtd
|
�d���7g}g}g}|j�D]�\}}t|d�}t|d�}t!|d�}||i}|j#|�|j#|�d|j$vr?t't)j*|d��s|j#t|d����|j#d88���|r�tj,|t|�|||��|t/|�z
}|j#|�tdt/|��d|
�d|�d��tddj1t3|d��������|std��td|���tddj1|����t7j8||�\}}|�!|t:j<d <td!|���|�t7j>|�ytAjB|||�"�tAjD|�tAjF|||�#�td$t@jHjK|�%����y#t$rtd��wxYw#t4$r}td|
�d|���Yd}~���d}~wwxYw)&a�89    Imports TensorFlow Events files from a directory into a Trackio project. Each90    subdirectory in the log directory will be imported as a separate run.91 92    Args:93        log_dir (`str` or `Path`):94            The str or Path to the directory containing TensorFlow Events files.95        project (`str`):96            The name of the project to import the TensorFlow Events files into. Must not97            be an existing project.98        name (`str`, *optional*):99            The name prefix for runs (if not provided, will use directory names). Each100            subdirectory will create a separate run.101        space_id (`str`, *optional*):102            If provided, the project will be logged to a Hugging Face Space instead of a103            local directory. Should be a complete Space name like `"username/reponame"`104            or `"orgname/reponame"`, or just `"reponame"` in which case the Space will105            be created in the currently-logged-in Hugging Face user's namespace. If the106            Space does not exist, it will be created. If the Space already exists, the107            project will be logged to it.108        dataset_id (`str`, *optional*):109            If provided, a persistent Hugging Face Dataset will be created and the110            metrics will be synced to it every 5 minutes. Should be a complete Dataset111            name like `"username/datasetname"` or `"orgname/datasetname"`, or just112            `"datasetname"` in which case the Dataset will be created in the113            currently-logged-in Hugging Face user's namespace. If the Dataset does not114            exist, it will be created. If the Dataset already exists, the project will115            be appended to it. If not provided, the metrics will be logged to a local116            SQLite database, unless a `space_id` is provided, in which case a Dataset117            will be automatically created with the same name as the Space but with the118            `"_dataset"` suffix.119        private (`bool`, *optional*):120            Whether to make the Space private. If None (default), the repo will be121            public unless the organization's default is private. This value is ignored122            if the repo already exists.123    r)�
SummaryReaderz�The `tbparse` package is not installed but is required for `import_tf_events`. Please install trackio with the `tensorboard` extra: `pip install trackio[tensorboard]`.rz@' already exists. Cannot import TF events into existing project.zTF events directory not found: �dir_name)�
extra_columnsz#No TensorFlow events data found in r�mainrMz* Skipping directory z: no scalar data found�tagrPr�	wall_timerrz scalar events from directory 'rrz* Metrics in this run: rz* Error processing directory z: Nz1No valid TensorFlow events data could be importedz* Total imported events: z* Created runs: r r!)rr)r
r#r")&�tbparserU�ImportErrorrr$r%rr&r'r8�scalarsr*�groupbyr:r3r6r7r1rr4r(�isnar9r;r<�set�	Exceptionrr>r?r@rArrBrCrDrErF)rSrr	r124rrr
rU�path�readerrG�total_imported�
imported_runsrV�group_df�run_namerrrrMrNrYrPrrO�es                          rQ�import_tf_eventsri�s���Z125�)����g�&����y� `�a�126�	127���=�D��;�;�=��"A�$�� H�I�I��3�t�9�Z�L�
A�F�	���B�	�x�x��>�t�f�E�F�F��N��M�!�j�j��4���(�7	��2�~�!��#���"�V�1�X�J�/���~�~��-�h�Z�7M�N�O���L��E��J�"�+�+�-���3��#�e�*�o���c�'�l�+���3�v�;�'����,���#�#�G�,����T�"��(�"2�"2�2�4��G�G�C��,�-�<��%�%�c�#�k�*:�&;�<��%�%�b�)�!.�$��&�&�#��H�
�!-��)���#�l�"3�3���$�$�X�.��!�#�l�"3�!4�4S�T\�S]�]g�hp�gq�qr�s���/��	�	�#�h�u�o�:N�0O�/P�Q�R��i5�t��L�M�M�	�%�n�%5�1286�7�	��T�Y�Y�}�5�6�1297�8� �A�A�(�J�W��H�j���+5��130�131�'�(�
�J�:�,�W�X���
�*�*�7�3��)�)���W�	132�	�&�&�x�0��!�!�'�8�5�A�
�,�V�-=�-=�-D�-D�h�-D�-W�,X�Y�	133��M�134��
v�135�	136�137��^�	��1�(��2�a�S�A�B���	�s*�L3�,.M�EM�3M�	M0�M+�+M0)NNNNF)r?�pathlibr�pandasr(�trackiorr�trackio.sqlite_storagerr8r4rRri��rQ�<module>rps��	���!�0���!���L138��D�j�L139�
�L140���*�L141��D�j�	L142�143�d�144�L145��D�[�
L146��L147�148�L149�d��!���W150�
�4�Z�W151�
�W152���*�W153��D�j�	W154�155�d�156�W157��D�[�
W158��W159�160�W161ro