SHUBHAMOS/meta-pytorch-hackathon
0
1�
2 T��i � 3 �� � S SK rS SKrS SKrS SKrSSKJr SSK J4r5 S SK J r S SK
Jr S r\R " SS 9 r\R$ " S65 \R$ " S 5 \R& " 5 \R( " SS 9 \R* " S
SSSS9r\R. " / SQSSS9r\R2 " SSS9r\R$ " S5 \R$ " S5 \R$ " S5 \R$ " S 5 \R$ " S 5 SSS5 \R( " S S 9 \R$ " S 5 r\R8 " S S!9r SSS5 SSS5 \R= \\\/\\ /S"9 SSS5 \R> " \\S#S$9rS% r \!S&:X a \ " 5 gg! , ( d f N�= f! , ( d f Nh= f! , ( d f Nq= f! , ( d f Ne= f)'� N� )�app)�TASKS)� run_agent)�CORSMiddlewarec � � U ( d( [ R R SS5 n U ( d g U [ R S' [ XSS9nSU; a7 SUS 3S4$ UR S 0 5 n/ S8QUR S S 5 S
UR SS 5 S
UR SS 5 S
UR SS 5 S
UR SS 5 S
/S.n[ R9 " U5 nSUR S5 ( a SOS SUR SS 5 S
3nXe4$ ! [ a nS[ U5 3S4s SnA$ SnAff = f)z.Bridge for Gradio to call the inference agent.�HF_TOKEN� )z+Error: Please provide a Hugging Face Token.NF)�hf_token�verbose�errorzError: N�scores)z Final ScorezClassification AccuracyzPriority AccuracyzResolution RatezUrgent Handling� final_scorer z.3f�classification_accuracy�priority_accuracy�resolution_rate�urgent_handling)�Metric�Valueu) ### Benchmark Complete! 🚀10**Status:** �passedu PASS ✅u FAIL ❌z11**Total Reward:** � total_rewardz
Fatal Error: )�os�environ�getr �pd� DataFrame� Exception�str)r �task_id�resultr �df_data�df�summary�es �
server/app.py�run_benchmark_uir& s� � � ��:�:�>�>�*�b�1���F� .�!)��12�13�:� � �7�u�E�� �f� � �V�G�_� -�.��4� 4� ���H�b�)��{��:�:�m�Q�/��4��:�:� 7��;�C�@��:�:� 1�1�5�c�:��:�:� /��3�C�8��:�:� /��3�C�8� � 14�� �\�\�'�
"��>�V�Z�Z�X`�Ma�Ma�z�gq�>r� sG� HN� HR� HR� Sa� cd� He� fi� Gj� k���{��� � .� �s�1�v�h�'��-�-��.�s$ �,D � CD � 15E�*D?�9E�?Ez'SHUBHAMOS: AI Email Triage Benchmarking)�titleu% # 📈 SHUBHAMOS Benchmark ControllerzJTest your AI agent's email triage capabilities using the OpenEnv protocol.)�scalezHugging Face Tokenzhf_...�passwordz;Required to call the AI Router (Qwen/Qwen2.5-72B-Instruct).)�label� placeholder�type�info)�easy�medium�hardr. zSimulation Task)�choices�valuer* u Run Agent Benchmark 🚀�primary)�variantz---z### API Endpointsz- [FastAPI Docs](/docs)z#- [Environment Spec](/openenv.yaml)z- [Dashboard](/dashboard)� z ### Results will appear here...zPerformance Metrics)r* )�fn�inputs�outputs�/)�pathc � � SSK n [ [ R R SS5 5 nU R SSUS9 g)z1CLI entry point for the OpenEnv 'server' command.r N�PORTi� zserver.app:appz0.0.0.0)�host�port)�uvicorn�intr r r �run)r? r> s r% � start_serverrB \ s1 � �� �r�z�z�~�~�f�d�+� ,�D� �K�K� �y�t�K�<� �__main__)"�gradio�grr �json�pandasr �serverr � fastapi_app�tasksr � inferencer �fastapi.middleware.corsr r& �Blocks�demo�Markdown�Row�Column�Textbox� token_input�Radio� task_select�Button�run_btn�result_summary� Dataframe� result_table�click�mount_gradio_apprB �__name__� rC r% �<module>r` s� �� � � � � &� � � 2�%.�R �Y�Y�>�?�4��K�K�7�8��K�K�\�]� ����
�Y�Y�Q�
� �*�*�*� $� �R� �K� �(�(�2� �'��K�16 �i�i� :�I�N�G� �K�K�� � �K�K�+� ,� �K�K�1� 2� �K�K�=� >� �K�K�3� 4�# �& �Y�Y�Q�
� �[�[�)J� K�N� �<�<�.C�D�L� �) 17�0 �M�M� ��[�)� � �.� � �9 @�J ���+�t�#�6��=� �z���N� �S �
��& �
��) 18��� @�?�sT �6G�2G�BF"�G�.#F3�G� G�"19F0 �,G�320G �=G�21G �G�22G#