pranavKHF/code_vulnerability_detection
0
1�
2 ��i� � �\ � d Z ddlZddlmZ ddlmZmZ G d� dej � Zd� Z y)zQCodeT5 Vulnerability Detection model3Binary Classication Safe(0) vs Vulnerable(1)� N)�T5ForConditionalGeneration�RobertaTokenizerc �2 � � e Zd ZdZd� fd� Zdd�Zd� Z� xZS )�VulnerabilityCodeT5z(CodeT5 model for vulnerability detectionc 4 � �� t �| � � t j |� | _ | j j5 j }t j t j d� t j ||� t j � t j d� t j ||� � | _ || _
y )Ng�������?)�super�__init__r �from_pretrained�encoder_decoder�config�d_model�nn�6Sequential�Dropout�Linear�ReLU�7classifier�8num_labels)�self�9model_namer � hidden_size� __class__s ��1d:\Code Vulnerability Detector CLEAN\src\model.pyr z VulnerabilityCodeT5.__init__ s� �� �
���� 9� I� I�*� U�� � �*�*�1�1�9�9� � �-�-� �J�J�s�O� �I�I�k�;� /� �G�G�I� �J�J�s�O� �I�I�k�:� .� 10��� %��� c �� � | j j ||d�� }|j }|dd�ddd�f }| j |� }d}|� t j11 � } | ||� }|||d�S )z�12 Forward pass13 Args:14 input_ids : tokenized code [batch_size, seq_len]15 attention_mask : attention mask [batch_size, seq_len]16 labels: ground truth labels [batch_size]17 T)� input_ids�attention_mask� return_dictNr )�loss�logits�
hidden_states)r �encoder�last_hidden_stater r �CrossEntropyLoss)18r r r �labels�encoder_outputs� hidden_state�
pooled_outputr r �loss_fns19 r �forwardzVulnerabilityCodeT5.forward s� � � �.�.�6�6� �)� � 7� 20�� '�8�8� �$�Q��1�W�-�
� � � � �/�� �� � ��)�)�+�G��6�6�*�D� � � )�21� 22r c � � | j � t j � 5 | j ||� }t j |d d�� }t j23 |d�� }ddd� ||fS # 1 sw Y fS xY w)zMake Predictionsr � )�dimN)�eval�torch�no_gradr* �softmax�argmax)r r r �outputs�probs� predictionss r �predictzVulnerabilityCodeT5.predictA ss � � � � � �
�]�]�_� 5��l�l�9�n�=�G��M�M�'�(�"3��;�E� �,�,�u�!�4�K� 5�24 �E�!�!� 5�25 �E�!�!�s �AA5�5B)zSalesforce/codet5-base� �N)�__name__�26__module__� __qualname__�__doc__r r* r6 �
__classcell__)r s @r r r s �� �2�%�&!27�F"r r c �B � t d� | j � D � � S )zCount trainable parametersc 3 �V K � | ]! }|j s�|j � �� �# y �wr8 )�
requires_grad�numel)�.0�ps r � <genexpr>z#count_parameters.<locals>.<genexpr>M s � �� �H�Q����q�w�w�y�H�s �)�))�sum�28parameters)�models r �count_parametersrH K s � � �H�%�"2�"2�"4�H� H�Hr )29r< r/ �torch.nnr � transformersr r �Moduler rH � r r �<module>rM s. ��0�
� � E�A"�"�)�)� A"�FIr