dzungpham/graphcodebert-code-classification

1---2license: mit3metrics:4- accuracy5- f16- precision7- recall8base_model:9- microsoft/unixcoder-base10library_name: transformers11tags:12- detection13- AI-generated14- transformers15- bert16---17 18## Task Overview19 20The rapid advancement of generative models has made it increasingly challenging to distinguish machine-generated code from human-written code, particularly across different programming languages, domains, and generation techniques.21 22SemEval-2026 Task 13 focuses on developing systems capable of detecting machine-generated code under diverse conditions. The evaluation emphasizes generalization to unseen programming languages, generator families, and application scenarios.23 24The task is divided into three subtasks.25 26---27 28### Subtask A: Binary Machine-Generated Code Detection29 30**Goal:**  31Given a code snippet, determine whether it is:32 33- Fully human-written, or  34- Fully machine-generated35 36**Training Languages:** C++, Python, Java  37**Training Domain:** Algorithmic (e.g., LeetCode-style problems)38 39**Evaluation Settings:**40 41| Setting                              | Language                | Domain               |42|--------------------------------------|-------------------------|----------------------|43| (i) Seen Languages & Seen Domains    | C++, Python, Java       | Algorithmic          |44| (ii) Unseen Languages & Seen Domains | Go, PHP, C#, C, JS      | Algorithmic          |45| (iii) Seen Languages & Unseen Domains| C++, Python, Java       | Research, Production |46| (iv) Unseen Languages & Domains      | Go, PHP, C#, C, JS      | Research, Production |47 48**Dataset Size:** 49- Train: 500,000 samples (238,000 human-written, 262,000 machine-generated)50- Validation: 100,000 samples51 52**Data Format:**  53Each dataset includes the following fields:54- `code`: The code snippet  55- `label`: Binary label (0 for human-written, 1 for machine-generated)  56- `language`: Programming language of the snippet  57 58Label mappings are provided in `task_A/label_to_id.json` and `task_A/id_to_label.json`.59 60**Evaluation Metric:**  61The primary metric for Subtask A is Macro F1-score, ensuring balanced performance across both classes.62 63**Submission Format:**  64Participants must submit a `.csv` file containing:65- `id`: Unique identifier for each code snippet  66- `label`: Predicted label (0 or 1)  67 68A sample submission file is available in the `task_A/` directory.69 70**Baseline Models:**  71Baseline implementations are provided in the `baselines/` directory, including starter code and pre-trained checkpoints for models such as GraphCodeBERT and UniXcoder.72 73**Restrictions:**74- No external training data may be used; only the provided datasets are allowed.  75- Specialized AI-generated code detectors are not permitted. General-purpose code models (e.g., CodeBERT, StarCoder) are allowed.76