datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
lca-project-level-code-completion
🏟️ Long Code Arena (Project-level code completion)
This is the benchmark for Project-level code completion task as part of the 🏟️ Long Code Arena benchmark.
Each datapoint contains the file for completion, a list of lines to complete with their categories (see the categorization below), and a repository snapshot that can be used to build the context.
All the repositories are published under permissive licenses (MIT, Apache-2.0, BSD-3-Clause, and BSD-2-Clause). The datapoints can… See the full description on the dataset page: https://huggingface.co/datasets/JetBrains-Research/lca-project-level-code-completion.code_x_glue_cc_code_completion_token
Dataset Card for "code_x_glue_cc_code_completion_token"
Dataset Summary
CodeXGLUE CodeCompletion-token dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/CodeCompletion-token
Predict next code token given context of previous tokens. Models are evaluated by token level accuracy.
Code completion is a one of the most widely used features in software development through IDEs. An effective code completion tool could improve software… See the full description on the dataset page: https://huggingface.co/datasets/google/code_x_glue_cc_code_completion_token.code-code-CodeCompletion-TokenLevel-Python
Dataset is imported from CodeXGLUE and pre-processed using their script.
Where to find in Semeru:
The dataset can be found at /nfs/semeru/semeru_datasets/code_xglue/code-to-code/CodeCompletion-token/dataset/py150 in Semeru
CodeXGLUE -- Code Completion (token level)
Update 2021.07.30: We update the code completion dataset with literals normalized to avoid sensitive information.
Here is the introduction and pipeline for token level code completion task.… See the full description on the dataset page: https://huggingface.co/datasets/semeru/code-code-CodeCompletion-TokenLevel-Python.code_x_glue_cc_code_completion_line
Dataset Card for "code_x_glue_cc_code_completion_line"
Dataset Summary
CodeXGLUE CodeCompletion-line dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/CodeCompletion-line
Complete the unfinished line given previous context. Models are evaluated by exact match and edit similarity.
We propose line completion task to test model's ability to autocomplete a line. Majority code completion systems behave well in token level completion, but fail in… See the full description on the dataset page: https://huggingface.co/datasets/google/code_x_glue_cc_code_completion_line.code-code-CodeCompletion-TokenLevel-Java
Dataset is imported from CodeXGLUE and pre-processed using their script.
Where to find in Semeru:
The dataset can be found at /nfs/semeru/semeru_datasets/code_xglue/code-to-code/CodeCompletion-token/dataset/javaCorpus in Semeru
CodeXGLUE -- Code Completion (token level)
Update 2021.07.30: We update the code completion dataset with literals normalized to avoid sensitive information.
Here is the introduction and pipeline for token level code completion task.… See the full description on the dataset page: https://huggingface.co/datasets/semeru/code-code-CodeCompletion-TokenLevel-Java.project-level-code-completion
BenchName (Project-level code completion)
This is the benchmark for Project-level code completion task as part of the BenchName benchmark.
Each datapoint contains the file for completion, a list of lines to complete with their categories (see the categorization below), and a repository snapshot that can be used to build the context.
All the repositories are published under permissive licenses (MIT, Apache-2.0, BSD-3-Clause, and BSD-2-Clause). The datapoints can be removed upon… See the full description on the dataset page: https://huggingface.co/datasets/icmlbenchname/project-level-code-completion.benchname-project-level-code-completion
🥷 BenchName (Project-level code completion)
This is the benchmark for Project-level code completion task as part of the 🥷 BenchName benchmark.
Each datapoint contains the file for completion, a list of lines to complete with their categories (see the categorization below), and a repository snapshot that can be used to build the context.
All the repositories are published under permissive licenses (MIT, Apache-2.0, BSD-3-Clause, and BSD-2-Clause). The datapoints can be removed upon… See the full description on the dataset page: https://huggingface.co/datasets/anon-iclr-submission/benchname-project-level-code-completion.context-aware-fim-code-completionsrepo-code-completion
Repository Level Code Completion Dataset for Evaluation
This is a dataset of repository snapshots before a commit where a python file has been added. One needs to complete added file with given content of repository composed in different ways.
How to load the data
via load_dataset:
from datasets import load_dataset
data_files = # choose from the table below
dataset = load_dataset("jenyag/repo-code-completion", data_files=data_files, split="train")
Options… See the full description on the dataset page: https://huggingface.co/datasets/jenyag/repo-code-completion.code-completionpython_code_starcoder_completionCodeCompletioncode_completion_for_data_analysistokenized_code_x_glue_cc_code_completion_line_pythoncode-alpaca-eval-debug-completionscode-code-galeras-code-completion-from-docstring-3k-dedupedcode_x_glue_cc_code_completion_line_pythonCode-Completion-Evaluation-Dataset-Packagespringboot-code-completion-dataset
Spring Boot Code Completion Dataset
概述
这是一个专为 Spring Boot 框架代码补全任务设计的中文优先数据集。通过从 GitHub 上 90 个高质量开源 Spring Boot 项目中提取方法级、类级代码片段及配置文件，构建而成。数据集重点突出 Spring Boot 典型特征（注解驱动、依赖注入、配置绑定等），适用于参数高效微调（PEFT，如 LoRA/QLoRA）下的代码生成研究。
数据集统计
总样本数：81,085 条
训练集：64,868 条
验证集：8,108 条
测试集：8,109 条
优先级 2 样本数（含典型 Spring Boot 特征，如 @RestController、@Service、@Entity 等）：19,740 条
优先级 2 占比：24.34%
数据格式（Alpaca 风格）
每条数据为 JSON 对象，包含以下字段：
{
"instruction": "请在Spring… See the full description on the dataset page: https://huggingface.co/datasets/Fuleee/springboot-code-completion-dataset.code-hint-completionscode-hint-3-completionssynthetic2-rlvr-code-compressed_filtered_olmo_completions_filteredcode-hint-2-completionscode-completion-gsm8ksynthetic2-rlvr-code-compressed_filtered_olmo_completionssynthetic2-rlvr-code-compressed_completionscode-completion-tokenizedklear-code-rlvr_completionsopen-code-reasoning-rlvr-stdio_completionscode-completion-finetune