datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
starcoder_cleanedThis is starcoderdata, but with leading boilerplate text/license text removed, and with short sequences filtered out. It also removes the extra tags at the beginning of some of the files, like <reponame>.
starcoder2data-extras
StarCoder2 Extras
This is the dataset of extra sources (besides Stack v2 code data) used to train the StarCoder2 family of models. It contains the following subsets:
Kaggle (kaggle): Kaggle notebooks from Meta-Kaggle-Code dataset, converted to scripts and prefixed with information on the Kaggle datasets used in the notebook. The file headers have a similar format to Jupyter Structured but the code content is only one single script.
StackOverflow (stackoverflow): stackoverflow… See the full description on the dataset page: https://huggingface.co/datasets/bigcode/starcoder2data-extras.starcoder2data-extras
StarCoder2 Extras
This is the dataset of extra sources (besides Stack v2 code data) used to train the StarCoder2 family of models. It contains the following subsets:
Kaggle (kaggle): Kaggle notebooks from Meta-Kaggle-Code dataset, converted to scripts and prefixed with information on the Kaggle datasets used in the notebook. The file headers have a similar format to Jupyter Structured but the code content is only one single script.
StackOverflow (stackoverflow): stackoverflow… See the full description on the dataset page: https://huggingface.co/datasets/animeshjoshi0086/starcoder2data-extras.starcoderdata-python-edu
starcoderdata-python-edu
StarCoder Training Dataset Cleaned and Scored
Dataset Details
Dataset Description
This dataset is a filtered version of StarCoder Training Dataset
that has been scored with the python-edu-scorer.
Dataset Sources
Repository: https://huggingface.co/collections/HuggingFaceTB/smollm-6695016cad7167254ce15966
Paper: SmolLM - blazingly fast and remarkably powerful
Citation
@misc{allal2024SmolLM,
title={SmolLM… See the full description on the dataset page: https://huggingface.co/datasets/jon-tow/starcoderdata-python-edu.stack-v2-starcoder2-3bstarcoder-python-instruct
StarCoder-Python-Qwen-Instruct
Dataset Description
This dataset contains Python code samples paired with synthetically generated natural language instructions. It is designed for supervised fine-tuning of language models for code generation tasks. The dataset is derived from the Python subset of the bigcode/starcoderdata corpus, and the instructional text for each code sample was generated using the Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8 model.
Creation… See the full description on the dataset page: https://huggingface.co/datasets/OLMo-Coding/starcoder-python-instruct.starcoder_labeled
Dataset Card for "starcoder_labeled"
Starcoder data, with several popular languages selected, short sequences filtered out, then labeled based on learning quality (educational value) and code quality.
A good heuristic is to take anything with >.5 code quality and >.3 learning quality. But you may want to vary the thresholds by language, depending on your target task.
self-instruct-starcoder
Self-instruct-starcoder
Summary
Self-instruct-starcoder is a dataset that was generated by prompting starcoder to generate new instructions based on some human-written seed instructions.
The underlying process is explained in the paper self-instruct. This algorithm gave birth to famous machine generated
datasets such as Alpaca and Code Alpaca which are two datasets
obtained by prompting OpenAI text-davinci-003 engine.
Our approach
While our method is… See the full description on the dataset page: https://huggingface.co/datasets/codeparrot/self-instruct-starcoder.starcoder-numpy-pandasstarcoder_filtered
Dataset Card for "starcoder_filtered"
A version of the starcoder dataset filtered based on data quality. Data was labeled with a rater model, and low-ranking rows were removed.
starcoderdata-gpt2starcoderdata-python-edu-lang-score
Dataset Card for Starcoder Data with Python Education and Language Scores
Dataset Summary
The starcoderdata-python-edu-lang-score dataset contains the Python subset of the starcoderdata dataset. It augments the existing Python subset with features that assess the educational quality of code and classify the language of code comments. This dataset was created for high-quality Python education and language-based training, with a primary focus on facilitating models that can… See the full description on the dataset page: https://huggingface.co/datasets/JanSchTech/starcoderdata-python-edu-lang-score.starcoderdata-repoCombines the data from starcoderdata and removes any repos with <= 10 stars.
starcoder-chemistrystarcoder-apis-0starcoderdata-apisstarcoder-apis-2starcoderdatastarcoderdatastarcoder-apis-1common_starcoder
Common Starcoder dataset
This dataset is generated from bigcode/starcoderdata.
Total GPT2 Tokens: 4,649,163,171
Generation Process
We filtered the original dataset with common language: C, Cpp, Java, Python and JSON.
We removed some columns for mixing up with other dataset: "id", "max_stars_repo_path", "max_stars_repo_name"
After removing the irrelevant fields, we shuffle the dataset with random seed=42.
We filtered the data on "max_stars_count" > 300 and shuffle again.… See the full description on the dataset page: https://huggingface.co/datasets/skymizer/common_starcoder.starcoder2-documentation
Dataset Card
This dataset is the code documenation dataset used in StarCoder2 pre-training, and it is also part of the-stack-v2-train-extras descried in the paper.
Dataset Details
Overview
This dataset comprises a comprehensive collection of crawled documentation and code-related resources sourced from various package manager platforms and programming language documentation sites. It focuses on popular libraries, free programming books, and other relevant… See the full description on the dataset page: https://huggingface.co/datasets/SivilTaram/starcoder2-documentation.starcoder-python5b5b gpt2 tokens
starcoder
Starcoder Dataset (The Stack - Sub-sampled)
This dataset is derived from the "Starcoder" version of The Stack, a 6.4 TB dataset of permissively licensed source code in 384 programming languages.
This repository contains the data organized into subsets, one for each programming language or data type.
How to Use
You can load any language-specific subset of the data using the datasets library. You must specify the name parameter with the desired language.
For example… See the full description on the dataset page: https://huggingface.co/datasets/Sam-Shin/starcoder.stack-dedup-python-testgen-starcoder-filter-v2
MultiPL-T Python Sources
Citation
If you use this dataset we request that you cite our work:
@misc{cassano:multipl-t,
title={Knowledge Transfer from High-Resource to Low-Resource Programming Languages for Code LLMs},
author={Federico Cassano and John Gouwar and Francesca Lucchetti and Claire Schlesinger and Anders Freeman and Carolyn Jane Anderson and Molly Q Feldman and Michael Greenberg and Abhinav Jangda and Arjun Guha},
year={2024}… See the full description on the dataset page: https://huggingface.co/datasets/nuprl/stack-dedup-python-testgen-starcoder-filter-v2.starcoderdata_100star_py
Dataset Card for "starcoderdata_100star_py"
More Information needed
starcoderdata-samplejailbreaks_dataset_with_perplexity_bigcode_starcoder2-3b_bigcode_starcoder2-7bstarcoder-bpb-3kbigcode__starcoder2-3b-details
Dataset Card for Evaluation run of bigcode/starcoder2-3b
Dataset automatically created during the evaluation run of model bigcode/starcoder2-3b
The dataset is composed of 44 configuration(s), each one corresponding to one of the evaluated task.
The dataset has been created from 2 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results.
An additional… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard/bigcode__starcoder2-3b-details.