datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
cornstack-python-v1
CoRNStack Python Dataset
The CoRNStack Dataset, accepted to ICLR 2025, is a large-scale high quality training dataset specifically for code retrieval across multiple
programming languages. This dataset comprises of <query, positive, negative> triplets used to train nomic-embed-code,
CodeRankEmbed, and CodeRankLLM.
CoRNStack Dataset Curation
Starting with the deduplicated Stackv2, we create text-code pairs from function docstrings and respective code. We filtered out… See the full description on the dataset page: https://huggingface.co/datasets/nomic-ai/cornstack-python-v1.python-toolcallsLogs from run_python_code tool used for benchmarking.
python-codes-25k
License
MIT
This is a Cleaned Python Dataset Covering 25,000 Instructional Tasks
Overview
The dataset has 4 key features (fields): instruction, input, output, and text.It's a rich source for Python codes, tasks, and extends into behavioral aspects.
Dataset Statistics
Total Entries: 24,813
Unique Instructions: 24,580
Unique Inputs: 3,666
Unique Outputs: 24,581
Unique Texts: 24,813
Average Tokens per example: 508
Features… See the full description on the dataset page: https://huggingface.co/datasets/flytech/python-codes-25k.Tested-143k-Python-AlpacaContributors: Nicolas Mejia Petit
Vezora's CodeTester Dataset
Introduction
Today, on March 6, 2024, we are excited to release our internal Python dataset with 143,327 examples of code. These examples have been meticulously tested and verified as working. Our dataset was created using a script we developed.
Dataset Creation
Our script operates by extracting Python code from the output section of Alpaca-formatted datasets. It tests each extracted piece of code… See the full description on the dataset page: https://huggingface.co/datasets/Vezora/Tested-143k-Python-Alpaca.github-repos-pythonThe Github repository retrieval source for [code-rag-bench], containing all Python files from the entire GitHub dump (in github-repos)
starcoder-python-instruct
StarCoder-Python-Qwen-Instruct
Dataset Description
This dataset contains Python code samples paired with synthetically generated natural language instructions. It is designed for supervised fine-tuning of language models for code generation tasks. The dataset is derived from the Python subset of the bigcode/starcoderdata corpus, and the instructional text for each code sample was generated using the Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8 model.
Creation… See the full description on the dataset page: https://huggingface.co/datasets/OLMo-Coding/starcoder-python-instruct.python_functions_reasoningThis is the Python (functions) coding reasoning dataset used to train
Notbad v1.0 Mistral 24B reasoning model.
The reasoning data were sampled from an RL-based self-improved
Mistral-Small-24B-Instruct-2501 model.
The Python functions and instructions were sourced from OpenCoder Dataset Stage1
and from open source projects on Github.
You can try Notbad v1.0 Mistral 24B on chat.labml.ai.
floorplans-cityscapes
Dataset Summary
This is a curated collection of floorplan images sourced from across the internet. It is intended for research in architectural AI, layout generation, and urban scene understanding.
Data format: Image files with associated integer labels.
Sources: Publicly available images from various web sources (This dataset is one unified collections).
Purpose: Educational and research use.
Dataset Structure
The dataset follows the standard Hugging Face Image… See the full description on the dataset page: https://huggingface.co/datasets/wheres-my-python/floorplans-cityscapes.Tested-22k-Python-AlpacaContributors: Nicolas Mejia Petit
Vezora's CodeTester Dataset
Introduction
Today, on November 2, 2023, we are excited to release our internal Python dataset with 22,600 examples of code. These examples have been meticulously tested and verified as working. Our dataset was created using a script we developed.
Dataset Creation
Our script operates by extracting Python code from the output section of Alpaca-formatted datasets. It tests each extracted piece of… See the full description on the dataset page: https://huggingface.co/datasets/Vezora/Tested-22k-Python-Alpaca.Python-Code-23k-ShareGPTThis dataset is in Vicuna/ShareGPT format. There are 23000+ set of conversations. Each set having 2 conversations.
Along with the Python code detailed explanation is provided.
This dataset was generated using GPT-3.5, GPT-4 etc.
Python-Code-LargePython-Code-Large
Python-Code-Large is a large-scale corpus of Python source code comprising more than 2 million rows of Python code. The dataset is designed to support research in large language model (LLM) pretraining, code intelligence, software engineering automation, and program analysis for the Python ecosystem.
By providing a high-volume, language-specific corpus, Python-Code-Large enables systematic experimentation in Python-focused model training, domain adaptation, and downstream… See the full description on the dataset page: https://huggingface.co/datasets/Lovett01/Python-Code-Large.rl-pythonPython-Code-LargePython-Code-Large
Python-Code-Large is a large-scale corpus of Python source code comprising more than 2 million rows of Python code. The dataset is designed to support research in large language model (LLM) pretraining, code intelligence, software engineering automation, and program analysis for the Python ecosystem.
By providing a high-volume, language-specific corpus, Python-Code-Large enables systematic experimentation in Python-focused model training, domain adaptation, and downstream… See the full description on the dataset page: https://huggingface.co/datasets/ajibawa-2023/Python-Code-Large.python-mental-execution-traces
Python Mental Execution Traces
A 12,000-row prompt/completion dataset for evaluating and training language models to mentally execute self-contained Python 3 snippets without running them. Completions provide the expected standard output together with a concise variable trace or explanation.
Dataset structure
The JSONL file contains two text fields:
prompt: a Python mental-execution problem.
completion: the expected stdout and concise reasoning or variable trace.… See the full description on the dataset page: https://huggingface.co/datasets/ILoveBuns/python-mental-execution-traces.litgpt-the-stack-dedup-python-filtered-gpt2Competitive-Programming-python-blend
Dataset Card for Competitive-Programming-python-blend
Summary
Competitive-Programming-python-blend is a mixed supervised fine-tuning dataset centered on competitive programming, code reasoning, and instruction-style problem solving. The blend is Python-first, but it also keeps a small amount of C++, agentless SWE, and reasoning-oriented chat supervision to broaden training coverage.
The current release is published as a single HF-friendly JSONL file, clean.jsonl.… See the full description on the dataset page: https://huggingface.co/datasets/Jackrong/Competitive-Programming-python-blend.python-execution-trace-output-prediction-cmskdimp
Python Execution Trace & Output Prediction
Self-contained Python programs paired with a concrete function call and the exact runtime output. Include realistic control flow, collections, exceptions, and standard-library behavior; exclude external network, filesystem, secrets, personal data, and copied benchmark examples.
About
This dataset was produced by the DataBounty community and published here as part of an open, karma-only program.
Accepted items: 1000… See the full description on the dataset page: https://huggingface.co/datasets/databounty-io/python-execution-trace-output-prediction-cmskdimp.python-code-instructions-85k
Python Code Instructions - 85K
Instruction-tuning dataset of Python functions paired with short natural-language instructions derived from repository docstrings.
What changed in this release
This release keeps the original public rows and format, but makes the dataset easier to use responsibly:
exact duplicate rows were removed again using normalized instruction + output hashing
deterministic train, validation, and test splits were added
the dataset card now documents… See the full description on the dataset page: https://huggingface.co/datasets/NickIBrody/python-code-instructions-85k.ds-python-uppython-bugsadaption-python-mental-execution-traces
This dataset is a remastered version prepared using Adaption's Adaptive Data platform.
adaption-python_mental_execution_traces
This dataset contains pairs of Python 3 code snippets and their corresponding mental execution results, including exact standard output and concise variable traces. Each sample challenges the model to simulate code logic involving lists, matrices, loops, and conditional statements without actual execution. The completions provide both the final printed… See the full description on the dataset page: https://huggingface.co/datasets/ILoveBuns/adaption-python-mental-execution-traces.kodcode-verified-python-235k
KodCode-Verified Python — 234,555 execution-verified Python SFT rows
One row per problem. Every assistant turn is code that passed its own unit tests when
actually run — real pytest against KodCode-V1's
tests, in a pinned interpreter, in a sandboxed subprocess. No LLM judge, no heuristic
filter, no model-generated answers.
Unlike the v3 release this supersedes, the corpus is deduplicated, decontaminated against
HumanEval/MBPP, and stripped of rows whose tests cannot constrain… See the full description on the dataset page: https://huggingface.co/datasets/F-A-I-L/kodcode-verified-python-235k.python-unit-test-training-pool
Python unit test training pool
A pool of public data for training a model to write tests for Python code. It is a
straight collection of open datasets, not a new corpus: every row comes from one of the
sources below, at the revision named, and the only rows removed are the ones an overlap
filter flagged against held-out material this pool is kept separate from.
Every row of the normalised layer pairs a program with tests for it. That is the point of
the pool, and it is why the… See the full description on the dataset page: https://huggingface.co/datasets/Emulated-Inc/python-unit-test-training-pool.Python_Vulnerability_Remediation
Python SAST Vulnerability and Remediation Dataset
Summary
This dataset is a collection of Python code snippets containing common security vulnerabilities, paired with their corresponding high-quality remediations. It is designed for fine-tuning language models to assist with Static Analysis Security Testing (SAST) by suggesting secure code fixes.
The dataset is primarily focused on vulnerabilities from the following Common Weakness Enumerations (CWEs):
CWE-89 (SQL… See the full description on the dataset page: https://huggingface.co/datasets/cmonplz/Python_Vulnerability_Remediation.29K_Python_Docstring_Pairs
29K High-Quality Python Docstring Pairs
Author: Michael Hernandez (XxCotHGxX)License: CC BY 4.0Cleaned from: XxCotHGxX/242K_Python_Docstring_Pairs
Overview
A curated, high-quality subset of Python function–docstring pairs for use in code documentation generation, docstring completion, and code understanding tasks.
The original 242K dataset was scraped from open-source Python repositories but contained a significant proportion of functions without docstrings (84% of… See the full description on the dataset page: https://huggingface.co/datasets/XxCotHGxX/29K_Python_Docstring_Pairs.code-text-python
Dataset is imported from CodeXGLUE and pre-processed using their script.
Where to find in Semeru:
The dataset can be found at /nfs/semeru/semeru_datasets/code_xglue/code-to-text/python in Semeru
CodeXGLUE -- Code-To-Text
Task Definition
The task is to generate natural language comments for a code, and evaluted by smoothed bleu-4 score.
Dataset
The dataset we use comes from CodeSearchNet and we filter the dataset as the following:
Remove… See the full description on the dataset page: https://huggingface.co/datasets/semeru/code-text-python.python-functions-training-pool
Python function-writing training pool
A pool of public data for training a model to write Python functions. It is a straight
collection of open datasets, not a new corpus: every row comes from one of the sources
below, at the revision named, and the only rows removed are the ones an overlap filter
flagged against held-out material this pool is kept separate from.
Rows in the normalised layer: 5756045.
Rows in the raw layer: 6258415.
The two layers
pool/ holds the… See the full description on the dataset page: https://huggingface.co/datasets/Emulated-Inc/python-functions-training-pool.leetcode-python-dataset
leetcode-python-dataset
Code for building and publishing the justindal/leetcode-python-dataset dataset on Hugging Face.
Merges two open-source LeetCode datasets into a unified schema with consistent formatting, field normalisation, and solution validation.
Dataset
Split
Rows
Source
train
2856
newfacade + greengerong
valid
310
slug-group split from train
test
228
newfacade only
Schema
default config (training)
Each row is a… See the full description on the dataset page: https://huggingface.co/datasets/justindal/leetcode-python-dataset.python_codepython_evol_instruct_51k