datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
text-code-galeras-code-generation-from-docstring-3k-dedupedthe-stack-smol-python-docstrings
Dataset Card for "the-stack-smol-filtered-python-docstrings"
More Information needed
the-stack-dedup-python-filtered-docstrings-gpt2code_docstring_corpusHF version of Edinburgh-NLP's Code docstrings corpus
29K_Python_Docstring_Pairs
29K High-Quality Python Docstring Pairs
Author: Michael Hernandez (XxCotHGxX)License: CC BY 4.0Cleaned from: XxCotHGxX/242K_Python_Docstring_Pairs
Overview
A curated, high-quality subset of Python function–docstring pairs for use in code documentation generation, docstring completion, and code understanding tasks.
The original 242K dataset was scraped from open-source Python repositories but contained a significant proportion of functions without docstrings (84% of… See the full description on the dataset page: https://huggingface.co/datasets/XxCotHGxX/29K_Python_Docstring_Pairs.the-stack-dedup-python-filtered-docstringsThis is a dataset originated from bigcode/the-stack-dedup with some filters applied.
The filters filtered in this dataset are:
remove_function_no_docstring
remove_class_no_docstring
remove_delete_markers
mined_docstrings_pypiStats:
total 352M
149228 drwxr-xr-x 2 micha micha 324K May 12 11:13 .
134775 drwxr-xr-x 4 micha micha 4.0K May 12 11:13 ..
168137 -rw-r--r-- 1 root root 6.2K May 12 11:06 2dwavesim.parquet
159971 -rw-r--r-- 1 root root 3.0K May 12 08:19 aad_token_verify.parquet
164105 -rw-r--r-- 1 root root 3.6K May 12 08:51 aadict.parquet
165435 -rw-r--r-- 1 root root 437 May 12 09:36 aats.parquet
166392 -rw-r--r-- 1 root root 437 May 12 10:13 abarms.parquet
158836 -rw-r--r-- 1 root root… See the full description on the dataset page: https://huggingface.co/datasets/michaelfeil/mined_docstrings_pypi.thevault-docstringstyle
Dataset Card for "thevault-docstringstyle"
More Information needed
arkts-code-docstring
Recommended Version: ArkTS-CodeSearch-Pro
We strongly recommend using ArkTS-CodeSearch-Pro for all new research and development.The Pro version is the actively maintained release and provides improved data quality through stricter open-source license filtering, repository quality control (stars > 0), comment normalization, removal of license/copyright headers and code-only comments, minimum docstring length filtering, and global code/query deduplication.
The original… See the full description on the dataset page: https://huggingface.co/datasets/hreyulog/arkts-code-docstring.the-stack-dedup-python-docstrings-1.0-percent-unified
Dataset Card for "the-stack-dedup-python-docstrings-1.0-percent-unified"
More Information needed
arkts-code-docstring-pro
ArkTS-CodeSearch-Pro: A Cleaned and License-Aware ArkTS Dataset
Original Dataset | Pro Dataset | Paper | Evaluation Code | Dataset Processing Code | Fine-Tuned Model
This dataset is an improved version of ArkTS-CodeSearch. It contains function-level information from ArkTS (.ets) projects, including function source code, docstrings, AST representations, and repository metadata. It is intended for code retrieval, code understanding, and AST-based research.
Compared with the… See the full description on the dataset page: https://huggingface.co/datasets/hreyulog/arkts-code-docstring-pro.python-docstring-human-gpt-generated-mixpython_code_docstring_ast_corpus
Overview
This dataset contains 34,000+ rows of code-docstring-ast data along with additional metadata. Data was gathered from various Python libraries and frameworks and their
publicly available GitHub repos. This dataset was created for the purpose of training the CodeT5+ transformer on AST-enhanced code-to-doc tasks.
Sources
The dataset was gathered from various GitHub repos sampled from this repo by Vinta.
The 26 repos are:
matplotlib
pytorch
cryptography
django… See the full description on the dataset page: https://huggingface.co/datasets/Mir-2002/python_code_docstring_ast_corpus.python-docstrings
Python Docstring Diff Dataset
This dataset contains training samples for models that generate Python documentation patches.
Each example provides a Python source file with its docstrings removed and a corresponding unified diff patch that restores the documentation.
The dataset is designed for training or evaluating language models that assist with:
Automatic code documentation
Docstring generation
Code review automation
Developer tooling
Dataset Structure
Each entry contains the… See the full description on the dataset page: https://huggingface.co/datasets/helloadhavan/python-docstrings.code-code-galeras-code-completion-from-docstring-3k-dedupedarkts-code-docstring-for-anonymous
ArkTS-CodeSearch-Pro: A Cleaned and License-Aware ArkTS Dataset
Evaluation Code |Dataset processing Code | Fine-Tuned Model
This dataset contains function-level information from ArkTS (.ets) projects, including function source code, normalized docstrings, AST representations, and repository metadata. It is intended for code retrieval, code understanding, and AST-based research.
Compared with the original release, this Pro version provides the following improvements:… See the full description on the dataset page: https://huggingface.co/datasets/hreyulog/arkts-code-docstring-for-anonymous.python_googlestyle_docstringsmined_docstrings_pypi_embeddedDataset embedded with jina-code-v2
stack-smol-docstrings
Stack-Smol-Docstrings
This dataset contains Python functions extracted from the-stack-smol,
filtered for high-quality docstrings and implementations. Each sample includes
the function's docstring, implementation, and a masked version of the code where the
function is replaced with a comment.
The dataset is designed for code completion tasks where a model needs to restore a
function that has been replaced with a comment. The model is provided with:
The full file context with the… See the full description on the dataset page: https://huggingface.co/datasets/ASSERT-KTH/stack-smol-docstrings.CodeXGLUE-Code-Docstringcode_docstringscode-docstring-datasetdocstring-chat-dsworkshop-code-docstringworkshop-ast-docstringworkshop-code-ast-docstringstack-v2-cpp-with-docstring-dataset_v1workshop-ast-compress-docstringjuraj-juraj-python-docstring-human-gpt-generated-mixstyle_complexity_docstring_2300