datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
DreamBank-annotated
Presentation
DreamBank, an open corpus of more than 27,000 dream narratives, mostly written in English.
Annotations were produced using dream-t5, a LaMini-Flan-T5 model finetuned on Hall and Van de Castle annotations to predict character and emotion. I've introduced this task in this paper:
Gustave Cortal. 2024. Sequence-to-Sequence Language Models for Character and Emotion Detection in Dream Narratives. In Proceedings of the 2024 Joint International Conference on Computational… See the full description on the dataset page: https://huggingface.co/datasets/gustavecortal/DreamBank-annotated.open-thoughts-4-30k-code-qwen3-32b-annotated-32768-tokens
Dataset Card for Open-Thoughts-4-30K-Code-Qwen3-32B-Annotated-32768-Tokens
Overview
This dataset is a variant of marin-community/open-thoughts-4-30k-code-qwen3-32b-annotated with an extended maximum sequence length. The responses in the generated_text column were generated with max output tokens = 32768 (instead of 7500 in the original dataset), allowing for longer and more complete chain-of-thought reasoning.
Generation Details
Model: Qwen/Qwen3-32B… See the full description on the dataset page: https://huggingface.co/datasets/marin-community/open-thoughts-4-30k-code-qwen3-32b-annotated-32768-tokens.Magpie-Tanuki-8B-annotated-96k
Magpie-Tanuki-8B-annotated-96k
Magpieの手法をweblab-GENIAC/Tanuki-8B-dpo-v1.0に対して適用し作成したデータセットであるAratako/Magpie-Tanuki-8B-97kに対して、cyberagent/calm3-22b-chatを用いてinstructionに対して難易度、クオリティ、カテゴリをアノテーションしたデータセットです。
アノテーションのプロンプト
calm3によるアノテーションにはそれぞれ以下のプロンプトを利用しました。
難易度のアノテーション
# 指示
まず、与えられたユーザーの意図を特定し、その後、ユーザーのクエリの内容に基づいて難易度レベルをラベル付けしてください。
## ユーザーのクエリ
```
{input}
```
## 出力フォーマット
ユーザーのクエリに基づき、まずユーザーの意図を特定し、そのクエリを解決するために必要な知識を明示してください。
その後、難易度レベルを `very… See the full description on the dataset page: https://huggingface.co/datasets/Aratako/Magpie-Tanuki-8B-annotated-96k.Crab-manually-annotated-role-playing-evaluation-dataset
📄 Paper
|
📄 Github
💬 Role-playing Model
|
💬 Role-palying Evaluation Model
💬 Training Dataset
|
💬 Evaluation Benchmark
|
💬 Annotated Role-playing Evaluation Dataset
|
💬 Human-preference Dataset
1. Introduction
This is the dataset for fine-tuning a evaluator for roly-playing tasks. The fine-tuned evaluator can be… See the full description on the dataset page: https://huggingface.co/datasets/HeAAAAA/Crab-manually-annotated-role-playing-evaluation-dataset.the-stack-v2-dedup-sql-annotateThis dataset focuses the SQL corpus of the Stack v2 dedup dataset and provides annotations to each SQL file therein.
sqlparse is used to parse the SQL code, then count keywords and symbols.
Below are the annotation columns.
Column Name
Column Description
Keyword.DML
Data Manipulation Language commands for modifying database records, e.g. SELECT, INSERT, UPDATE, DELETE, COMMIT, MERGE, ROLLBACK.
Keyword.DDL
Data Definition Language commands for defining or altering database… See the full description on the dataset page: https://huggingface.co/datasets/onekq-ai/the-stack-v2-dedup-sql-annotate.annotated-wiki-2016
Annotated Wikipedia 2016
English Wikipedia (~2016 snapshot) processed into JSON, with inline hyperlinks preserved as gold-aligned entity-link annotations. Each article carries its full plain text plus a list of (surface_form, target_uri, character_offset) tuples — one per wikilink in the source.
This release just re-shards the original ~5 GB stored zip (extracted/AA/wiki00 … extracted/NN/wiki47, 35,148 JSONL chunk files) into parquet with a unified pyarrow schema. No filtering, no… See the full description on the dataset page: https://huggingface.co/datasets/alvations/annotated-wiki-2016.annotated-isabelleExtensively annotated Isabelle source code, suitable for pretraining, about 500M tokens in Qwen3's tokenizer.
Data source: Isabelle/HOL + AFP 2025-02-12. Cases from the PISA benchmark are removed.
Example:
lemma (in group) diff_neutralizes: ✐‹contributor ‹Paulo EmÃlio de Vilhena››
assumes "subgroup H G" "R ∈ rcosets H"
shows "⋀r1 r2. ⟦ r1 ∈ R; r2 ∈ R ⟧ ⟹ r1 ⊗ (inv r2) ∈ H"
proof (-)
(*goal: ‹⋀r1 r2. ⟦r1 ∈ R; r2 ∈ R⟧ ⟹ r1 ⊗ inv r2 ∈ H›*)
fix r1 and r2
assume r1: "r1 ∈ R" and r2: "r2… See the full description on the dataset page: https://huggingface.co/datasets/ANTPG/annotated-isabelle.OpenMath-Difficulty-Annotated
📐 OpenMath-Difficulty-Annotated
🚀 Overview
OpenMath-Difficulty-Annotated is a curated subset of OpenMathInstruct-2 containing 10,176 math problems, enhanced with precise difficulty metadata.
While the original solutions are preserved from NVIDIA's dataset, we employed a 120B Parameter Model (LLM-as-a-Judge) to analyze and grade every single problem on a scale of 1 to 5.
This allows developers of Small Language Models (1B-3B) to filter out "Olympiad-level" noise… See the full description on the dataset page: https://huggingface.co/datasets/HAD653/OpenMath-Difficulty-Annotated.FEDERICO-GARCIA-LORCA-canciones-poemas-romances-annotated
Federico García Lorca - Annotated Poetry Dataset
A curated and annotated dataset of 283 poems by Federico García Lorca, spanning 9 of his major works (1921--1940). Each poem is enriched with publication metadata and GPT-4-generated thematic and contextual annotations.
Use Case: LLM Generalization Evaluation
This dataset was created to evaluate how well large language models can generalize literary style from a small, domain-specific corpus. It has been used to fine-tune… See the full description on the dataset page: https://huggingface.co/datasets/xaviviro/FEDERICO-GARCIA-LORCA-canciones-poemas-romances-annotated.OpenDebateEvidence-Annotated-Anonymized
OpenDebateEvidence-Annotated (Anonymized)
An LLM-annotated subset of OpenDebateEvidence debate evidence, with all
debater-identifying columns removed.
This is an anonymized, Parquet-converted redistribution of
Hellisotherpeople/OpenDebateEvidence-Annotated.
85,600 rows, 44 columns: 25 annotation fields plus the 19 retained OpenDebateEvidence
evidence fields. The original pipe-delimited CSV had 70 columns, of which 26 were
dropped. See Anonymization.
Companion datasets:… See the full description on the dataset page: https://huggingface.co/datasets/Hellisotherpeople/OpenDebateEvidence-Annotated-Anonymized.PPe_detection_annotated
![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/66bc95f284
SFT_DATA-openthoughts-1k_rows-baseline-QwQ-AnnotatedYou can train using these datasets with LLaMA-Factory if you add this to your data/datasets.json files.
"example_dataset": {
"hf_hub_url": "SkillFactory/SFT_DATA-openthoughts-1k_rows-baseline-QwQ-Annotated",
"formatting": "sharegpt",
"columns": {
"messages": "conversations"
},
"tags": {
"user_tag": "user",
"assistant_tag": "assistant",
"role_tag": "role",
"content_tag": "content"
},
"subset": "sft_train"
}
applescript-lines-annotated
Dataset Card for "applescript-lines-annotated"
Description
This is a dataset of single lines of AppleScript code scraped from GitHub and GitHub Gist and manually annotated with descriptions, intents, prompts, and other metadata.
Content
Each row contains 8 features:
text - The raw text of the AppleScript code.
source - The name of the file from which the line originates.
type - Either compiled (files using the .scpt extension) or uncompiled (everything else).… See the full description on the dataset page: https://huggingface.co/datasets/HelloImSteven/applescript-lines-annotated.SFT_DATA-openthoughts-10k_rows-baseline-QwQ-AnnotatedYou can train using these datasets with LLaMA-Factory if you add this to your data/datasets.json files.
"example_dataset": {
"hf_hub_url": "SkillFactory/SFT_DATA-openthoughts-10k_rows-baseline-QwQ-Annotated",
"formatting": "sharegpt",
"columns": {
"messages": "conversations"},
"tags": {
"user_tag": "user",
"assistant_tag": "assistant",
"role_tag": "role",
"content_tag": "content"
},
"subset": "sft_train"
}
Syntactic-Semantic-Annotated-Italian-Corpus
Annotazione Sintattico-Funzionale e Disambiguazione della Lingua Italiana
This dataset was generated by fetching random first paragraphs from Italian Wikipedia (it.wikipedia.org)
and then processing them using Gemini AI with the following goal:
Processing Goal: riduci la ambiguità aggiungi tag grammaticali (soggetto) (verbo) eccetera. e tag funzionali es. (indica dove è nato il soggetto) (indica che il soggetto possiede l'oggetto) eccetera
Source Language: Italian (from Wikipedia)… See the full description on the dataset page: https://huggingface.co/datasets/Dddixyy/Syntactic-Semantic-Annotated-Italian-Corpus.SFT_DATA-cd3args-baseline-R1-AnnotatedYou can train using these datasets with LLaMA-Factory if you add this to your data/datasets.json files.
"example_dataset": {
"hf_hub_url": "SkillFactory/SFT_DATA-cd3args-baseline-Qwen2.5-1.5B-Instruct-R1",
"formatting": "sharegpt",
"columns": {
"messages": "conversations"
},
"tags": {
"user_tag": "user",
"assistant_tag": "assistant",
"role_tag": "role",
"content_tag": "content"
},
"subset": "sft_train"
}
opencode_reasoning2_hard_codeforces2000_pr03_qwen35_fp8_thinking_annotated_10k_seed20260513
Qwen3.5 FP8 Annotations for 10K K2-Think OCR2 Coding Steps
This dataset contains Qwen3.5 FP8 step-level correctness annotations for K2-Think reasoning traces on a hard Codeforces subset of OpenCodeReasoning-2.
Summary
Source trace dataset: opencode_reasoning2_hard_codeforces2000_pr03_k2_thinking_extracted_pilot10
Source rows: 10 hard coding problem traces
Candidate step rule: claim with non-empty aligned_token_ids
Candidate steps: 15,267
Manifest-selected annotated… See the full description on the dataset page: https://huggingface.co/datasets/JingweiNi/opencode_reasoning2_hard_codeforces2000_pr03_qwen35_fp8_thinking_annotated_10k_seed20260513.