datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
quantum-physics-0.6-corpus
quantum-physics-0.6-corpus
Dataset Description
This is a domain-specific corpus created using ontology-guided filtering from FineWeb-Edu.
Dataset Creation
Source: HuggingFaceFW/fineweb-edu
Filtering Method: Semantic similarity to subdomain centroids (embedding-based)
Pipeline: Ontology-Guided Domain Corpus Builder
Dataset Structure
Each chunk contains:
text: The text content (256-512 tokens)
subdomain_id: Assigned subdomain
similarity_score:… See the full description on the dataset page: https://huggingface.co/datasets/konsman/quantum-physics-0.6-corpus.quantum-physics-0.6
quantum-physics-0.6
Dataset Description
This is a domain-specific corpus created using ontology-guided filtering from FineWeb-Edu.
Dataset Creation
Source: HuggingFaceFW/fineweb-edu
Filtering Method: Semantic similarity to subdomain centroids (embedding-based)
Pipeline: Ontology-Guided Domain Corpus Builder
Dataset Structure
Each chunk contains:
text: The text content (256-512 tokens)
subdomain_id: Assigned subdomain… See the full description on the dataset page: https://huggingface.co/datasets/konsman/quantum-physics-0.6.quantum-physics-0.5-corpus
quantum-physics-0.5-corpus
Dataset Description
This is a domain-specific corpus created using ontology-guided filtering from FineWeb-Edu.
Dataset Creation
Source: HuggingFaceFW/fineweb-edu
Filtering Method: Semantic similarity to subdomain centroids (embedding-based)
Pipeline: Ontology-Guided Domain Corpus Builder
Dataset Structure
Each chunk contains:
text: The text content (256-512 tokens)
subdomain_id: Assigned subdomain
similarity_score:… See the full description on the dataset page: https://huggingface.co/datasets/konsman/quantum-physics-0.5-corpus.quantum-physics-corpus
quantum-physics-corpus
Dataset Description
This is a domain-specific corpus created using ontology-guided filtering from FineWeb-Edu.
Dataset Creation
Source: HuggingFaceFW/fineweb-edu
Filtering Method: Semantic similarity to subdomain centroids (embedding-based)
Pipeline: Ontology-Guided Domain Corpus Builder
Dataset Structure
Each chunk contains:
text: The text content (256-512 tokens)
subdomain_id: Assigned subdomain
similarity_score: Cosine… See the full description on the dataset page: https://huggingface.co/datasets/konsman/quantum-physics-corpus.quantum-physics-corpus-rebel-ready
quantum-physics-corpus-rebel-ready
Dataset Description
Quality-filtered corpus using classifier-based filtering (Phase 6/7 standard).
Filtering Criteria
Standard thresholds (Phase 6/7):
domain_relevance >= 2 ("Clearly about quantum physics")
technical_depth >= 2 ("Advanced undergraduate/graduate level")
Dataset Structure
Each chunk contains:
text: The text content
domain_relevance: Quality score (0-3)
technical_depth: Quality score (0-3)… See the full description on the dataset page: https://huggingface.co/datasets/konsman/quantum-physics-corpus-rebel-ready.quantum-physics-corpus-rechunked