datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
quantum-computing-2048-fineweb-sampleTest dataset for testing a data sampling and filtering script.
Configuration for filtering:
dataset: "HuggingFaceFW/fineweb-edu"
output_file: "quantum_computing_entries.jsonl"
state_file: "quantum_computing_dataset_state.json"
# Processing parameters
keywords:
- "quantum computing"
- "qubit"
- "quantum entanglement"
- "quantum supremacy"
- "quantum algorithm"
- "quantum error correction"
max_entries: 1024
min_tokens: 1024
max_tokens: 2048
min_int_score: 4
# Shuffling… See the full description on the dataset page: https://huggingface.co/datasets/Lambent/quantum-computing-2048-fineweb-sample.quantumcomputingecosystem_dataset