agentlans/en-document-topic-classification
English Document Topic Classification Dataset English-language web pages classified by document topic, designed to train robust text classifiers and provide ready-to-use data for specific web topics. Purpose: Train generalized document classifiers or extract clean, single-topic corpora for specific downstream tasks. Configurations: Each document topic is available in its own dedicated dataset configuration (e.g., HomeGardening, GamesRecreation). Splits: The All configuration… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/en-document-topic-classification.
English Document Topic Classification Dataset
English-language web pages classified by document topic, designed to train robust text classifiers and provide ready-to-use data for specific web topics.
- Purpose: Train generalized document classifiers or extract clean, single-topic corpora for specific downstream tasks.
- Configurations: Each document topic is available in its own dedicated dataset configuration (e.g.,
HomeGardening,GamesRecreation). - Splits: The
Allconfiguration contains every document topic combined, featuring a 10% stratified test split.
Curation & Filtering Pipeline
This dataset is derived from a subset of the first 1 million rows of `allenai/c4`, utilizing annotations from `agentlans/en-document-classification`.
Samples are included only when the nvidia_domain field matches weborganizer_topic according to the following mapping.
These 27 categories cover 62% of the samples from the original dataset.
<details> <summary>Click here for the table</summary>
</details>
Class Distribution (All Split)
Licensing
Distributed under the Open Data Commons Attribution License (ODC-BY), matching the licensing terms of upstream sources `allenai/c4` and `agentlans/en-document-classification`.