datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
fairness-pruning-pairs-en
Fairness Pruning Prompt Pairs — English
Prompt pair dataset for neuronal bias mapping in Large Language Models. Designed to identify which MLP neurons encode demographic bias through differential activation analysis.
This dataset is part of the Fairness Pruning research project, which investigates bias mitigation through activation-guided MLP width pruning in LLMs.
Dataset Summary
Each record contains a pair of prompts that are identical except for a single… See the full description on the dataset page: https://huggingface.co/datasets/oopere/fairness-pruning-pairs-en.fairness-pruning-pairs-es
Fairness Pruning Prompt Pairs — Spanish
Prompt pair dataset for neuronal bias mapping in Large Language Models. Designed to identify which MLP neurons encode demographic bias through differential activation analysis, with a focus on Spanish-language bias patterns.
This dataset is part of the Fairness Pruning research project, which investigates bias mitigation through activation-guided MLP width pruning in LLMs. It is the Spanish companion to the English dataset, enabling… See the full description on the dataset page: https://huggingface.co/datasets/oopere/fairness-pruning-pairs-es.bge-retrieval-data-ivf-passage-pruning-438Kbge-retrieval-data-ivf-pruning-438Kbge-retrieval-data-ivf-cluster-pruning-438Kstablebridge-pruning-eval
Stablebridge Pruning Evaluation Dataset
Evaluation dataset for the Stablebridge context pruner/highlighter model, measuring sentence-level pruning quality on US stablecoin regulatory documents.
Dataset Structure
File
Records
Description
queries.jsonl
93
Regulatory queries (JSONL with _id and text fields)
corpus.jsonl
38
US stablecoin regulatory documents (full text)
qrels/test.tsv
2,704
Query-document relevance judgments
pruning_labels/test.jsonl
10,006… See the full description on the dataset page: https://huggingface.co/datasets/sugiv/stablebridge-pruning-eval.fr-wiki-popular-200-tokenizer-pruning
French Wikipedia corpus for tokenizer pruning
Prepared by Daniil Koblov. Contains 200 complete plain-text article extracts:
180 training articles and 20 held-out articles, split with Python's random seed 1337.
Candidates come from the 2025 monthly French Wikipedia top-1000 pageview lists.
Articles must appear in at least three months. Ranking uses month recurrence,
then the sum of reciprocal monthly ranks. The first 200 qualifying articles are
shuffled and split. Non-article… See the full description on the dataset page: https://huggingface.co/datasets/dakoblov/fr-wiki-popular-200-tokenizer-pruning.bge-retrieval-data-ivf-query-pruning-fixed-438Kbge-retrieval-data-ivf-cluster-pruning-200Kbge-retrieval-data-ivf-pruning-200Kbge-retrieval-data-ivf-passage-pruning-100Kbge-retrieval-data-ivf-query-pruning-fixed-200Kbge-retrieval-data-ivf-pruning-100Ktoken_pruningreasoning-pruning-pt-math-25-gemma4-r2reasoning-pruning-pt-bbh-logical-25-gemma4-r2bge-retrieval-data-ivf-pruning-50Ktruthy-dpo-v0.1-pruning-test-resultNot a serious or useable dataset, just used this https://github.com/Pleias/marginalia library and edited this notebook to check the quality of the selections in the jondurbin/truthy-dpo-v0.1 dataset
bge-retrieval-data-ivf-passage-pruning-200Kbge-retrieval-data-ivf-query-pruning-fixed-100Kbge-retrieval-data-ivf-passage-pruning-fixed-100KDataset-Pruningbge-retrieval-data-ivf-cluster-pruning-50Kbge-retrieval-data-ivf-passage-pruning-fixed-50Kbge-retrieval-data-ivf-passage-pruning-fixed-200Kreasoning-pruning-pt-aqua-rat-25-gemma4-r2bge-retrieval-data-ivf-passage-pruning-fixed-438Kstablebridge-regulatory-pruning-evallang-pruning-uk-uber-text-2pruning-dataset