cirimus/modernbert-large-bias-type-classifier

Overview

This model was fine-tuned from ModernBERT-large on a synthetic dataset of biased statements and questions, generated by Mistal 7B as part of the GUS-Net paper. The model is designed to identify and classify text bias into multiple categories, including racial, religious, gender, age, and other biases, making it a valuable tool for bias detection and mitigation in natural language processing tasks.

Model Details

• —Base Model: ModernBERT-large
• —Fine-Tuning Dataset: Synthetic biased corpus
• —Number of Labels: 11
• —Problem Type: Multi-label classification
• —Language: English
• —License: MIT
• —Fine-Tuning Framework: Hugging Face Transformers

Example Usage

Here’s how to use the model with Hugging Face Transformers:

from transformers import pipeline

# Load the model
classifier = pipeline(
    "text-classification",
    model="cirimus/modernbert-large-bias-type-classifier",
    return_all_scores=True
)

text = "Tall people are so clumsy."
predictions = classifier(text)

# Print predictions
for pred in sorted(predictions[0], key=lambda x: x['score'], reverse=True)[:5]:
    print(f"{pred['label']}: {pred['score']:.3f}")

# Output:
# physical: 1.000
# socioeconomic: 0.002
# gender: 0.002
# racial: 0.001
# age: 0.001

How the Model Was Created

The model was fine-tuned for bias detection using the following hyperparameters:

• —Learning Rate: 3e-5
• —Batch Size: 16
• —Weight Decay: 0.01
• —Warmup Steps: 500
• —Optimizer: AdamW
• —Evaluation Metrics: Precision, Recall, F1 Score (weighted), Accuracy

Dataset

The synthetic dataset consists of biased statements and questions generated by Mistal 7B as part of the GUS-Net paper. It covers 11 bias categories:

1. 1.Racial
2. 2.Religious
3. 3.Gender
4. 4.Age
5. 5.Nationality
6. 6.Sexuality
7. 7.Socioeconomic
8. 8.Educational
9. 9.Disability
10. 10.Political
11. 11.Physical

Evaluation Results

The model was evaluated on the synthetic dataset’s test split. The overall metrics using a threshold of 0.5 are as follows:

Macro Averages:

Per-Label Results:

Intended Use

The model is designed to detect and classify bias in text across 11 categories. It can be used in applications such as:

• —Content moderation
• —Bias analysis in research
• —Ethical AI development

Limitations and Biases

• —Synthetic Nature: The dataset consists of synthetic text, which may not fully represent real-world biases.
• —Category Overlap: Certain biases may overlap, leading to challenges in precise classification.
• —Domain-Specific Generalization: The model may not generalize well to domains outside the synthetic dataset’s scope.

Environmental Impact

• —Hardware Used: NVIDIA RTX4090
• —Training Time: ~2 hours
• —Carbon Emissions: ~0.08 kg CO2 (calculated via ML CO2 Impact Calculator).

Citation

If you use this model, please cite it as follows:

@inproceedings{JunquedeFortuny2025c,
  title = {Bias Detection with ModernBERT-Large},
  author = {Enric Junqué de Fortuny},
  year = {2025},
  howpublished = {\url{https://huggingface.co/cirimus/modernbert-large-bias-type-classifier}},
}