theluantran/cefr-bert-classifier

CEFR BERT Classifier

A fine-tuned RoBERTa-based transformer model for classifying English text by CEFR (Common European Framework of Reference for Languages) proficiency levels.

The source code to train this model can be found at: https://github.com/luantran/One-model-to-grade-them-all

Model Description

This model is part of an ensemble CEFR text classification system that combines multiple approaches to estimate language proficiency levels. The BERT/RoBERTa classifier leverages pre-trained transformer representations fine-tuned on CEFR-labeled data to capture deep contextual and linguistic patterns characteristic of different proficiency levels. The other models part of this ensemble are:

• —https://huggingface.co/theluantran/cefr-naive-bayes
• —https://huggingface.co/theluantran/cefr-doc2vec

Labels

• —A1: Beginner
• —A2: Elementary
• —B1: Intermediate
• —B2: Upper Intermediate
• —C1/C2: Advanced/Proficient

Model Details

• —Base Model: FacebookAI/xlm-roberta-base
• —Task: Multi-class text classification (5 classes)
• —Training Data: 100k samples

Performance

• —In-Domain Test Accuracy: 98.17%
• —In-Domain QWK: 0.9908
• —Out-of-Domain Test Accuracy: 25.43%
• —Out-of-Domain QWK: 0.3367

Usage

Using Transformers

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "theluantran/cefr-bert-classifier"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

text = "Your text here"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)

with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
    predicted_class = predictions.argmax().item()

label_map = {0: 'A1', 1: 'A2', 2: 'B1', 3: 'B2', 4: 'C1/C2'}
print(f"Predicted CEFR Level: {label_map[predicted_class]}")
print(f"Confidence: {predictions[0][predicted_class].item():.2%}")

Training Configuration

• —Epochs: 4
• —Batch Size: 16
• —Learning Rate: 2e-05
• —Max Length: 512
• —Weight Decay: 0.01

License

This model is released for research and educational purposes. The training data is proprietary and not included.