Ian-Khalzov/article-topic-service-scibert

Article Topic Service SciBERT

SciBERT text classifier for scientific article topic prediction from article title and abstract.

Labels

• —Artificial Intelligence
• —Natural Language Processing
• —Computer Vision
• —Machine Learning
• —Computer Science Theory and Algorithms
• —Mathematics
• —Statistics
• —Electrical Engineering
• —Astrophysics
• —Condensed Matter Physics
• —Quantum Physics
• —Quantitative Biology

Dataset

Balanced 12-class subset built from librarian-bots/arxiv-metadata-snapshot.

• —Train: 30,000 examples
• —Validation: 3,600 examples
• —Test: 3,600 examples

Metrics

• —Validation accuracy: 0.8350
• —Validation macro F1: 0.8351
• —Test accuracy: 0.8356
• —Test macro F1: 0.8351
• —Title-only test accuracy: 0.7522
• —Title-only test macro F1: 0.7495

Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

model_id = "Ian-Khalzov/article-topic-service-scibert"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Title: Large language models for scientific document classification\n\nAbstract: We study..."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256)
with torch.inference_mode():
    probs = torch.softmax(model(**inputs).logits[0], dim=-1)

predicted_label = model.config.id2label[int(probs.argmax())]
print(predicted_label)

Notes

The current baseline is strongest on physics-heavy classes and weakest on the broad Machine Learning category, where topical overlap with AI, NLP, CV, and Statistics remains high.