Team Ai
Modelpublic

AntoineMC/distilbart-mnli-github-issues

sourceHugging Faceapache-2.0updated 4y agoView on Hugging Face
4likes34downloads
Model Card

GitHub issues classifier (using zero shot classification)

Predicts wether a statement is a feature request, issue/bug or question

This model was trained using the **Zero-shot classifier distillation** method with the BART-large-mnli model as teacher model, to train a classifier on Github issues from the Github Issues Prediction dataset

Labels

As per the dataset Kaggle competition, the classifier predicts wether an issue is a bug, feature or question. After playing around with different labels pre-training I've used a different mapping of labels that yielded better predictions (see notebook here for details), labels being

  • —issue
  • —feature request
  • —question

Training data

  • —15k of Github issues titles ("unlabeledtitlessimple.txt")
  • —Hypothesis used: "This request is a {}"
  • —Teacher model used: valhalla/distilbart-mnli-12-1
  • —Studend model used: distilbert-base-uncased

Results

Agreement of student and teacher predictions: 94.82%

See this notebook for more info on feature engineering choice made

How to train using your own dataset

  • —Download training dataset from https://www.kaggle.com/datasets/anmolkumar/github-bugs-prediction
  • —Modify and run convert.py, updating the paths to convert to a CSV
  • —Run distill.py with the csv file (see here for more info)

Acknowledgements