Modelpublic

loganh274/nlp-testing-setfit

sourceHugging Faceapache-2.0updated 9mo agoView on Hugging Face

0likes22downloads

Model Card

SetFit Sentiment Analysis Model

This is a SetFit model fine-tuned for sentiment classification on customer feedback data.

Model Description

Property	Value
Base Model	BAAI/bge-base-en-v1.5
Total Parameters	109,482,240
Trainable Parameters	109,482,240
Body Parameters	109,482,240
Head Parameters	0
Model Size	417.64 MB
Labels	[0, 1, 2, 3, 4]
Number of Classes	5
Serialization	safetensors

Training Configuration

Parameter	Value
Batch Size	16
Epochs	[1, 16]
Training Samples	540
Test Samples	100
Loss Function	CosineSimilarityLoss
Metric for Best Model	embedding_loss

Training Progress

—Initial Loss: 0.2366
—Final Loss: 0.0893
—Eval Loss: 0.0984
—Training Runtime: 800.2981 seconds
—Samples/Second: 13.4950

Evaluation Results

Metric	Score
Accuracy	0.8800
F1 (Weighted)	0.8805
F1 (Macro)	0.8805
Precision (Weighted)	0.8883
Precision (Macro)	0.8883
Recall (Weighted)	0.8800
Recall (Macro)	0.8800

Per-Class Performance

              precision    recall  f1-score   support

           0       0.90      0.90      0.90        20
           1       0.75      0.75      0.75        20
           2       0.79      0.95      0.86        20
           3       1.00      0.80      0.89        20
           4       1.00      1.00      1.00        20

    accuracy                           0.88       100
   macro avg       0.89      0.88      0.88       100
weighted avg       0.89      0.88      0.88       100

Visualizations

Evaluation Metrics Overview

Confusion Matrix

Training Loss Curve

Learning Rate Schedule

Usage

python

from setfit import SetFitModel

# Load the model
model = SetFitModel.from_pretrained("loganh274/nlp-testing-setfit")

# Single prediction
text = "This product exceeded my expectations!"
prediction = model.predict([text])
print(f"Sentiment: {prediction[0]}")

# Batch prediction
texts = [
    "Amazing quality, highly recommend!",
    "It's okay, nothing special.",
    "Terrible experience, very disappointed.",
]
predictions = model.predict(texts)
probabilities = model.predict_proba(texts)

for text, pred, prob in zip(texts, predictions, probabilities):
    print(f"Text: {text}")
    print(f"  Prediction: {pred}, Confidence: {max(prob):.2%}")

Label Mapping

Label	Sentiment
0	Negative
1	Somewhat Negative
2	Neutral
3	Somewhat Positive
4	Positive

Environment

Package	Version
Python	3.11.14
SetFit	1.1.3
PyTorch	2.9.1
scikit-learn	1.8.0
Transformers	N/A

Citation

If you use this model, please cite the SetFit paper:

bibtex

@article{tunstall2022efficient,
  title={Efficient Few-Shot Learning Without Prompts},
  author={Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
  journal={arXiv preprint arXiv:2209.11055},
  year={2022}
}

License

Apache 2.0