StanfordSCALE/assertion_sentence_has_comparison_terms

Assertion: sentence has comparison terms

This classifier was trained for EduBehaviors: Assertion-based schemas for auditable dialogue coding and is usable through the Python package EduBehaviors-kit. This classifier was trained on an LLM-annotated subset of teacher utterances from the TalkMoves Dataset. See the Datasets section below for more information.

Training Details

Datasets

This model's columns are assertion_sentence_has_comparison_terms and split_sentence_has_comparison_terms.

Base rate (share of rows labeled as True): 6.2% overall — 6.2% train, 6.3% dev, 6.2% test.

Labels and annotation

Labels were generated with LLM annotators. Krippendorff's alpha for this assertion is 0.657.

Hyperparameters

Evaluation

Results

Limitations

• —Labels come from LLM annotators, not human coders. Agreement between annotators with Krippendorff's Alpha is 0.657.
• —Trained on teacher utterances only. Behaviour on student speech is untested.

How to Use

Message Structure

The model was trained on text built as:

{utterance}

The utterance is passed through as-is.

Running instructions

pip install setfit

from setfit import SetFitModel

model = SetFitModel.from_pretrained("StanfordSCALE/assertion_sentence_has_comparison_terms")

text = 'What I want you to focus on today is how can you relate the two diameters to the slant height and then how can you kind of think about the relationship between all three measurements'
model.predict([text])        # -> array([1]) when the assertion holds
model.predict_proba([text])  # -> [[P(no), P(yes)]]

Citation

@misc{assertion_sentence_has_comparison_terms,
  author = {Stanford SCALE Initiative},
  title  = {Assertion classifier: sentence has comparison terms},
  year   = {2026},
  url    = {https://huggingface.co/StanfordSCALE/assertion_sentence_has_comparison_terms}
}