agentlans/granite-embedding-107m-multilingual-chat-difficulty

granite-embedding-107m-multilingual-chat-difficulty

A fine-tuned model that estimates the difficulty of multilingual, multi-turn human–AI conversations based on reasoning complexity.

• —Input: A condensed conversation in the format <|user|>prompt<|assistant|>reply...
• —Output: A normalized difficulty score (lower scores indicate easier conversations)

Based on ibm-granite/granite-embedding-107m-multilingual.

Evaluation results:

• —Loss: 0.5663
• —MSE: 0.5663
• —Tokens processed: 51,173,120

Model description

This model maps multi-turn chat logs to a continuous difficulty representation, enabling comparison across languages and reasoning styles.

Use cases include:

• —Categorizing multilingual chat transcripts by reasoning depth.
• —Supporting dataset curation or curriculum design.
• —Serving as a difficulty scoring component in evaluation pipelines.

Intended uses and limitations

Use cases

• —Estimating reasoning difficulty in multilingual conversations.
• —Comparing dialogue complexity across datasets.
• —Benchmarking conversational reasoning.

Limitations

• —Not suitable for assessing factual accuracy, coherence, or sentiment.
• —May not generalize well to highly domain-specific data.
• —Produces relative difficulty scores, not absolute intelligence measures.

Training procedure

Hyperparameters

Results

Framework versions

• —Transformers: 5.0.0.dev0
• —PyTorch: 2.9.1+cu128
• —Datasets: 4.4.1
• —Tokenizers: 0.22.1

See also

agentlans/bge-small-en-v1.5-prompt-difficulty for single-turn English conversations and prompts