ethanker/lfm2_350m_commit_diff_summarizer

lfm2350mcommitdiffsummarizer (LoRA)

A lightweight helper model that turns Git diffs into Conventional Commit–style messages. It outputs strict JSON with a short title (≤ 65 chars) and up to 3 bullets, so your CLI/agents can parse it deterministically.

Model Details

Model Description

• —Purpose: Summarize git diff patches into concise, Conventional Commit–compliant titles with optional bullets.
• —I/O format:

• —Input: prompt containing the diff (plain text).
• —Output: JSON object: {"title": "...", "bullets": ["...", "..."]}.
• —Model type: LoRA adapter for causal LM (text generation)
• —Language(s): English (commit message conventions)
• —Finetuned from: unsloth/LFM2-350M-unsloth-bnb-4bit (4-bit quantized base, trained with QLoRA)

Model Sources

• —Repository: This model card + adapter on the Hub under ethanke/lfm2_350m_commit_diff_summarizer

Uses

Direct Use

• —Convert patch diffs into Conventional Commit messages for PR titles, commits, and changelogs.
• —Provide human-readable summaries in agent UIs with guaranteed JSON structure.

Recommendations

• —Enforce JSON validation; if invalid, retry with a JSON-repair prompt.
• —Keep a regex gate for Conventional Commit titles in your pipeline.

How to Get Started

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch, json

BASE = "unsloth/LFM2-350M-unsloth-bnb-4bit"
ADAPTER = "ethanke/lfm2_350m_commit_diff_summarizer"  # replace with your repo id

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.float16)

tok = AutoTokenizer.from_pretrained(BASE, use_fast=True)
mdl = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
mdl = PeftModel.from_pretrained(mdl, ADAPTER)

diff = "...your git diff text..."
prompt = (
  "You are a commit message summarizer.\n"
  "Return a concise JSON object with fields 'title' (<=65 chars) and 'bullets' (0-3 items).\n"
  "Follow the Conventional Commit style for the title.\n\n"
  "### DIFF\n" + diff + "\n\n### OUTPUT JSON\n"
)

inputs = tok(prompt, return_tensors="pt").to(mdl.device)
with torch.no_grad():
    out = mdl.generate(**inputs, max_new_tokens=200, do_sample=False)
text = tok.decode(out[0], skip_special_tokens=True)

# naive JSON extraction
js = text[text.rfind("{"): text.rfind("}")+1]
obj = json.loads(js)
print(obj)

Training Details

Training Data

• —Dataset: Maxscha/commitbench (diff → commit message).
• —Filtering: kept only samples whose first non-empty line of the message matches Conventional Commits: ^(feat|fix|docs|style|refactor|perf|test|build|ci|chore|revert)(\([^)]+\))?(!)?:\s.+$
• —Note: The dataset card indicates non-commercial licensing. Confirm before commercial deployment.

Training Procedure

• —Method: Supervised fine-tuning (SFT) with TRL SFTTrainer + QLoRA (PEFT).
• —Prompting: Instruction + ### DIFF + ### OUTPUT JSON target (title/bullets).
• —Precision: fp16 compute on 4-bit base.
• —Hyperparameters (v0.1):

• —max_length=2048, per_device_train_batch_size=2, grad_accum=4
• —lr=2e-4, scheduler=cosine, warmup_ratio=0.03
• —epochs=1 over capped subset
• —LoRA: r=16, alpha=32, dropout=0.05, targets: q/k/v/o + MLP proj

Evaluation

• —Validation: filtered split from CommitBench.
• —Metrics (example run):

• —eval_loss ≈ 1.18 → perplexity ≈ 3.26
• —eval_mean_token_accuracy ≈ 0.77
• —Suggested task metrics: JSON validity rate, CC-title compliance, title length ≤ 65 chars, bullets ≤ 3.

Environmental Impact

• —Hardware: 1× NVIDIA GTX 3060 12 GB (local)
• —Hours used: ~2 h (prototype)

Technical Specifications

• —Architecture: LFM2-350M (decoder-only) + LoRA adapter
• —Libraries: transformers, trl, peft, bitsandbytes, datasets, unsloth

Contact

• —Open an issue on the Hub repo or message ethanke on Hugging Face.

Framework versions

• —PEFT 0.17.1
• —TRL (SFTTrainer)
• —Transformers (recent version)