hks350d/git-diff-to-commit-gemma-3-270m

  pip install mlx-lm transformers

Generate a concise and descriptive commit message for this git diff:

{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant that generates git commit messages. Always respond in English only. Do not use any other language."},
    {"role": "user", "content": "Generate a concise and descriptive commit message for this git diff:\n\n```diff\n<diff text>\n```"},
    {"role": "assistant", "content": "<single-line commit message>"}
  ]
}

#!/usr/bin/env python3
"""
Standalone script to generate git commit messages using the fine-tuned Gemma model.
Requires: mlx-lm, transformers
Install with: pip install mlx-lm transformers
"""

import subprocess
import sys
from mlx_lm import load, generate
from transformers import AutoTokenizer

def get_staged_diff():
    """Get the staged git diff from the current repository."""
    try:
        result = subprocess.run(
            ['git', 'diff', '--staged', '--no-color'], 
            capture_output=True, text=True, check=True
        )
        return result.stdout.strip()
    except subprocess.CalledProcessError:
        print("Error: Could not get git diff. Make sure you're in a git repository with staged changes.")
        return None

def format_prompt(diff_text, tokenizer):
    """Format the diff into the exact training data format."""
    system_prompt = "You are a helpful assistant that generates git commit messages. Always respond in English only. Do not use any other language."
    user_message = f"Generate a concise and descriptive commit message for this git diff:\n\n```diff\n{diff_text}\n```"
    
    # Format using Gemma chat template
    messages = [
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_message}
    ]
    
    prompt = tokenizer.apply_chat_template(
        messages, 
        tokenize=False, 
        add_generation_prompt=True
    )
    return prompt

def generate_commit_message(diff_text, model_path="your-username/git-diff-to-commit-gemma-3-270m"):
    """Generate a commit message from a git diff."""
    
    # Load model and tokenizer
    print("Loading model...")
    model, mlx_tokenizer = load(model_path)
    hf_tokenizer = AutoTokenizer.from_pretrained("google/gemma-3-270m-it")
    
    # Format the prompt
    prompt = format_prompt(diff_text, hf_tokenizer)
    
    # Generate response
    print("Generating commit message...")
    response = generate(
        model, 
        mlx_tokenizer, 
        prompt=prompt, 
        max_tokens=100,
        temp=0.7,
        top_p=0.9,
        verbose=False
    )
    
    # Extract just the generated part (after the prompt)
    generated_text = response[len(prompt):].strip()
    
    # Return the first non-empty line
    lines = [line.strip() for line in generated_text.split('\n') if line.strip()]
    return lines[0] if lines else "Unable to generate commit message"

def main():
    """Main function - can be used with staged diff or provided diff text."""
    
    if len(sys.argv) > 1:
        # Use provided diff file
        diff_file = sys.argv[1]
        try:
            with open(diff_file, 'r') as f:
                diff_text = f.read().strip()
        except FileNotFoundError:
            print(f"Error: File {diff_file} not found.")
            return
    else:
        # Get staged diff from git
        diff_text = get_staged_diff()
        if not diff_text:
            print("No staged changes found. Stage some changes with 'git add' first.")
            return
    
    if not diff_text:
        print("No diff content to process.")
        return
    
    # Generate and print commit message
    commit_message = generate_commit_message(diff_text)
    print(f"\nSuggested commit message:")
    print(f"  {commit_message}")

if __name__ == "__main__":
    main()

   python generate_commit.py

   python generate_commit.py my_changes.diff

   from generate_commit import generate_commit_message
   
   diff = """diff --git a/app.py b/app.py
   index e69de29..f4c3b4a 100644
   --- a/app.py
   +++ b/app.py
   @@ -0,0 +1,3 @@
   +def add(a, b):
   +    return a + b
   """
   
   message = generate_commit_message(diff)
   print(message)

Generate a concise and descriptive commit message for this git diff:

Possible outputs:
- fix: use new_zp instead of old_zp for magnitude calculation and add zeropoint change warning
- fix: correct zeropoint usage in photometry and add warning for large zeropoint changes
- refactor: update magnitude calculation to use new zeropoint and add change detection

## Training summary

- Base model: `google/gemma-3-270m-it` (Gemma 3, 270M, instruction-tuned).
- Method: LoRA fine-tuning with MLX (`mlx_lm lora`). Prompt masking was enabled so the model learns from the assistant response.
- **Training data**: `data/train_gpt-oss-20b.jsonl` in this repository - a dataset converted to chat format with diffs fenced as ```diff and English, single-line commit messages as targets. This dataset is Python-focused.
- Data format: Each training example uses the exact user prompt format shown above in the chat template structure.
- Context/config highlights: max sequence length ~2048 tokens; diffs truncated to ~3,000 characters during preprocessing/inference to be model-friendly.
- **Important**: To achieve best results, match the exact input format used in the training data.

## Evaluation

- The repo includes a lightweight evaluation that compares generated messages to a reference using a simple string similarity (SequenceMatcher) across multiple runs (varying the RNG seed). Results and artifacts are saved under `evaluation_results/`.

## Limitations and risks

- Diff size sensitivity: Very large diffs may be truncated; consider summarizing large changes.
- Domain bias: Training set emphasized Python diffs; behavior may be better for Python-heavy repos.
- Hallucinations: As with any LLM, may produce generic or mismatched messages if the diff is ambiguous.
- Security: Do not feed secrets; generated text may inadvertently paraphrase sensitive context.
- Language: System prompt enforces English responses.

## Intended use

- Assist developers by proposing a concise commit subject from a given git diff.
- Not a replacement for human judgment; review messages before committing.

## How to format inputs yourself

If you’re not using the CLI helpers, follow this structure with the Gemma chat template:

- system: English-only instruction for commit message generation (see above)
- user: instruction + the diff in ```diff code fences
- assistant: the target single-line subject (for training) or left empty (for inference)

The repository’s `format_commit_message_prompt` builds the correct prompt for Gemma 3.

## License and credits

- Base model: Google Gemma 3 (`google/gemma-3-270m-it`). Use subject to the Gemma license terms.
- Fine-tuning code: MLX and utilities in this repository. See repository license for details.