Modelpublic

RedTeamLab/Gemma-4-12B-Sol-Traces-v1

sourceHugging Facegemmaupdated 3mo agoView on Hugging Face

0likes245downloads

Model Card

Gemma-4-12B-Sol-Traces-v1

Repository coding-agent model fine-tuned from unsloth/gemma-4-12B-it with LoRA on a verified original-synthetic corpus compiled from Hermes Agent session logs. The traces do not originate from OpenCode.

Verified training configuration

Parameter	Value
Base model	`unsloth/gemma-4-12B-it`
Fine-tuning	LoRA, r=16; language + attention layers
Dataset	25,000 accepted trajectories: 21,174 train / 1,324 validation / 2,502 test
Epochs	1
Learning rate	1e-4, cosine schedule, 3% warmup
Effective batch size	8 (2 × 4 gradient accumulation)
Maximum sequence length	8,192 tokens
Loss	Assistant-only; tool responses excluded
GPU	Modal H100 80GB
Completed steps	377
Training loss	0.080072
Validation loss	0.025797
Runtime	10,848 s (3h 00m 48s)
Peak VRAM	46.72 GiB
Throughput	1,138.7 tokens/s

Dataset and trajectory policy

Sol Traces are compiled from Hermes Agent session logs produced while running deterministic, seed-based coding scenarios through a reference executor. The scenarios define repository templates, task requirements, and verification commands; accepted records retain the corresponding tool-use events and verification outcomes. Records are included only when their configured post-task validation succeeds.

Actual v1 coverage: 224 language/task/variant repository families across TypeScript, JavaScript, Python, shell, configuration, Go, Rust, and JVM/Java. Task categories are debugging, feature, refactoring, testing, build configuration, integration, and documentation review.

The schema defines list_files, read_file, search_code, run_command, and apply_patch. The v1 reference policy emits list_files, read_file, run_command, and apply_patch; it has no search_code calls. This model is therefore best understood as a fine-tune for the verified scripted v1 workflow, not a broadly trained autonomous coding agent.

Files

File	Description
`gemma-4-12b-sol-traces-v1-Q4_K_M.gguf`	Quantized merged model (Q4KM)
`gemma-4-12b-sol-traces-v1-f16.gguf`	Full merged F16 model
`training_stats.json`	Full training metrics
`training_report.json`	Duplicate training report retained for compatibility
`dataset_manifest.json`	Accepted-record counts, split ratios, and rejection summary

Usage

bash

llama-cli \
  -m gemma-4-12b-sol-traces-v1-Q4_K_M.gguf \
  -ngl 99 \
  --chat-template gemma \
  -p "List the repository files matching *.py"

Limitations

—Fixed five-tool schema; search_code was not used in v1 training trajectories.
—Single-trajectory sessions; no training for memory across independent conversations.
—Synthetic repository fixtures and a deterministic reference policy may not generalize to arbitrary real-world codebases.