Codeseys/composer-replication-framework

1# Examples Index2 3Five CPU-runnable examples demonstrating the framework end-to-end on4real HF causal LMs. They form a progression from simplest to most5methodologically complete:6 7| # | Example | Trace source | Channels | Wall-clock | Closes |8|---|---|---|---|---|---|9| 1 | [`qwen_05b_quickstart/`](qwen_05b_quickstart/) | minimal toy | LM-CE only | ~30s | "does the package import + run at all" |10| 2 | [`gsm8k_grpo/`](gsm8k_grpo/) | hand-written GSM8K (100 rows) | GRPO with `alpha=beta=0` | ~60s | Plain-GRPO baseline reference |11| 3 | [`gsm8k_grpo_with_sdpo/`](gsm8k_grpo_with_sdpo/) | hand-written GSM8K (B=2) | GRPO + SDPO column | ~25s | SDPO column wiring on synthetic prompts |12| 4 | [`sdpo_with_real_traces/`](sdpo_with_real_traces/) | `ClaudeCodeIngester` reading a hand-authored session JSONL | GRPO + SDPO column | ~30s | **Partial V5** — ingestion path validated; wiring smoke (misaligned) |13| **5** | **[`sdpo_with_real_traces_production/`](sdpo_with_real_traces_production/)** | **`ClaudeCodeIngester` → adapter → `ComposerDataCollator`** (with-error fixture) | **GRPO + SDPO (production-aligned)** | **~2min** | **V5 closure** — full production pipeline with error-site detection + properly-aligned SDPO mask |14 15**Recommended walk-through order**: 1 → 2 → 3 → 4 → 5. Each builds on16the previous in scope.17 18## Why five?19 20- **#1** verifies the package is installable and the loss composition21  works at all (no SDPO, no DPO — pure LM-CE on a toy model).22- **#2** uses the production `ComposerReplicationTrainer` (TRL `GRPOTrainer`23  subclass) on a real GSM8K dataset with a regex-extract reward. This24  is the recipe a new user copy-pastes to start.25- **#3** drops the TRL trainer wrapper and calls `compose_loss` directly26  on hand-crafted hint contexts. The simplest place to see "alpha_sdpo=0.527  changes the loss" with all the wiring visible.28- **#4** uses real ingested Claude Code session JSONL (via29  `ClaudeCodeIngester`) but builds the SDPO batch by hand —30  demonstrates the ingester works but the SDPO mask covers misaligned31  content. Wiring smoke, not production-grade.32- **#5** is the production-grade sibling to #4: adds the33  `claude_states_to_trace_examples` adapter and uses34  `ComposerDataCollator` to build properly-aligned SDPO batches with35  hint injection at actual error sites. **This is what you should copy36  for real training.**37 38## What every example asserts39 40Each `run.py` ends with a verification block that asserts:41 42- The targeted channel(s) actually fired (`sdpo_jsd > 0` when alpha_sdpo > 0)43- The composed loss isn't trivially equal to `lm_ce` alone44- Gradient norms are finite and non-zero at every step45 46Failure of any assertion exits non-zero and the script prints which47channel didn't fire. This is the user's smoke test, not just a demo.48 49## Production training50 51For real training (GPU, larger models, longer rollouts), use52`ComposerReplicationTrainer` directly with a `ComposerDataCollator`53that emits SDPO + DPO columns — exactly the path example #554demonstrates. See `docs/INTEGRATION_RECIPES.md` for the production55wiring patterns.56