Tonic/fr-on-device

Baguettotron vs Luth models

Apples-to-apples comparison of Baguettotron (PleIAs, 321M) and 5 Luth models (kurakurai, 0.4B–1.7B) from the Luth Models collection.

Features

• —All models, all outputs: Each prompt runs through all 6 models; outputs appear in tabs grouped by parameter size.
• —System prompt: Optional system prompt supported for both Baguettotron (Qwen-style) and Luth (chat template) model families.
• —Ultimate footprint: Per-model disk size and VRAM estimates; combined footprint for all models. A GGUF & LEAP bundle reference table lists PleIAs Baguettotron GGUF variants and Liquid LFM2 GGUF sizes (from LEAP / PleIAs/Baguettotron-GGUF).
• —Per-family generation settings: Two columns (Baguettotron | Luth) with sensible defaults: Baguettotron tuned for reasoning (e.g. temp 0.5, 512 tokens); Luth for instruct (e.g. temp 0.7, repeat_penalty 1.05).
• —Transformers-only: No quantization; all models run in BF16/FP16.

Size tiers

Luth Qwen3 load report (lm_head “MISSING”)

For Luth-0.6B-Instruct and Luth-1.7B-Instruct (Qwen3-based), the loader may report lm_head.weight | MISSING. This is expected: their config.json has "tie_word_embeddings": true, so the output head is tied to the embedding layer and not stored separately in the checkpoint. The model runs correctly.

Baguettotron EOS quirk

Baguettotron's tokenizer uses "<|im_end>" (no trailing pipe) for EOS. The app uses manual prompt formatting and stop sequences to avoid multi-token tokenization. See quirk.md for details.

Bundling Luth models (LEAP)

The script bundle_luth.py downloads a Luth model, validates it for LEAP, creates a GGUF bundle on the device, and investigates the result (sizes, optional inference).

pip install -r requirements-bundle.txt
leap-bundle login <api-key>   # from https://leap.liquid.ai/profile#/api-keys
# Single model:
python bundle_luth.py --model kurakurai/Luth-LFM2-350M
# All 5 Luth models (download → validate → create → download GGUF → investigate):
python bundle_luth.py --all

Options: --all (every Luth model in sequence; LEAP free tier = 5 requests/24h), --work-dir, --quantization (e.g. Q4KM, Q8_0), --dry-run (download + validate only, no create), --skip-create, --request-id <id> to download an existing bundle.

Download and inspect bundles: Use download_bundles.py to fetch completed bundle outputs by request ID. Per Liquid AI docs, artifacts are .gguf (default) or .bundle (ExecuTorch). The script inspects both and can run a short inference on .gguf:

python download_bundles.py --list                    # list requests, download all completed
python download_bundles.py --request-ids 1 2        # download specific IDs
python download_bundles.py --inspect-only --infer   # inspect existing downloads and run inference

Deployment

• —Hugging Face Spaces: Set hardware to Zero GPU (or standard GPU). Decorate GPU-dependent functions with @spaces.GPU per Spaces ZeroGPU docs.
• —Local: Run python app.py; requires a GPU with ~10 GB VRAM for all 6 models.