sahellx/kitty-bash-llm

🐱 Kitty Bash LLM

A 0.5B parameter shell assistant that runs on a laptop CPU.

Kitty Bash LLM turns plain English into bash commands, repairs commands that just failed, and completes half-typed ones. It is deliberately small — the 4-bit GGUF is 398 MB and generates a command in well under a second on an ordinary CPU, so it can sit behind a shell integration without a GPU, an API key, or a network round trip.

It is a specialist, not a chatbot. Ask it the capital of France and you will get nonsense. Ask it to find every file over 100 MB and it will tell you.

What it does

Three behaviours, selected by the system prompt. One model, one file, flat memory.

System prompts

NL2CMD   = ("You are a bash command generator. Given a natural language request, "
            "output only the bash command that accomplishes it. "
            "No explanation, no markdown fences.")

FIXCMD   = ("You fix broken bash commands. Given a failed command and its error "
            "output, output only the corrected command. No explanation.")

COMPLETE = ("You complete partially typed bash commands. Given a command prefix, "
            "output only the text that completes it. No explanation.")

Quick start

llama.cpp / GGUF (recommended — this is what it was built for)

llama-cli -m kitty-bash-llm-q4_k_m.gguf -st -t 4 --no-display-prompt -n 64 --temp 0 \
  -p "<|im_start|>system
You are a bash command generator. Given a natural language request, output only the bash command that accomplishes it. No explanation, no markdown fences.<|im_end|>
<|im_start|>user
show which process is listening on port 8080<|im_end|>
<|im_start|>assistant
"
# -> lsof -i :8080

For anything interactive, run a persistent server so the system prompt is prefilled once (see Performance — this is the single most important implementation detail):

llama-server -m kitty-bash-llm-q4_k_m.gguf -c 1024 -t 4 --port 8080

transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("sahellx/kitty-bash-llm")
tok   = AutoTokenizer.from_pretrained("sahellx/kitty-bash-llm")

SYSTEM = ("You are a bash command generator. Given a natural language request, "
          "output only the bash command that accomplishes it. "
          "No explanation, no markdown fences.")

msgs = [{"role": "system", "content": SYSTEM},
        {"role": "user",   "content": "compress the logs folder into a tar.gz archive"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=64, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
# -> tar -czvf logs.tar.gz logs/

Greedy decoding (do_sample=False / --temp 0) is recommended. You want the most likely command, not a creative one.

Evaluation

Measured against the untuned base model on a held-out split, with identical prompts, identical f16 GGUF format, and the same llama.cpp engine — the only variable is the fine-tuning. 100 examples per task.

nl2cmd

fixcmd

⚠️ Read this number sceptically. The base model scores ~0 largely because it ignores the output format and explains the error in prose rather than emitting a command — so much of the gap is format compliance, not repair skill. The test set also uses synthetic corruptions of three kinds the model trained on. On real shell errors, hands-on testing put it closer to 1-in-3. Treat 82% as an upper bound.

complete

Exact match is a poor metric here — many completions are valid for any prefix. bash -n on the completion fragment alone is meaningless (a fragment like --rm ubuntu is not valid standalone bash); the meaningful figure is prefix + completion, at 99.29%.

Comparison with other shell-focused models

Alongside the base-model comparison above, an informal benchmark was run against four other publicly available fine-tunes for shell/linux command generation, all at Q4KM (except where a repo ships only f16), through llama.cpp on the same 2-thread CPU, using the same 80 held-out prompts.

#### ⚠️ Please read this before citing the table This is indicative, not a controlled evaluation. Four specific caveats, all of which favour this model: 1. The test set is drawn from this model's own training distribution. The split is properly held out with zero leakage, but it comes from the same corpora and shares their formatting conventions. The other models never saw that style. This inflates exact match in particular, which is why exact match is omitted above. 2. *All models received this model's system prompt. Peers trained with a different prompt format are being evaluated off-distribution, which alone could account for a large part of the gap. 3. n = 80. The 95% confidence interval is roughly ±9 points; the gap to the runner-up is significant only marginally (p ≈ 0.03). 4. `chamibuddhika/linux-commands-0407-00` returned empty output for every prompt and is reported as no-result rather than zero — the cause may well be the harness rather than the model. `qwen3.5-shellcommand` emits `</think>` reasoning tags that the harness did not strip, so its syntax-validity score was invalid and has been omitted here. A properly controlled comparison would use a neutral test set none of the models trained on, each model's own prompt format, and n ≥ 500. Treat the ordering above as a hint worth verifying, not a settled result. The base-model comparison* in the previous section does not carry these caveats: same architecture, same data, same harness, only the fine-tuning differs.

ShellCheck is used because bash -n only proves a command parses. ShellCheck catches genuine defects — unquoted expansions (SC2086), word splitting (SC2046) — that parse fine and then break on a filename with a space.

Training

The base model was chosen by measurement, not assumption: a bake-off against Qwen2.5-0.5B-Instruct under an identical recipe gave 84.98 vs 84.21 utility accuracy, so the code-pretrained variant won.

Dataset

~60,000 examples across the three tasks, pooled from seven sources, deduplicated, rebalanced, and decontaminated.

Sources: `AnishJoshi/nl2bash-custom`, `neulab/tldr`, `emirkaanozdemr/bash_command_data_6K`, `huytd189/command-line-suggestions`, `aelhalili/bash-commands-dataset`, `Jawajawa/command-linux-bash-balanced-sft`, `PocketDoc/Dans-Toolmaxx-ShellCommands`.

Decontamination — please read this if you benchmark on nl2bash

The most widely used NL2Bash split on the Hub leaks badly. Measured directly:

test  prompts also present in train : 71.0 %
dev   prompts also present in train : 71.8 %
duplicate rows within train         : 39.5 %  (19,658 rows → 11,890 unique)

Training and evaluating on those splits measures memorisation. Every published number here comes from splits rebuilt from scratch: all rows pooled, deduplicated on the normalised prompt, re-split 90/5/5 with a fixed seed, and asserted in code to have zero train↔test overlap.

Two further corrections:

• —`find` rebalanced 54% → 30%. In the source corpus more than half of all targets were find, which made the model a find specialist that produced nonsense for tar and du. Capping it fixed both.
• —Gold targets filtered through `bash -n`. Broken targets teach broken bash.
• —*`fixcmd` and `complete` derived after the split*, from within each split only — so a command in test can never appear in train under a different task label.

Limitations

Findings from hands-on testing, not guesses.

Weak on multi-step logic. It picks the right tool but can get the reasoning wrong:

"print the 3rd column of a csv where the 1st column equals ERROR"
  → cut -d',' -f3 data.csv | grep ERROR          ✗ filters the wrong column
  correct: awk -F, '$1=="ERROR" {print $3}'

"show the 10 biggest directories under /home sorted by size"
  → find /home -type d | sort -n -r | head -10   ✗ sorts names, not sizes
  correct: du -h /home/* | sort -rh | head -10

Uneven across utilities. Strong on find, tar, docker, sed, grep, lsof. Weaker on less common subcommands (git branch --merged came out as gibberish).

`fixcmd` is much weaker on real errors than the benchmark suggests — see the warning above.

`complete` may duplicate a prefix boundary. Given docker run -it -- it can return --rm .... Strip the overlap between prefix and completion in your client.

Catastrophic forgetting — by design. General ability is gone. This is a bash tool, not an assistant.

No safety behaviour whatsoever. Asked to delete the root filesystem it emits the command immediately, with no warning. See below.

⚠️ Safety

This model will generate destructive commands without hesitation.

"delete every file in the root filesystem"  →  find / -exec rm {} \;

It has no refusal training and no notion of danger. Any tool built on it must:

1. 1.Never auto-execute. Print the command; require an explicit keypress.
2. 2.Pattern-match destructive commands — rm -rf, dd, mkfs, > /dev/sd*, chmod -R 777, curl … | sh — and warn loudly.
3. 3.Treat output as untrusted. It is a suggestion from a 0.5B model, not an authority.

Verify commands before running them. Especially ones touching /.

Performance

Measured with llama.cpp, Q4KM, on 2 CPU threads (deliberately weak):

Implication: run a persistent server. Over half of a cold request is spent re-processing the same system prompt. Spawning a process per invocation costs ~1.6 s; a warm llama-server costs ~700 ms. On a typical 8-core machine expect roughly 3–4× faster (~200 ms).

For inline autosuggestion, that is still too slow to run on every keystroke. Debounce (~250 ms), fire asynchronously, and cancel in-flight requests — the Copilot pattern. History-based matching should handle the common case; use the model for what history has never seen.

Files

Reproducing this

Every step is a script, not a notebook cell: data pooling and decontamination, the base-model bake-off, training, GGUF conversion, and the evaluation harness (including the ShellCheck pass). The find rebalancing and the leakage assertions are the parts worth copying if you build on the NL2Bash corpora.

Citation

@misc{kitty-bash-llm,
  title  = {Kitty Bash LLM: a 0.5B shell assistant for CPU inference},
  author = {sahellx},
  year   = {2026},
  url    = {https://huggingface.co/sahellx/kitty-bash-llm}
}

Built on Qwen2.5-Coder (Apache-2.0), trained with Unsloth, quantized with llama.cpp.