Modelpublic

devopsforflops/functiongemma-270m-delia-dispatcher

sourceHugging Faceapache-2.0updated 10mo agoView on Hugging Face

0likes61downloads

Model Card

FunctionGemma 270M - Delia Dispatcher

A fine-tuned version of google/functiongemma-270m-it for Delia LLM orchestration.

This tiny model (270M params) acts as a fast dispatcher, routing user requests to the appropriate backend:

—call_coder - Code generation tasks
—call_reviewer - Code review and analysis
—call_planner - Architecture and planning (also handles ambiguous requests)
—call_executor - Running commands and scripts

Key Features

—Minimalist schema: Single reasoning parameter per tool
—Thought tokens: Brief CoT scratchpad before tool calls
—EOS hardening: Explicit stop tokens prevent infinite loops
—Negative samples: 13% ambiguous examples → planner (graceful handling)
—GBNF grammar: Constrained decoding for 100% valid output

Usage

With llama.cpp (recommended for speed)

bash

# Download the GGUF
wget https://huggingface.co/devopsforflops/functiongemma-270m-delia-dispatcher/resolve/main/functiongemma-270m-delia-dispatcher-f16.gguf

# Download the grammar
wget https://huggingface.co/devopsforflops/functiongemma-270m-delia-dispatcher/resolve/main/dispatcher.gbnf

# Run with grammar constraint
./llama-cli -m functiongemma-270m-delia-dispatcher-f16.gguf \
  --grammar-file dispatcher.gbnf \
  -p "<start_of_turn>user
Write a fibonacci function<end_of_turn>
<start_of_turn>model"

With Transformers

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("devopsforflops/functiongemma-270m-delia-dispatcher")
tokenizer = AutoTokenizer.from_pretrained("devopsforflops/functiongemma-270m-delia-dispatcher")

prompt = """<start_of_turn>user
Review this code for bugs<end_of_turn>
<start_of_turn>model"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0]))

Output Format

<start_of_turn>user
{request}<end_of_turn>
<start_of_turn>model
thought
{brief reasoning}
<tool_call>{"name": "call_X", "arguments": {"reasoning": "..."}}</tool_call><end_of_turn>

Training

Fine-tuned with Unsloth using LoRA:

—Epochs: 3
—LoRA rank: 32
—Training examples: 92 (balanced across 4 tools + 13% ambiguous)
—Final loss: 0.46

Files

File	Description
`functiongemma-270m-delia-dispatcher-f16.gguf`	GGUF model (F16, 518MB)
`model.safetensors`	Transformers model
`dispatcher.gbnf`	GBNF grammar for constrained decoding
`dispatcher_tools.json`	Tool schema (4 tools)
`train.jsonl`	Training data

License

Apache 2.0 (same as base model)

Part of Delia

This model is designed for use with Delia, an LLM orchestration system that routes requests to optimal backends.