QuantLLM/functiongemma-270m-it-4bit-gguf

<div align="center">

🦙 functiongemma-270m-it-4bit-gguf

google/functiongemma-270m-it converted to GGUF format

 ![Format]() ![Quantization]()

<a href="https://github.com/codewithdark-git/QuantLLM">⭐ Star QuantLLM on GitHub</a>

</div>

📖 About This Model

This model is [google/functiongemma-270m-it](https://huggingface.co/google/functiongemma-270m-it) converted to GGUF format for use with llama.cpp, Ollama, LM Studio, and other compatible inference engines.

🚀 Quick Start

Option 1: Python (llama-cpp-python)

from llama_cpp import Llama

# Load the model
llm = Llama.from_pretrained(
    repo_id="QuantLLM/functiongemma-270m-it-4bit-gguf",
    filename="functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf",
)

# Generate text
output = llm(
    "Write a short story about a robot learning to paint:",
    max_tokens=256,
    echo=True
)
print(output["choices"][0]["text"])

Option 2: Ollama

# Download the model
huggingface-cli download QuantLLM/functiongemma-270m-it-4bit-gguf functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf --local-dir .

# Create Modelfile
echo 'FROM ./functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf' > Modelfile

# Import to Ollama
ollama create functiongemma-270m-it-4bit-gguf -f Modelfile

# Chat with the model
ollama run functiongemma-270m-it-4bit-gguf

Option 3: LM Studio

1. 1.Download the .gguf file from the Files tab above
2. 2.Open LM Studio → My Models → Add Model
3. 3.Select the downloaded file
4. 4.Start chatting!

Option 4: llama.cpp CLI

# Download
huggingface-cli download QuantLLM/functiongemma-270m-it-4bit-gguf functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf --local-dir .

# Run inference
./llama-cli -m functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf -p "Hello! " -n 128

📊 Model Details

📦 Quantization Details

This model uses Q4_K_M quantization:

All Available GGUF Quantizations

🚀 Created with QuantLLM

<div align="center">


Convert any model to GGUF, ONNX, or MLX in one line!

from quantllm import turbo

# Load any HuggingFace model
model = turbo("google/functiongemma-270m-it")

# Export to any format
model.export("gguf", quantization="Q4_K_M")

# Push to HuggingFace
model.push("your-repo", format="gguf")

<a href="https://github.com/codewithdark-git/QuantLLM"> <img src="https://img.shields.io/github/stars/codewithdark-git/QuantLLM?style=social" alt="GitHub Stars"> </a>

[📚 Documentation](https://github.com/codewithdark-git/QuantLLM#readme) · [🐛 Report Issue](https://github.com/codewithdark-git/QuantLLM/issues) · [💡 Request Feature](https://github.com/codewithdark-git/QuantLLM/issues)

</div>