Modelpublic

tinyopsec/Qwen3.8-4B-Empero-AI-FullStack-GGUF

sourceHugging Faceapache-2.0updated 28d agoView on Hugging Face

0likes2.4kdownloads

Model Card

Qwen3.8-4B-Empero-AI-FullStack — GGUF Quantizations

This repository contains GGUF quantizations of iBotIA/Qwen3.8-4B-Empero-AI-FullStack.

📦 Available Quantizations

File	Bits	Size (approx.)	Use case
`model_f16.gguf`	16-bit	~8.7 GB	Maximum quality, reference
`model_q8_0.gguf`	8-bit	~4.7 GB	Near-lossless, high VRAM
`model_q6_k.gguf`	6-bit	~3.6 GB	Excellent quality
`model_q5_k_m.gguf`	5-bit	~3.1 GB	Great quality/size balance
`model_q5_k_s.gguf`	5-bit	~3.0 GB	Slightly smaller than K_M
`model_q4_k_m.gguf`	4-bit	~2.5 GB	Recommended default
`model_q4_k_s.gguf`	4-bit	~2.4 GB	Smaller 4-bit variant
`model_q3_k_l.gguf`	3-bit	~2.1 GB	Low VRAM, decent quality
`model_q3_k_m.gguf`	3-bit	~1.9 GB	Balanced 3-bit
`model_q3_k_s.gguf`	3-bit	~1.7 GB	Minimum 3-bit
`model_q2_k.gguf`	2-bit	~1.3 GB	Extreme compression

IQ quants (IQ4_XS, etc.) coming soon — require imatrix calibration.

🚀 Usage

llama.cpp

bash

./llama-cli -m model_q4_k_m.gguf -p "Your prompt here" -n 512

LM Studio

Download any .gguf file and load it directly in LM Studio.

Ollama

bash

ollama run hf.co/tinyopsec/Qwen3.8-4B-Empero-AI-FullStack-GGUF:Q4_K_M

Python (llama-cpp-python)

python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="tinyopsec/Qwen3.8-4B-Empero-AI-FullStack-GGUF",
    filename="model_q4_k_m.gguf",
)
output = llm("Your prompt here", max_tokens=512)
print(output["choices"][0]["text"])

🔧 Quantization Details

—Source model: iBotIA/Qwen3.8-4B-Empero-AI-FullStack
—Tool: llama.cpp
—Base format: F16 GGUF
—IQ calibration data: groups_merged.txt by kalomaze

💡 Which quant should I use?

VRAM	Recommended
2 GB	Q2_K
3 GB	Q3KM
4 GB	Q4KM ✅
6 GB	Q5KM
8 GB	Q6_K
12 GB+	Q8_0 / F16

📄 License

Refer to the original model license.