Modelpublic

yimn-Aghosh/zegrate-ai

sourceHugging Faceapache-2.0updated 1d agoView on Hugging Face

14likes2.5kdownloads

Model Card

Zegrate AI — Phase 2 (DoRA fine-tune of Qwen2.5-14B-Instruct)

Zegrate is an open conversational AI with a humor-first personality, trained in phases on a single GPU. This repo hosts the Phase 1 merged base model (root model-*.safetensors) plus rolling Phase 2 DoRA adapters under checkpoint-*/.

Model details

—Base: Qwen/Qwen2.5-14B-Instruct
—Method: 4-bit QLoRA + DoRA, adamw_8bit, gradient checkpointing
—Data: multi-shard curated conversational data with humor emphasis (39k steps, 8 shards)
—Context: 32k tokens
—Status: Phase 2 training in progress — adapters update as shards complete

Usage (latest adapter)

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "yimn-Aghosh/zegrate-ai", device_map="auto", load_in_4bit=True)
tok = AutoTokenizer.from_pretrained("yimn-Aghosh/zegrate-ai")
model = PeftModel.from_pretrained(base, "yimn-Aghosh/zegrate-ai",
                                  subfolder="checkpoint-23540")

Use the highest-numbered checkpoint-* folder — that is the newest adapter. When Phase 2 finishes, the final adapter will also be promoted to the repo root.

Limitations

—In-training weights: expect rough edges until Phase 2 completes
—English-focused; humor-forward tone is deliberate, not a bug
—14B class — strong assistant, not frontier-scale

License

Apache 2.0, following Qwen2.5's licensing.