Modelpublic

Atikarahmanda/Qwen3-4B-SFT-Multitask

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes17downloads

Model Card

Qwen3-4B SFT — Multitask Bahasa Indonesia

Model hasil fine-tuning (LoRA, sudah di-merge ke base weights) dari `aitf-kpm-ugm/Qwen3-4B-CPT-Base` untuk berbagai tugas NLP Bahasa Indonesia.

Detail Model

Atribut	Nilai
Base model	`aitf-kpm-ugm/Qwen3-4B-CPT-Base`
Metode fine-tune	LoRA (r=64, alpha=128)
Status adapter	Merged ke base weights
Bahasa output	Bahasa Indonesia
Format chat	Alpaca
Precision	bfloat16
Max seq length	2048
Training epochs	3
Best checkpoint	step 2400
Best val loss	0.3566

Task yang Didukung

1.Sentimen Analysis — klasifikasi positif / netral / negatif
2.Justifier — klasifikasi is_relevant: true / false
3.Kategorisasi Issue — klasifikasi subcategorylabel

Cara Pakai

`python

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

REPO = "Atikarahmanda/Qwen3-4B-SFT-Multitask"

tokenizer = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained(
    REPO,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model.eval()

messages = [
    {"role": "system", "content": "Sistem prompt sesuai task."},
    {"role": "user",   "content": "Input artikel di sini."},
]

text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
input_len = inputs.input_ids.shape[1]

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=False,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.pad_token_id,
    )

print(tokenizer.decode(out[0, input_len:], skip_special_tokens=True))

Training Details

—Framework: Unsloth + TRL SFTTrainer
—LoRA config: r=64, alpha=128
—Optimizer: AdamW 8-bit
—LR scheduler: Cosine, warmup ratio 0.03
—Effective batch size: 6 x 8 = 48
—train_on_responses_only: Ya

Lisensi

Mengikuti lisensi base model: Apache 2.0.