harshbhatt7585/arithmetic-king-1b

Arithmetic King 1B

Repository id: harshbhatt7585/arithmetic-king-1b.

This model is a PEFT LoRA adapter trained with TRL GRPO on synthetic arithmetic episodes. It is tuned to answer in XML format:

• —<reasoning>...</reasoning>
• —<answer>...</answer>

Artifact Type

This repo contains adapter weights only (not full base model weights). Use with base model:

• —meta-llama/Llama-3.2-1B-Instruct

Training Configuration

• —Trainer: TRL GRPOTrainer
• —Fine-tuning method: LoRA (PEFT)
• —Environment: arithmetic reasoning episodes
• —Reward: correctness reward + XML-format bonus
• —Output style target: short reasoning plus final integer answer

Intended Use

• —Arithmetic-reasoning RLVR experiments
• —GRPO/LoRA workflow demonstrations
• —Adapter-centric fine-tuning studies on small instruct models

Limitations

• —Trained on synthetic arithmetic prompts only
• —Limited transfer to broader reasoning/math tasks
• —May produce malformed XML or incorrect answers
• —Not suitable for high-stakes use

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_id = "meta-llama/Llama-3.2-1B-Instruct"
adapter_id = "harshbhatt7585/arithmetic-king-1b"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base_model = AutoModelForCausalLM.from_pretrained(base_id)
model = PeftModel.from_pretrained(base_model, adapter_id)

prompt = "Solve: (12 + 3) * 2. Return XML with <reasoning> and <answer>."
inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(out[0], skip_special_tokens=True))

License

Adapter usage inherits base model license and terms:

• —meta-llama/Llama-3.2-1B-Instruct