ylliprifti/hackathon-2025

GridScript™ DSL Expert - Fine-Tuned Llama 3.2 3B

This model is a fine-tuned version of Llama-3.2-3B using LoRA (Low-Rank Adaptation) for GridScript™ domain-specific language expertise.

Model Details

• —Base Model: meta-llama/Llama-3.2-3B
• —Fine-tuning Method: LoRA (PEFT)
• —Language: English
• —Domain: GridScript™ DSL for multidimensional data modeling
• —Training Data: 1,028 prompt-completion pairs

Training Configuration

Usage

With Transformers + PEFT

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-3B",
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load fine-tuned adapter
model = PeftModel.from_pretrained(base_model, "ylliprifti/hackathon-2025")
tokenizer = AutoTokenizer.from_pretrained("ylliprifti/hackathon-2025")

# Generate
prompt = "How do I use FLOWROLL to get a trailing 3-month total?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Merge Adapter (Optional)

# Merge LoRA weights into base model for faster inference
merged_model = model.merge_and_unload()
merged_model.save_pretrained("merged-model")

Training Data

This model was fine-tuned on 1,028 prompt-completion pairs covering GridScript™ DSL usage:

• —Functions: FLOWROLL() (rolling aggregations) and DIMMATCH() (dimensional alignment)
• —Question Types: How-to guides, troubleshooting, syntax help, conceptual explanations
• —Tone Variations: Casual, formal, technical, frustrated user, curious learner, problem-focused
• —Format: Universal prompt-completion format (not chat templates)

Data Composition

• —Original training set: 429 examples
• —Conceptual Q&A: 99 examples
• —Augmented variations: 500 examples (10 batches with different tones)
• —Total: 1,028 training examples

What This Model Does

This model specializes in:

• —✅ Explaining GridScript™ FLOWROLL() and DIMMATCH() functions
• —✅ Troubleshooting common errors (blanks, dimension mismatches, period ordering)
• —✅ Providing correct syntax examples with proper parameters
• —✅ Understanding context from various question styles (casual to formal)
• —❌ Not a general-purpose model - trained exclusively on GridScript™ DSL

Limitations

• —Domain-Specific: Only trained on GridScript™ FLOWROLL and DIMMATCH functions
• —No Other Functions: Does not know about other GridScript™ functions (SUM, IF, etc.)
• —Inherits Base Model Limitations: Subject to Llama 3.2 3B's general limitations
• —Not Production-Ready: Intended for hackathon/demo purposes without extensive evaluation
• —Fictional DSL: GridScript™ is a fictional language created for this training project

Example Queries

The model can answer questions like:

• —"How do I use FLOWROLL to get a trailing 3-month total?"
• —"Why does DIMMATCH fail when aligning revenue to customer list?"
• —"What happens if I set PeriodCount to 1?"
• —"FLOWROLL gives blanks for the first 5 periods. Why?"
• —"Can I use DIMMATCH on time dimensions?"

Training Details

• —Hardware: NVIDIA RTX Quadro 8000 (48GB)
• —Training Time: ~5 epochs
• —Optimization: Pre-tokenized dataset for faster training
• —Loss Masking: Only completion tokens used for loss (prompts masked with -100)
• —EOS Handling: Model learns to generate proper end-of-sequence tokens

License

This model is released under the MIT License. The base model (Llama 3.2) is subject to Meta's license terms.

Fine-tuned using MLOps pipeline with LoRA, PEFT, and custom tokenization for DSL training