darwinkernelpanic/CodeLlama-7b-Instruct-hf-luau

Model Card for CodeLlama-7B-Instruct-Luau

Fine-tuned version of codellama/CodeLlama-7b-Instruct-hf targeted toward the Luau programming language, Roblox’s Lua-derived scripting language.

This model is distributed as a LoRA adapter and is intended to improve the base model’s performance on Roblox-specific scripting tasks.

Model Details

Model Description

This model is a parameter-efficient fine-tuning (LoRA) of CodeLlama 7B Instruct, specialized for generating, explaining, and refactoring Luau code.

The fine-tuning focuses on Roblox development patterns, including common services, APIs, gameplay scripting idioms, and client/server logic. The model is designed to assist developers during prototyping, learning, and general scripting workflows.

• —Developed by: darwinkernelpanic
• —Funded by: Not applicable
• —Shared by: darwinkernelpanic
• —Model type: Causal Language Model (decoder-only, LoRA adapter)
• —Language(s) (NLP): English
• —License: Apache-2.0
• —Finetuned from model: codellama/CodeLlama-7b-Instruct-hf

Model Sources

• —Repository: https://huggingface.co/darwinkernelpanic/CodeLlama-7b-Instruct-hf-luau
• —Paper: Code Llama: Large Language Models for Code (Meta AI)
• —Demo: Not available

Uses

Direct Use

This model can be used directly for:

• —Writing Luau scripts for Roblox
• —Explaining Roblox APIs and services
• —Refactoring or debugging Luau code
• —Prototyping gameplay systems and utilities
• —Learning Luau and Roblox scripting concepts

The model is intended as a developer assistant, not an autonomous system.

Downstream Use

Potential downstream uses include:

• —Further fine-tuning on proprietary Roblox frameworks
• —Integration into IDEs or editor tooling
• —Chat-based assistants for Roblox development
• —Educational or documentation tooling

Out-of-Scope Use

This model should not be used for:

• —Safety-critical or production-critical systems
• —Legal, medical, or financial advice
• —Malware, exploit, or cheat development
• —Fully automated code deployment without review

Bias, Risks, and Limitations

• —Inherits biases and limitations from the base CodeLlama model
• —May hallucinate Roblox APIs or outdated behaviors
• —Does not validate code at runtime
• —Output correctness depends on prompt quality

Recommendations

Users should:

• —Review all generated code manually
• —Test scripts in Roblox Studio
• —Cross-check with official Roblox documentation
• —Treat outputs as suggestions rather than authoritative solutions

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model = "codellama/CodeLlama-7b-Instruct-hf"
adapter_model = "darwinkernelpanic/CodeLlama-7b-Instruct-hf-luau"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(base_model)
model = PeftModel.from_pretrained(model, adapter_model)

prompt = "Write a Luau function that creates a Part and parents it to Workspace."
inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=300,
    temperature=0.7,
    do_sample=True
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

The model was fine-tuned on a curated mixture of:

• —Luau scripts
• —Roblox API usage examples
• —Open-source Roblox projects
• —Synthetic instruction-style prompts

All data was filtered to avoid private, proprietary, or sensitive content.

Training Procedure

The model was trained using parameter-efficient fine-tuning with LoRA while keeping the base model weights frozen.

Preprocessing

• —Code formatting normalization
• —Instruction-style prompt structuring
• —Removal of low-quality or irrelevant samples

Training Hyperparameters

• —Training regime: fp16 mixed precision

Speeds, Sizes, Times

• —Base model size: ~7B parameters
• —Trainable parameters: <1% (LoRA adapters only)
• —Adapter checkpoint size: ~100–200 MB

Evaluation

Testing Data, Factors & Metrics

Testing Data

• —Hand-written Luau prompts
• —Roblox-specific scripting scenarios

Factors

• —Luau syntax correctness
• —Roblox API familiarity
• —Instruction-following behavior

Metrics

• —Qualitative human evaluation
• —Manual code review and comparison with base model

Results

The LoRA adapter demonstrates improved performance over the base model in:

• —Generating idiomatic Luau
• —Correct Roblox service usage
• —Following game-development-oriented instructions

Summary

The model performs best when used as a Roblox development assistant and is not intended for general-purpose natural language tasks.

Model Examination

No formal interpretability or probing analysis was conducted.

Environmental Impact

Carbon emissions were not formally measured.

• —Hardware Type: Consumer-grade GPU
• —Hours used: < 24 hours
• —Cloud Provider: None (local training)
• —Compute Region: Not applicable
• —Carbon Emitted: Not estimated

Technical Specifications

Model Architecture and Objective

• —Decoder-only Transformer
• —Next-token prediction objective
• —LoRA adapters applied to attention layers

Compute Infrastructure

Hardware

• —Single consumer-grade GPU

Software

• —PyTorch
• —Transformers
• —PEFT

Citation

BibTeX:

@misc{darwinkernelpanic2025luau,
  title={CodeLlama 7B Instruct Luau LoRA},
  author={darwinkernelpanic},
  year={2025},
  howpublished={Hugging Face},
  note={LoRA fine-tuned for Luau / Roblox scripting}
}

APA:

darwinkernelpanic. (2025). CodeLlama 7B Instruct Luau LoRA. Hugging Face.

Model Card Authors

darwinkernelpanic

Model Card Contact

Use the Hugging Face repository issues or the author’s profile.

Framework versions

• —PEFT 0.18.0