Datasetpublic

VIRUS374/ai-developer-dataset

AI Developer Dataset A large-scale instruction-tuning dataset for fine-tuning an open-weight LLM into a universal AI developer assistant. The model trained on this dataset should be especially good at: PROGRAMMING + WEB DEVELOPMENT + UI/UX + ANIMATIONS + BOTS + SCRIPTS + AUTOMATION + BACKEND + API + DATABASES + DEBUGGING + LINUX + DEPLOYMENT + AI DEVELOPMENT + SECURITY. 📊 Statistics Total examples: 1,124,699 File size: 2.13 GB Format: JSONL (conversational)… See the full description on the dataset page: https://huggingface.co/datasets/VIRUS374/ai-developer-dataset.

sourceHugging Facemitupdated 5d agoView on Hugging Face

0likes52downloads

Dataset Card

AI Developer Dataset

A large-scale instruction-tuning dataset for fine-tuning an open-weight LLM into a universal AI developer assistant.

The model trained on this dataset should be especially good at: PROGRAMMING + WEB DEVELOPMENT + UI/UX + ANIMATIONS + BOTS + SCRIPTS + AUTOMATION + BACKEND + API + DATABASES + DEBUGGING + LINUX + DEPLOYMENT + AI DEVELOPMENT + SECURITY.

📊 Statistics

—Total examples: 1,124,699
—File size: 2.13 GB
—Format: JSONL (conversational)
—Average assistant length: ~1,222 characters
—Schema: {"messages":[{"role":"system|user|assistant","content":"..."}, ...], "metadata":{"category","subcategory","complexity","source"}}

📚 Sources

This dataset is a curated merge of high-quality open-source code instruction datasets from HuggingFace, plus handcrafted examples in Russian covering UI/UX, security, and other specialized topics.

Source	Examples
`hf:glaiveai/glaive-code-assistant-v2`	213,670
`hf:glaiveai/glaive-code-assistant`	135,195
`hf:mwitiderrick/glaive-code-assistant`	104,336
`hf:b-mc2/sql-create-context`	78,573
`hf:TokenBender/code_instructions_122k_alpaca_style`	78,433
`hf:iamtarun/code_instructions_120k_alpaca`	78,433
`hf:ajibawa-2023/Code-74k-ShareGPT`	68,790
`hf:Locutusque/code-feedback-sharegpt`	62,866
`hf:m-a-p/Code-Feedback`	62,866
`hf:HuggingFaceH4/Code-Feedback`	61,923
`hf:WizardLMTeam/WizardLM_evol_instruct_70k`	60,411
`hf:nickrosh/Evol-Instruct-Code-80k-v1`	58,265
`hf:Data-Agora/magpie_code_gpt4o_mini_50000`	36,818
`hf:ajibawa-2023/Python-Code-23k-ShareGPT`	17,737
`hf:HuggingFaceH4/CodeAlpaca_20K`	6,307
`handcrafted`	76

📁 Categories

Category	Examples	%
programming	1,046,060	93.0%
backendapidb	78,579	7.0%
uiuxdesign	10	0.0%
web_development	9	0.0%
bots_automation	8	0.0%
security	8	0.0%
ai_ml	7	0.0%
debugging	6	0.0%
linuxserverdeploy	6	0.0%
websearchdocs	6	0.0%

🎯 Complexity Distribution

Level	Examples	%
intermediate	686,519	61.0%
advanced	438,166	39.0%
beginner	12	0.0%
expert	2	0.0%

📋 Schema

Each line is a single JSON object:

json

{
  "messages": [
    {"role": "system", "content": "Category: programming\nSubcategory: code_alpaca\n..."},
    {"role": "user", "content": "Write a Python function to..."},
    {"role": "assistant", "content": "Here is the implementation..."}
  ],
  "metadata": {
    "category": "programming",
    "subcategory": "code_alpaca",
    "complexity": "intermediate",
    "source": "hf:HuggingFaceH4/CodeAlpaca_20K"
  }
}

🛠️ Quality Control

Every example passes:

—JSON-serializable (validated on 100% of examples)
—Non-empty user and assistant content
—Assistant length between 200 and 25,000 characters
—Per-source deduplication by user-content fingerprint (SHA-256, first 16 hex chars)
—Schema validation (messages + metadata required)

🚀 Usage

python

from datasets import load_dataset

# Load the full dataset
ds = load_dataset("VIRUS374/ai-developer-dataset", data_files="dataset.jsonl", split="train")
print(ds)
# Dataset({
#     features: ["messages", "metadata"],
#     num_rows: 1,124,699
# })

# Or use streaming for memory-efficient iteration
ds = load_dataset("VIRUS374/ai-developer-dataset", data_files="dataset.jsonl", split="train", streaming=True)
for example in ds:
    messages = example["messages"]
    # ... feed to tokenizer with chat template

🏋️ Fine-tuning Recipe (QLoRA)

python

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
from trl import SFTTrainer
from datasets import load_dataset
import torch

MODEL = "meta-llama/Llama-3.1-8B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(MODEL)
tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(
    MODEL,
    torch_dtype=torch.bfloat16,
    load_in_4bit=True,
    device_map="auto",
)
model = prepare_model_for_kbit_training(model)

lora_config = LoraConfig(
    r=16, lora_alpha=32,
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
    lora_dropout=0.05, bias="none", task_type="CAUSAL_LM",
)
model = get_peft_model(model, lora_config)

ds = load_dataset("VIRUS374/ai-developer-dataset", data_files="dataset.jsonl", split="train")

def format_example(example):
    text = tokenizer.apply_chat_template(
        example["messages"], tokenize=False, add_generation_prompt=False,
    )
    return {"text": text}

ds = ds.map(format_example, remove_columns=ds.column_names)

trainer = SFTTrainer(
    model=model,
    train_dataset=ds,
    args=TrainingArguments(
        output_dir="./output",
        num_train_epochs=3,
        per_device_train_batch_size=2,
        gradient_accumulation_steps=8,
        warmup_steps=50,
        learning_rate=2e-4,
        bf16=True,
        logging_steps=10,
        save_steps=500,
    ),
    max_seq_length=4096,
)
trainer.train()

📜 License & Attribution

This dataset aggregates multiple open-source datasets. Each retains its original license:

—HuggingFaceH4/CodeAlpaca_20K — Apache 2.0
—HuggingFaceH4/Code-Feedback — Apache 2.0
—nickrosh/Evol-Instruct-Code-80k-v1 — Apache 2.0
—WizardLMTeam/WizardLM_evol_instruct_70k — Apache 2.0 (Non-commercial)
—TokenBender/code_instructions_122k_alpaca_style — Apache 2.0
—iamtarun/code_instructions_120k_alpaca — Apache 2.0
—m-a-p/Code-Feedback — CC BY-NC 4.0
—Locutusque/code-feedback-sharegpt — Apache 2.0
—ajibawa-2023/Code-74k-ShareGPT — Apache 2.0
—ajibawa-2023/Python-Code-23k-ShareGPT — Apache 2.0
—b-mc2/sql-create-context — CC BY 4.0
—glaiveai/glaive-code-assistant — Apache 2.0
—glaiveai/glaive-code-assistant-v2 — Apache 2.0
—mwitiderrick/glaive-code-assistant — Apache 2.0
—Data-Agora/magpie_code_gpt4o_mini_50000 — Apache 2.0
—Handcrafted examples (VIRUS374) — MIT

Important: If you use this dataset for commercial purposes, please review the WizardLM and m-a-p licenses — they may have non-commercial restrictions.

🙏 Acknowledgements

Thanks to all the original dataset creators:

—HuggingFace H4 team
—WizardLM team
—TokenBender
—iamtarun
—m-a-p
—Locutusque
—ajibawa-2023
—b-mc2
—glaiveai
—mwitiderrick
—Data-Agora

📝 Citation

bibtex

@dataset{virus374_ai_developer_2024,
  title  = {AI Developer Dataset},
  author = {VIRUS374},
  year   = {2024},
  url    = {https://huggingface.co/datasets/VIRUS374/ai-developer-dataset},
  note   = {Aggregated from open-source code instruction datasets + handcrafted examples}
}