FineEnvs/SmolDataEnvs-sft

<div align="center">

<img src="https://huggingface.co/datasets/FineEnvs/SmolDataEnvs-sft/resolve/main/banner.png" alt="SmolDataEnvs" width="100%">

🛠️ SmolDataEnvs: SFT


</div>

5.5K+ RL tasks for hill-climbing small models in code and data science.

<div align="center">

<img src="https://huggingface.co/datasets/FineEnvs/SmolDataEnvs-sft/resolve/main/curves.gif" alt="Reward and held-out pass@k climbing over 1,119 GRPO steps" width="100%">

_{A 2B model on these tasks. Left: what it optimises. Right: 144 held-out tasks it never trains on.<br> Two runs over the same 5,000 tasks: <b>shuffled</b> against a <b>curriculum</b> ordered easiest to hardest.}

</div>

4,677 worked examples of an agent doing data science the right way. Each row is a complete, verified-correct trajectory: read the question, poke at the data with a shell tool, reason, compute, write the answer. Every one of them solved its task and passed the deterministic grader, so you are fine-tuning on demonstrations that are known to be correct rather than merely plausible.

Drop-in ready for TRL: conversational messages plus tools.

What's inside

• —4,677 correct trajectories, one per task
• —Difficulty: easy 1,402 · medium 2,640 · hard 635
• —One tool throughout: bash

What's in a row

• —`messages`: the full conversation in OpenAI/TRL chat format: system → user (the task) → assistant (reasoning plus tool_calls) → tool (command output) → … → final assistant answer
• —`tools`: the bash tool's JSON schema, for apply_chat_template(..., tools=...)
• —task_id, difficulty (1–5), difficulty_tier, n_turns, source_agent

Fine-tune with TRL

from datasets import load_dataset
from trl import SFTConfig, SFTTrainer

ds = load_dataset("FineEnvs/SmolDataEnvs-sft", split="train")
trainer = SFTTrainer(
    model="HuggingFaceTB/SmolLM3-3B",
    train_dataset=ds,
    args=SFTConfig(output_dir="smoldataenvs-sft", max_length=8192),
)
trainer.train()

A runnable notebook and a single-file script for HF Jobs are in FineEnvs/04-smoldataenvs.

Where it comes from

Built from the jupyter-agent dataset, real data-science notebooks over 471 Kaggle datasets. Every question–answer pair was extracted and then verified: strong agent models had to solve the task in a live sandbox and reproduce the gold answer under deterministic grading. Anything ambiguous or un-checkable was dropped. So every task here is known-solvable and unambiguously gradable.

Verified by a checker, not judged by a model. Grading is an exact comparison against a known answer, through a ladder of checks: exact match → numeric with tolerances → list and percent normalisation → symbolic equivalence. No LLM sits in the reward path, so the signal does not drift when you change the grader's model, because there isn't one.

The family

Citation

@misc{fineenvs,
  author = {Kolavi, Adithya S},
  title  = {FineEnvs: Open Source RL Environments for LLM Agents},
  year   = {2026},
  url    = {https://github.com/adithya-s-k/FineEnvs}
}