Spreadsheet-RL/Spreadsheet-RL-8B

Spreadsheet-RL-8B

<p align="center"> <img src="spreadsheet-rl.png" alt="Spreadsheet-RL logo" width="700"> </p>

**Project Page** | **Paper** | **Dataset** | **Code**

Spreadsheet-RL-8B is the RL-trained 8B spreadsheet agent checkpoint from Spreadsheet-RL: Advancing Large Language Model Agents on Realistic Spreadsheet Tasks via Reinforcement Learning. It starts from `Qwen/Qwen3-8B` and is post-trained with outcome-based reinforcement learning in Spreadsheet Gym, a multi-turn Microsoft Excel environment with spreadsheet-native tools, sandboxed code execution, and Excel-based recalculation rewards.

This checkpoint is intended to be used with the Spreadsheet-RL agent harness and tool environment. Loading it as a plain chat model can be useful for inspection, but it will not reproduce the reported results without Spreadsheet Gym, the tool set, and the reward/evaluation pipeline.

News

• —2026-08-01: Released the Spreadsheet-RL-8B model checkpoint on Hugging Face at `Spreadsheet-RL/Spreadsheet-RL-8B`.

Model Details

Training Configuration

For full details, please see the paper. The released 8B checkpoint uses:

Results

Spreadsheet-RL improves Qwen3-8B through spreadsheet-native interaction design, comprehensive tool access, and RL post-training.

The released step-35 checkpoint is the best observed checkpoint and improves the full-harness pre-RL result by 5.6 percentage points.

The corresponding 4B release starts from `Qwen/Qwen3-4B-Thinking-2507`. Its 2507 checkpoint postdates the original Qwen3 series, including Qwen/Qwen3-8B, so model size and base-checkpoint vintage are confounded. Under the full Spreadsheet-RL harness, the 8B model trails the 4B model both before RL (16.7 versus 19.3) and after RL (22.3 versus 23.4); this should not be interpreted as a clean size-only comparison. Instead, the 8B gain from 16.7 to 22.3 shows that Spreadsheet-RL remains effective at a larger model scale, and we release the post-RL checkpoint to make that scaling result reproducible.

Usage

Install the standard Transformers stack and load the checkpoint:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Spreadsheet-RL/Spreadsheet-RL-8B"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

For task evaluation and agent rollouts, use the full Spreadsheet-RL codebase with the released dataset and Spreadsheet Gym:

hf download Spreadsheet-RL/Spreadsheet-RL --repo-type dataset --local-dir data
git clone https://github.com/Spreadsheet-RL/Spreadsheet-RL.git

The default training/evaluation harness is maintained in the code repository under configs/, scripts/, reward/, and verl/.

Citation

@misc{chi2026spreadsheetrl,
  title         = {Spreadsheet-RL: Advancing Large Language Model Agents on Realistic Spreadsheet Tasks via Reinforcement Learning},
  author        = {Banghao Chi and Yining Xie and Mingyuan Wu and Jingcheng Yang and Jize Jiang and Zhaoheng Li and Shengyi Qian and Minjia Zhang and Klara Nahrstedt and Rui Hou and Xiangjun Fan and Hanchao Yu},
  year          = {2026},
  eprint        = {2605.22642},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  doi           = {10.48550/arXiv.2605.22642},
  url           = {https://arxiv.org/abs/2605.22642}
}