FSDP
Datasets
All datasets matching “FSDP”FSD-PT-PleIAs-SYNTH-15M-EN-NonMem-Mem
FSD-PT PleIAs SYNTH 15M English Subset
A 15M-document English-only subset of PleIAs/SYNTH for pretraining small reasoning models.
Source
Original dataset: PleIAs/SYNTH (79.6M samples, 41B words, CC-BY-4.0).
Subsetting Method
Filtered to language == "en" only
Round-robin sampled across exercise types for diversity
Final count: 15,000,000 documents
Columns
Field
Type
Description
query
string
Input query/prompt
synthetic_reasoning
string… See the full description on the dataset page: https://huggingface.co/datasets/AmanPriyanshu/FSD-PT-PleIAs-SYNTH-15M-EN-NonMem-Mem.ScaleSeek-GRPOv3-fsdp-state
ScaleSeek GRPOv3 FSDP training state
verl FSDP shards for the v3.1 GRPO run, steps 160 and 300, carrying the AdamW moments and
dataloader position that merged HF weights do not. Shards are written per world size, so they
load only on 4 GPUs. Use them to continue that exact run:
huggingface-cli download Allenda/ScaleSeek-GRPOv3-fsdp-state --repo-type dataset \
--include "global_step_160/*" --local-dir $RL_CKPT_DIR/<run>/
echo 160 >… See the full description on the dataset page: https://huggingface.co/datasets/Allenda/ScaleSeek-GRPOv3-fsdp-state.llama_4_fsdptt-x10-fsdp2-fa2llama-fsdp-hw-statsqwen3-0.6b-fsdp2
