Datasetpublic

Emulated-Inc/procedural-reasoning-training-pool

Procedural reasoning training pool Reasoning questions from 101 procedural generators, each of which writes a question, computes its own answer and ships a verifier that scores an attempt at it, plus a collection of solved Sudoku puzzles. Every answer is short and exactly checkable, so a trained model can be marked against the key by a program and no judge is needed. Laid out twice. Train on either layer or on both. pool.jsonl Every generator rewritten into one… See the full description on the dataset page: https://huggingface.co/datasets/Emulated-Inc/procedural-reasoning-training-pool.

sourceHugging Facecc-by-4.0updated 28d agoView on Hugging Face

0likes257downloads

Dataset Card

Procedural reasoning training pool

Reasoning questions from 101 procedural generators, each of which writes a question, computes its own answer and ships a verifier that scores an attempt at it, plus a collection of solved Sudoku puzzles. Every answer is short and exactly checkable, so a trained model can be marked against the key by a program and no judge is needed. Laid out twice. Train on either layer or on both.

`pool.jsonl`

Every generator rewritten into one shape, 161495 rows, one JSON object per line, with these fields.

Field	What it holds
`id`	a row identifier unique within this file
`generator`	the generator that wrote the row, one of those listed below
`family`	the package of the library the generator belongs to
`part`	`core` for the twenty drawn deeply, `wider` for the rest, `collected` for the downloaded puzzles
`question`	the question, exactly as its generator wrote it
`answer`	the generator's own answer, or `null`
`worked_answer`	a worked answer or an accepted answer where the generator records one
`worked_answer_field`	which of its metadata fields that came from
`metadata`	the generator's own record for the row, which its verifier reads
`source`	the directory under `sources/` the row came from
`source_seed`	the seed the row was drawn under
`provenance_class`	how the row came to exist
`licence`	the licence of the part it came from

158999 rows carry an answer and 70182 carry a worked answer. The answer is the generator's own answer field. Five generators write no answer at all, because any response that satisfies the puzzle is correct and their verifier checks the response rather than compares it. Three of them record an acceptable answer in their metadata, which is what worked_answer holds for those rows, and 1001 rows hold neither an answer nor a worked answer and are a prompt with a verifier and no key. The verifier for a row is the generator named in its generator field, reached through the library at version 0.1.25: create that generator and hand it the answer together with the row's own question, answer and metadata. Several verifiers give partial credit for a near miss, so read the score rather than only its sign. Every shipped key was read by the verifier of the generator that wrote the row, exactly as that verifier reads a model's answer, and a row whose own key it did not score at 1.0 is not here. 6 rows were dropped that way.

Worth knowing about that version. The library's own __version__ string still reads 0.1.19 in this release, so the distribution version is what is pinned. Install reasoning-gym==0.1.25 and read the version back from the distribution metadata rather than from the module.

Rows are written one generator after another. One row per canonical question within a generator, the canonical form being the question with runs of whitespace collapsed. Repeats are skipped during the draw rather than removed afterwards. Within a generator a question appears once.

`sources/`

The same items untouched, 776495 rows. The library's own entries as drawn, one jsonl file per generator under the directory of the library package it belongs to, with only the dropped rows taken out, and the downloaded collection under its own directory in the parquet files and with the columns its repository publishes. Nothing here was renamed, reshaped, reordered or deduplicated, so use this layer if you want to pose the questions yourself, or want a column the rewritten rows drop, such as the printed grid the Sudoku collection carries beside its flat one.

The generators

20 generators were drawn at 5000 items each under seed 40260912, and 81 at 500 items each under seed 50260912. A generator whose question space is narrower than the count asked for contributed what it had, and no question on the held-out list named under Filtering comes from a generator of the second group. Rows in the rewritten layer per generator, with the package it belongs to.

Generator	Family	Part	Rows	With an answer	With a worked answer
`basic_arithmetic`	`arithmetic`	`core`	4975	4975	4975
`calendar_arithmetic`	`arithmetic`	`core`	4985	4985	0
`chain_sum`	`arithmetic`	`core`	4994	4994	4994
`gcd`	`arithmetic`	`core`	4998	4998	0
`time_intervals`	`arithmetic`	`core`	5000	5000	0
`complex_arithmetic`	`algebra`	`core`	4959	4959	0
`polynomial_multiplication`	`algebra`	`core`	5000	5000	0
`simple_equations`	`algebra`	`core`	5000	5000	0
`aiw`	`logic`	`core`	4940	4940	0
`knights_knaves`	`logic`	`core`	5000	5000	5000
`self_reference`	`logic`	`core`	2698	2698	0
`family_relationships`	`graphs`	`core`	5000	5000	0
`shortest_path`	`graphs`	`core`	5000	5000	3174
`mini_sudoku`	`games`	`core`	4999	4999	4999
`color_cube_rotation`	`cognition`	`core`	5000	5000	0
`number_sequence`	`cognition`	`core`	4711	4711	4711
`palindrome_generation`	`algorithmic`	`core`	4995	4995	0
`sentence_reordering`	`algorithmic`	`core`	4974	4974	0
`spell_backward`	`algorithmic`	`core`	4993	4993	0
`word_sequence_reversal`	`algorithmic`	`core`	5000	5000	0
`ab`	`algorithmic`	`wider`	500	500	0
`acre`	`induction`	`wider`	500	500	0
`advanced_geometry`	`geometry`	`wider`	500	500	0
`base_conversion`	`algorithmic`	`wider`	499	499	0
`bf`	`code`	`wider`	500	500	0
`binary_alternation`	`algorithmic`	`wider`	500	500	500
`binary_matrix`	`algorithmic`	`wider`	500	500	500
`bitwise_arithmetic`	`arithmetic`	`wider`	500	500	0
`boxnet`	`games`	`wider`	500	0	0
`caesar_cipher`	`algorithmic`	`wider`	500	500	0
`circuit_logic`	`logic`	`wider`	500	500	500
`coin_flip`	`probability`	`wider`	258	258	0
`count_bits`	`arithmetic`	`wider`	500	500	500
`count_primes`	`algorithmic`	`wider`	500	500	500
`countdown`	`games`	`wider`	500	500	500
`course_schedule`	`graphs`	`wider`	500	500	500
`cryptarithm`	`algorithmic`	`wider`	500	500	0
`decimal_arithmetic`	`arithmetic`	`wider`	500	500	0
`decimal_chain_sum`	`arithmetic`	`wider`	500	500	500
`dice`	`arithmetic`	`wider`	500	500	0
`emoji_mystery`	`games`	`wider`	500	500	0
`figlet_font`	`cognition`	`wider`	500	500	0
`fraction_simplification`	`arithmetic`	`wider`	500	500	0
`futoshiki`	`games`	`wider`	500	500	500
`game_of_life`	`algorithmic`	`wider`	500	500	0
`game_of_life_halting`	`algorithmic`	`wider`	500	500	0
`graph_color`	`algorithmic`	`wider`	500	0	500
`group_anagrams`	`algorithmic`	`wider`	500	500	500
`gsm_symbolic`	`arithmetic`	`wider`	499	499	0
`intermediate_integration`	`algebra`	`wider`	500	500	0
`isomorphic_strings`	`algorithmic`	`wider`	500	500	500
`jugs`	`algorithmic`	`wider`	500	500	0
`kakurasu`	`games`	`wider`	500	500	500
`knight_swap`	`games`	`wider`	500	500	402
`largest_island`	`graphs`	`wider`	500	500	500
`lcm`	`arithmetic`	`wider`	500	500	0
`leg_counting`	`arithmetic`	`wider`	500	500	0
`letter_counting`	`algorithmic`	`wider`	500	500	0
`letter_jumble`	`algorithmic`	`wider`	500	500	0
`list_functions`	`induction`	`wider`	500	500	0
`mahjong_puzzle`	`games`	`wider`	500	500	500
`manipulate_matrix`	`algorithmic`	`wider`	500	500	500
`maze`	`games`	`wider`	500	500	0
`modulo_grid`	`cognition`	`wider`	500	500	0
`n_queens`	`games`	`wider`	500	500	500
`needle_haystack`	`cognition`	`wider`	500	500	0
`number_filtering`	`algorithmic`	`wider`	500	500	0
`number_format`	`arithmetic`	`wider`	500	500	500
`number_sorting`	`algorithmic`	`wider`	500	500	0
`palindrome_partitioning`	`algorithmic`	`wider`	500	500	500
`path_star`	`graphs`	`wider`	500	500	0
`polynomial_equations`	`algebra`	`wider`	500	500	0
`pool_matrix`	`algorithmic`	`wider`	500	500	500
`power_function`	`arithmetic`	`wider`	500	500	500
`prime_factorization`	`arithmetic`	`wider`	500	500	0
`products`	`arithmetic`	`wider`	500	500	500
`propositional_logic`	`logic`	`wider`	496	0	496
`puzzle24`	`games`	`wider`	500	500	0
`quantum_lock`	`graphs`	`wider`	500	500	0
`ransom_note`	`algorithmic`	`wider`	500	500	500
`rectangle_count`	`cognition`	`wider`	500	500	500
`rotate_matrix`	`algorithmic`	`wider`	500	500	500
`rotten_oranges`	`algorithmic`	`wider`	500	500	500
`rubiks_cube`	`cognition`	`wider`	500	0	499
`rush_hour`	`games`	`wider`	500	0	0
`simple_geometry`	`geometry`	`wider`	500	500	0
`simple_integration`	`algebra`	`wider`	500	500	0
`sokoban`	`games`	`wider`	500	500	0
`spiral_matrix`	`algorithmic`	`wider`	500	500	500
`string_insertion`	`algorithmic`	`wider`	500	500	500
`string_manipulation`	`algorithmic`	`wider`	500	500	500
`string_splitting`	`algorithmic`	`wider`	216	216	216
`string_synthesis`	`algorithmic`	`wider`	216	216	216
`sudoku`	`games`	`wider`	500	500	500
`survo`	`games`	`wider`	500	500	500
`syllogism`	`logic`	`wider`	500	500	0
`tower_of_hanoi`	`games`	`wider`	90	90	0
`tsumego`	`games`	`wider`	500	500	0
`word_ladder`	`algorithmic`	`wider`	500	500	0
`word_sorting`	`algorithmic`	`wider`	500	500	0
`zebra_puzzles`	`logic`	`wider`	500	500	0
`sudoku`	`sudoku_puzzles`	`collected`	25000	25000	25000

The sources

`sources/algebra`

The 6 generators the library keeps in its algebra package, drawn from version 0.1.25 of the reasoning-gym library under seeds 40260912, 50260912. 16459 rows in 6 files, all of which also appear in pool.jsonl. Provenance class rule-generated, licence cc-by-4.0. Every line is one of the library's own entries, whose keys are question, answer and metadata.

`sources/algorithmic`

The 34 generators the library keeps in its algorithmic package, drawn from version 0.1.25 of the reasoning-gym library under seeds 40260912, 50260912. 34393 rows in 34 files, all of which also appear in pool.jsonl. Provenance class rule-generated, licence cc-by-4.0. Every line is one of the library's own entries, whose keys are question, answer and metadata.

`sources/arithmetic`

The 18 generators the library keeps in its arithmetic package, drawn from version 0.1.25 of the reasoning-gym library under seeds 40260912, 50260912. 31451 rows in 18 files, all of which also appear in pool.jsonl. Provenance class rule-generated, licence cc-by-4.0. Every line is one of the library's own entries, whose keys are question, answer and metadata.

`sources/code`

The 1 generator the library keeps in its code package, drawn from version 0.1.25 of the reasoning-gym library under seed 50260912. 500 rows in 1 file, all of which also appear in pool.jsonl. Provenance class rule-generated, licence cc-by-4.0. Every line is one of the library's own entries, whose keys are question, answer and metadata.

`sources/cognition`

The 7 generators the library keeps in its cognition package, drawn from version 0.1.25 of the reasoning-gym library under seeds 40260912, 50260912. 12211 rows in 7 files, all of which also appear in pool.jsonl. Provenance class rule-generated, licence cc-by-4.0. Every line is one of the library's own entries, whose keys are question, answer and metadata.

`sources/games`

The 17 generators the library keeps in its games package, drawn from version 0.1.25 of the reasoning-gym library under seeds 40260912, 50260912. 12589 rows in 17 files, all of which also appear in pool.jsonl. Provenance class rule-generated, licence cc-by-4.0. Every line is one of the library's own entries, whose keys are question, answer and metadata.

`sources/geometry`

The 2 generators the library keeps in its geometry package, drawn from version 0.1.25 of the reasoning-gym library under seed 50260912. 1000 rows in 2 files, all of which also appear in pool.jsonl. Provenance class rule-generated, licence cc-by-4.0. Every line is one of the library's own entries, whose keys are question, answer and metadata.

`sources/graphs`

The 6 generators the library keeps in its graphs package, drawn from version 0.1.25 of the reasoning-gym library under seeds 40260912, 50260912. 12000 rows in 6 files, all of which also appear in pool.jsonl. Provenance class rule-generated, licence cc-by-4.0. Every line is one of the library's own entries, whose keys are question, answer and metadata.

`sources/induction`

The 2 generators the library keeps in its induction package, drawn from version 0.1.25 of the reasoning-gym library under seed 50260912. 1000 rows in 2 files, all of which also appear in pool.jsonl. Provenance class rule-generated, licence cc-by-4.0. Every line is one of the library's own entries, whose keys are question, answer and metadata.

`sources/logic`

The 7 generators the library keeps in its logic package, drawn from version 0.1.25 of the reasoning-gym library under seeds 40260912, 50260912. 14634 rows in 7 files, all of which also appear in pool.jsonl. Provenance class rule-generated, licence cc-by-4.0. Every line is one of the library's own entries, whose keys are question, answer and metadata.

`sources/probability`

The 1 generator the library keeps in its probability package, drawn from version 0.1.25 of the reasoning-gym library under seed 50260912. 258 rows in 1 file, all of which also appear in pool.jsonl. Provenance class rule-generated, licence cc-by-4.0. Every line is one of the library's own entries, whose keys are question, answer and metadata.

`sources/sudoku_puzzles`

A published collection of 9x9 Sudoku puzzles with their solutions, in five files of falling clue counts from 63 down to 17. From abatilo/sudokubench at revision 8efe5b22131f4d9720126822e910cf11bfb347db, files kids.parquet, easy.parquet, medium.parquet, hard.parquet, insane.parquet. 640000 rows here, of which 25000 also appear in pool.jsonl: 5000 rows of each file, drawn under seed 40260912, rewritten into the shape the library's own Sudoku generator uses so that the same verifier scores them. Provenance class rule-generated, licence apache-2.0. Its own fields are startingcells, puzzle, puzzlepretty, solution, solution_pretty.

Worth knowing. The puzzles were made by a program, like the drawn rows, so the class is the same. What the collection adds is a wider clue range and a much larger stock of puzzles than a draw of one generator, in its own files

Provenance and licences

Every row here is rule-generated, which is the honest headline for this pool. A program wrote each question from a template over values it drew, computed the answer as it built the item, and ships the code that checks an attempt. No row was written by a person and no row was written by a language model.

Worth knowing before training on this. A model trained on the output of a generator can learn the generator rather than the rule behind it, and here the generator is what made every item. The generators are public and unbounded, so more rows of any of them can be drawn at any seed, and the value of this file is that it is a fixed, decontaminated, verifiable draw rather than that it is the only one available.

The pool as a whole is offered under cc-by-4.0. The drawn rows come out of the reasoning-gym library (github.com/open-thought/reasoning-gym), which is released under apache-2.0 and is credited here as the author of every question and every verifier. The Sudoku collection is apache-2.0. Each row carries its own licence in the licence field and each directory under sources/ is one part, so a subset under a single licence can be selected.

Filtering

391 rows were removed before publication, from both layers alike, because their question repeated a question on a held-out list word for word once whitespace was collapsed, and 191 more because one single held-out question accounted for four fifths or more of the row's word 8-grams. The list is not distributed here. 6 further rows went for the reason given above, their own verifier not accepting their own key. Nothing else was filtered: no generator, no length and no difficulty was selected for or against, so every generator contributes as many rows as its draw left, and the generator and family fields are how to select what you want.