scottfitz/text-to-sql-recipe-data

Text-to-SQL Recipe Data

The datasets the River text-to-SQL reinforcement-learning recipe trains and evaluates on, laid out the way its setup data command reads them. The recipe checks every file against SHA-256 pins in its code, so use the command rather than copying files by hand. From the recipe's directory (the text-to-sql example of River Recipes):

pip install '.[data]'
eval "$(python -m text_to_sql_recipe setup data)"

Contents

`revisql-verified/`: BIRD-Platinum training data.

• —dataset/: the ReViSQL authors' verified release of BIRD training tasks, with 2,064 training and 398 validation tasks. Each row carries the question, evidence, gold SQL, grading method and the task text the model sees (the database schema, the evidence and the question).
• —hints/: one training hint per training task, with a binding to the exact prompt, gold query and database it was written for. The hints were written by Kimi K2.6 at temperature 0 from each task's question, evidence, schema and gold query.
• —verifier/gate-policy.json: per database, the schema and the audited foreign-key status that the optional VeriEQL counterexample gate reads.
• —subsets/bird-platinum-hard.json: the 689 BIRD-Platinum-Hard training tasks, the questions strong models most often failed on every attempt early in reinforcement-learning training (see scottfitz/BIRD-Platinum-Hard).
• —databases/: BIRD's 69 training databases, unchanged.

`arcwise/`: Arcwise-Plat-SQL evaluation data.

• —498 evaluation tasks over the 11 BIRD Mini-Dev databases (databases/).
• —The gold SQL is the edition the ReViSQL paper evaluated (uiuc-kang-lab/text_to_sql_benchmarks at commit bef1f2e42b).
• —Questions 518 and 701 use result-preserving rewrites of their gold queries, because the canonical queries exceed the grading time limit.

`release.json`: every file with its size and SHA-256.

Sources and licences

This release is distributed under CC BY-SA 4.0, the licence of the data it derives from.

• —BIRD (Li et al., NeurIPS 2023): the databases, questions and evidence. CC BY-SA 4.0.
• —BIRD-Platinum (uiuc-kang-lab/bird-platinum): corrected BIRD training annotations. CC BY-SA 4.0. The verified task release used here is ReViSQL's (Jin et al., arXiv 2603.20004).
• —Arcwise-Plat-SQL: Jin, Choi, Zhu and Kang, "Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards" (VLDB 2026), uiuc-kang-lab/text_to_sql_benchmarks. CC BY-SA 4.0. It builds on Arcwise's corrections of BIRD Mini-Dev.
• —Training hints and the gate policy: ours, under CC BY-SA 4.0.