trl-lab/SQaLe-2-text-to-SQL-Queries

<h1 align="center">SQaLe: questions and SQL</h1>

<p align="center"> <a href="https://trl-lab.github.io/sqale"><b>Project page</b></a> · <a href="https://huggingface.co/datasets/trl-lab/SQaLe-2-text-to-SQL-Schemas"><b>Schemas and databases</b></a> · <a href="https://huggingface.co/collections/trl-lab/sqale-project"><b>Trained models</b></a> · <a href="https://pypi.org/project/SQaLe/"><b>Python library</b></a> · <a href="#citation"><b>Citation</b></a> </p>

SQaLe is a large semi-synthetic text-to-SQL dataset grounded in real-world database schemas, introduced in the paper SQaLe: a large realistic dataset to empower small specialised text-to-SQL models. It pairs 1,408,056 natural-language questions with 176,761 distinct SQL queries over 9,259 populated SQLite databases. The schemas come from SchemaPile, a collection of database schemas extracted from GitHub, and are extended to realistic sizes, with a median of 113 tables and 538 columns per schema. Every gold query was executed against its populated database and accepted by an LLM judge before it entered the corpus.

SQaLe is split across two datasets that join on schema_id:

Quickstart

The `SQaLe` Python library (source on GitHub) reads both datasets and writes the databases as SQLite files, so loading the questions and building their databases take one call each.

pip install "SQaLe>=0.2"

import sqlite3
from sqale import deserialize_sqale, load_questions

questions = load_questions(split="test", limit=100)
databases = deserialize_sqale(
    split="test",
    output_dir="./dbs",
    schema_ids={q["schema_id"] for q in questions},
)
db_path = {d["schema_id"]: d["db_path"] for d in databases}

q = questions[0]
conn = sqlite3.connect(db_path[q["schema_id"]])
print(q["questions"]["verbose"])
print(conn.execute(q["sql"]).fetchmany(5))

load_questions returns one dict per question with the fields listed under Fields, already parsed, so relevant_tables and execution_result are lists rather than JSON strings. It filters by split and difficulty, and it replaces the placeholder phrasings described under Known issues with None unless you pass drop_placeholders=False. deserialize_sqale writes one .db file per database, and with schema_ids only the databases you ask for. It downloads the split's parquet shards one at a time into the Hugging Face cache, so a run that stops early downloads only the shards it reaches. The train split of the databases is about 4 GB.

To train on every phrasing, flatten the questions into (question, SQL) pairs:

questions = load_questions(split="train")
pairs = [(text, q["sql"]) for q in questions for text in q["questions"].values() if text]

This keeps 1,260,936 pairs from train and 1,320,691 over both splits.

The library also installs a command-line tool that writes databases directly:

sqale-extract --split test --output ./dbs
sqale-extract --split test --output ./dbs --schema-id schema_012721

The parquet files also load directly with load_dataset("trl-lab/SQaLe-2-text-to-SQL-Queries") from the datasets library, with relevant_tables and execution_result as JSON strings.

<p align="center"> <img src="figures/pipeline.png" width="900" alt="The SQaLe generation pipeline"> </p> <p align="center">_{The generation pipeline: schema extension, table value synthesis, question generation, SQL generation with an exploring agent and an LLM judge, and reformulation into seven further styles. Figure from the paper.}</p>

At a glance

Example

From the test split (schema_012721_s3_q5_0447519808, difficulty moderate). The schema behind it has 192 tables, and the question was generated from a nine-table subschema.

SELECT pas.planet_id, pas.alert_id, COUNT(phe.event_id) AS total_historical_events
FROM planet_alert_status pas
LEFT JOIN planet_historical_events phe ON pas.planet_id = phe.planet_id
WHERE pas.alert_type = 'Trading Halt'
GROUP BY pas.planet_id, pas.alert_id
ORDER BY pas.planet_id, pas.alert_id

execution_result holds the first rows of the answer: [[2, 68, 0], [3, 20, 0], [5, 31, 2], [6, 14, 1], [6, 57, 1], …].

How SQaLe compares

SQaLe's schemas are the largest of any corpus compared, by an order of magnitude in columns per schema. It holds more natural-language questions and more distinct SQL statements than any other text-to-SQL corpus except SynSQL. Its tables hold far more rows than SynSQL's (a median of 69 against 2), which lets a question depend on values a model has to look up rather than guess. Query composition matches BIRD on operator diversity and goes further in nesting (23.4% against 7.7%) and in multi-join chains (40% of joining queries against 26%).

<table> <tr> <td><img src="figures/columnsperschema.png" alt="Columns per schema"></td> <td><img src="figures/sqllengthbydifficulty.png" alt="SQL length by difficulty"></td> <td><img src="figures/tablesper_query.png" alt="Tables per query"></td> </tr> </table> <p align="center">_{SQaLe contains more columns per schema (left), longer SQL queries at every difficulty (middle) and a longer tail of tables per query (right). Figure from the paper.}</p>

Simple queries run to a median of about 100 characters, and hard queries to a median of 379 with a much wider spread. 3.1% of SQaLe queries touch five or more tables, up to 20, while no BIRD or EHRSQL query touches more than four.

Domain coverage

<p align="center"> <img src="figures/umap_domains.png" width="480" alt="SQaLe, BIRD and EHRSQL questions in a joint UMAP projection"> </p>

A 13,103-question sample of SQaLe drawn from 6,617 schemas, embedded together with 500 BIRD dev and 500 EHRSQL questions in one UMAP projection. BIRD's questions cluster by database, and nearly all of those clusters fall inside SQaLe's distribution or on its boundary. Part of SQaLe also covers the medical domain of EHRSQL.

How the data was made

1. 1.Schema collection and extension. SQaLe starts from SchemaPile's real-world schemas. A tool-using LLM agent annotates each of the 14,597 source repositories with a short domain description, released as `trl-lab/schemapile_annotated`. Each schema is then extended with LLM-generated tables that keep its naming conventions, level of normalisation and foreign-key style.
2. 2.Table value synthesis. Tables are filled in foreign-key dependency order. For each table an LLM writes a Python function from the table's DDL, its original SchemaPile rows, the allowed values of its foreign keys and the schema's domain description. Fact and junction tables receive more rows and a skewed foreign-key distribution, so aggregations over the data stay non-trivial. Every table is checked for primary-key uniqueness and referential integrity before it is accepted.
3. 3.Question generation. Questions are generated over subschemas. Starting from a random seed table, sampling expands along foreign keys until it reaches a target table count of up to 20. The generator sees the subschema with sample rows and writes questions at three target difficulty levels that state the information need in full and ground every literal in the data. Two versions of the prompt each produce half of the corpus, and the second asks for analytical shapes such as per-group measures, rankings within groups and cohorts.
4. 4.SQL generation and validation. An agent answers each question by exploring the database with tools (listing tables, inspecting schemas, sampling rows, running test queries) and then submits a query. The query is executed, and execution errors are fed back for a retry. A second LLM call acts as a judge on the result, checking for empty or duplicate results and for semantic correctness, and a rejected query is rewritten with the judge's critique. Only questions whose query is accepted enter the corpus.
5. 5.Question style variation. An LLM rewrites every accepted question into seven further styles, each inheriting the original's SQL, so that phrasing is a property of the dataset that can be analysed.

Repository annotation uses Qwen/Qwen3.5-9B, and every later stage uses Qwen/Qwen3.6-35B-A3B-FP8 served with vLLM.

Quality checks

• —The judge. On 225 judge calls over 158 BIRD dev questions, with disagreements against execution accuracy reviewed by hand, the judge agrees with the labels on 86.2% of calls (κ = 0.68), against 76.9% for execution accuracy. 94.5% of the queries it accepts are aligned with their question.
• —The data. 95.2% of tables are populated, 95.1% of foreign-key cells are valid after repair, and 80.0% of foreign-key columns support a non-trivial GROUP BY.
• —Reproducibility. On the 423 test databases written by the SQaLe library (see Quickstart), re-running the 8,100 test queries reproduces the stored execution_result for 99.8% of them, counting floating-point rounding as a match.

Fields

Splits

The split is made at the schema level. 95% of schemas and their questions form train and 5% form test, so no test schema appears in training: 8,836 train and 423 test schemas, with 169,277 and 8,100 questions.

Models trained on SQaLe

The paper trains Qwen3.5-2B with GRPO from the base checkpoint on SQaLe, on BIRD train and on SynSQL-2.5M, with everything else held fixed. The model trained on SQaLe improves on the untrained base model by 33.0 points on BIRD dev and leads the other two on the SQaLe test set at every schema size. Execution accuracy (%), schema withheld, 300 questions per benchmark:

Each model repository includes sqale_agent.py, which runs the model as an agent on any SQLite file, including the databases built from this dataset.

Intended uses

• —Training text-to-SQL models, with execution-based rewards against the populated databases or with supervised targets. The evidence_supported phrasing is meant for single-shot models that answer in one pass.
• —Evaluating text-to-SQL systems and agents on large schemas, where finding the relevant tables is part of the task.
• —Studying robustness to phrasing, since every question comes in eight styles that share one gold query.

Known issues

• —Placeholder phrasings. 87,365 of the 1,408,056 phrasings are placeholders left by the style-variation step, such as ..., <string> or a bare difficulty label. 1,752 records have no usable verbose question, and about one record in ten (9.9% of train, 11.2% of test) has at least one missing or placeholder phrasing. load_questions in the SQaLe library replaces them with None, and sqale.is_usable_question applies the same check to any string.
• —Semi-synthetic values. Table rows are generated, not collected. Tables hold a median of 69 rows, far more than SynSQL's but fewer than the real database dumps behind BIRD, and 4.8% of tables are empty.
• —Judge-validated gold SQL. Gold queries were accepted by an LLM judge whose accepted queries were aligned in 94.5% of the validation calls, so a small share of gold queries will not match their question.
• —SQLite and English only. The SQL targets SQLite, and all questions are in English.

Citation

If you use SQaLe, please cite:

@misc{wolff2026sqale,
  title  = {{SQaLe}: A Large Realistic Dataset to Empower Small Specialised Text-to-{SQL} Models},
  author = {Wolff, Cornelius and Gomm, Daniel and Hulsebos, Madelon},
  year   = {2026}
}

Authors: Cornelius Wolff and Daniel Gomm (University of Amsterdam, Centrum Wiskunde & Informatica), Madelon Hulsebos (Centrum Wiskunde & Informatica). Questions and feedback are welcome in the Community tab of this repository.

SQaLe builds on SchemaPile, and its comparisons use BIRD, EHRSQL, Spider 2.0 and SynSQL-2.5M. We thank their authors for making them available.