datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
fineweb-2-duckdbs
DuckDB datasets for (dump, id) querying on FineWeb 2
This repo contains some DuckDB databases to check whether a given WARC UID exists in a FineWeb-2 dump. Usage example
is given below, but note especially that if you are using URNs (likely, if you are working with CommonCrawl data),
then you first have to extract the UID (the id column is of type UUID in the databases).
Download
All files:
huggingface-cli download BramVanroy/fineweb-2-duckdbs --local-dir… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/fineweb-2-duckdbs.fineweb-duckdbs
DuckDB datasets for id querying on FineWeb
This repo contains DuckDB databases to check whether a given WARC UID exists in a FineWeb dump. Usage example
is given below, but note especially that if you are using URNs (likely, if you are working with CommonCrawl data),
then you first have to extract the UID (the id column is of type UUID in the databases).
Download
All files:
huggingface-cli download BramVanroy/fineweb-duckdbs --local-dir duckdbs/fineweb/ --include *.duckdb… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/fineweb-duckdbs.duckdb_ci_testsbirdbench-duckdb
BirdBench Dataset in DuckDB format
BirdBench is a benchmark for text-to-SQL capabilities, now available in DuckDB format for improved performance and usability.
About BirdBench
BirdBench is a comprehensive benchmark dataset for evaluating text-to-SQL capabilities of language models. It features a diverse collection of databases spanning various domains including:
Business and finance
Entertainment and media
Sports and recreation
Health and medicine
Education
Travel and… See the full description on the dataset page: https://huggingface.co/datasets/ucalyptus/birdbench-duckdb.duckdb-text2sql-25k
Dataset Summary
The duckdb-text2sql-25k dataset contains 25,000 DuckDB text-2-sql pairs covering diverse aspects of DuckDB's SQL syntax.
We synthesized this dataset using Mixtral 8x7B, based on DuckDB's v0.9.2 documentation and Spider schemas that were translated to DuckDB syntax and enriched with nested type columns.
Each training sample consists of a natural language prompt, a corresponding (optional) schema, and a resulting query. Each pair furthermore has a category property… See the full description on the dataset page: https://huggingface.co/datasets/motherduckdb/duckdb-text2sql-25k.duckdb-qa-v3duckdb-nsql-scoresduckdbtestmsmarco-duckdb-ftsduckdb-docbench
DocBench: A Synthetic DuckDB Text-to-SQL Benchmark
DocBench is a synthetic Text-to-SQL benchmark dataset consisting of 2430 question/sql pairs derived from the DuckDB documentation, specifically designed to probe language models for knowledge of DuckDB-specific SQL functionality.
The dataset covers functions, aggregates, operators, statements, keywords, and multi-keyword expressions available in DuckDB 1.1.3 and its default extensions.
Dataset Structure
Each example… See the full description on the dataset page: https://huggingface.co/datasets/motherduckdb/duckdb-docbench.duckdb-dbt-qaduckdb-nsql-predictionsremote-duckdbsql-console-prompt
SQL Console Text 2 SQL Prompt
GitHub Gist
Feedback is welcome 🤗. This prompt was based on performance from Qwen on the DuckDB NSQL Benchmark, common dataset types and tasks typical for exploring HF Datasets.
This is the prompt used for the Text2SQL inside the SQL Console on Datasets.
Example Table Context
For the {table_context} we use the SQL DDL CREATE TABLE statement.
CREATE TABLE datasets (
"_id" VARCHAR,
"id" VARCHAR,
"author" VARCHAR… See the full description on the dataset page: https://huggingface.co/datasets/duckdb-nsql-hub/sql-console-prompt.duckdb-qa-v2distilabel-duckdb-queries
Dataset Card for distilabel-duckdb-queries
This dataset has been created with distilabel.
The pipeline script was uploaded to easily reproduce the dataset:
app.py.
It can be run directly using the CLI:
distilabel pipeline run --script "https://huggingface.co/datasets/gabrielmbmb/distilabel-duckdb-queries/raw/main/app.py"
Dataset Summary
This dataset contains a pipeline.yaml which can be used to reproduce the pipeline that generated it in distilabel… See the full description on the dataset page: https://huggingface.co/datasets/gabrielmbmb/distilabel-duckdb-queries.danbooru-tags-20260518-duckdb
Danbooru Tags Analysis (DuckDB Edition)
This dataset is an optimized, streamlined derivative of u-haru/danbooru-tags-20260518 (thanks so much for making it available, you rock!). It is specifically engineered to facilitate tag analysis for Illustrious and Anime-focused AI models, described in the Civitai article https://civitai.red/articles/30556.
Key Improvements
Size Optimization: Retains only a limited subset of columns essential for analytical workloads… See the full description on the dataset page: https://huggingface.co/datasets/PixelPerv/danbooru-tags-20260518-duckdb.data-jobs-duckdbduckdb-docsduckdb_testnyc-taxi-2025-duckdb
Taxi-Revenue-and-Surge-Analytics
Every taxi ride tells a data story. From raw trip logs to executive insight, analyzed 9.3M NYC Yellow Taxi trips by cleaning messy source data, building tested dimensional models, and uncovering revenue trends, surge patterns, and top-performing zones through an interactive dashboard
fineweb-bbc-news-embeddings-DuckDBSocial-Media-Posts-Dataset-Embeddings-Included-DUCKDB
📊 Social Media Posts Dataset (Embeddings Included)
Dataset Description
This dataset contains social media posts collected for the purpose of natural language analytics and semantic analysis.It is designed to support trend analysis, topic discovery, sentiment inference, and time-based analytics over historical social media data.
The dataset is intended to serve as the data backbone for a natural language analytics system where users can ask questions in plain English and… See the full description on the dataset page: https://huggingface.co/datasets/Bhavin1905/Social-Media-Posts-Dataset-Embeddings-Included-DUCKDB.noaa-gsod-2011-2020-duckdb-bq042
NOAA GSOD 2011-2020 DuckDB baseline
Single DuckDB file mirroring the BigQuery public tables
bigquery-public-data.noaa_gsod.gsod2011 .. gsod2020 and
bigquery-public-data.noaa_gsod.stations one-to-one (same column names,
same column types: STRING -> VARCHAR, FLOAT64 -> DOUBLE, INT64 -> BIGINT;
no pruning or filtering).
Provenance
Produced by environment/scripts/rebuild_baseline_from_bigquery.py in the
smithy task spider2-lite-bq042-gsod-laguardia-decade, which reads the… See the full description on the dataset page: https://huggingface.co/datasets/Anshifkk/noaa-gsod-2011-2020-duckdb-bq042.dpo_main_finetune_data_duckdbbb-duckdb-test
DuckDB Test Dataset
Authorized security testing under HF bug bounty program.
duckdb-qa-v1duckdb-qa-v4chembl-29-duckdb-bq430
ChEMBL-29 DuckDB mirror for Spider2-Lite bq430
Offline DuckDB mirror of the four bigquery-public-data.ebi_chembl.*
tables referenced by Spider2-Lite task bq430 (matched-molecular-pair
deltas):
main.activities_29 (18,635,916 rows, 27 VARCHAR cols)
main.compound_structures_29 ( 2,084,724 rows, 5 VARCHAR cols)
main.compound_properties_29 ( 2,088,293 rows, 23 VARCHAR cols)
main.docs_29 ( 81,544 rows, 17 VARCHAR cols)
Every column is preserved verbatim… See the full description on the dataset page: https://huggingface.co/datasets/Anshifkk/chembl-29-duckdb-bq430.duckdb