BramVanroy/fineweb-2-duckdbs
DuckDB datasets for (dump, id) querying on FineWeb 2 This repo contains some DuckDB databases to check whether a given WARC UID exists in a FineWeb-2 dump. Usage example is given below, but note especially that if you are using URNs (likely, if you are working with CommonCrawl data), then you first have to extract the UID (the id column is of type UUID in the databases). Download All files: huggingface-cli download BramVanroy/fineweb-2-duckdbs --local-dir… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/fineweb-2-duckdbs.
Upload fw2-zyp_Latn_removed.duckdb with huggingface_hub
Upload fw2-zyb_Latn_removed.duckdb with huggingface_hub
Upload fw2-zul_Latn_removed.duckdb with huggingface_hub
Upload fw2-zty_Latn_removed.duckdb with huggingface_hub
Upload fw2-ztq_Latn_removed.duckdb with huggingface_hub
Upload fw2-zsr_Latn_removed.duckdb with huggingface_hub
Upload fw2-zsm_Latn_removed.duckdb with huggingface_hub
Upload fw2-zsm_Arab_removed.duckdb with huggingface_hub
Upload fw2-zpz_Latn_removed.duckdb with huggingface_hub
Upload fw2-zpv_Latn_removed.duckdb with huggingface_hub
Upload fw2-zpu_Latn_removed.duckdb with huggingface_hub
Upload fw2-zpt_Latn_removed.duckdb with huggingface_hub
Upload fw2-zpq_Latn_removed.duckdb with huggingface_hub
Upload fw2-zpo_Latn_removed.duckdb with huggingface_hub
Upload fw2-zpm_Latn_removed.duckdb with huggingface_hub
Upload fw2-zpl_Latn_removed.duckdb with huggingface_hub
Upload fw2-zpj_Latn_removed.duckdb with huggingface_hub
Upload fw2-zpi_Latn_removed.duckdb with huggingface_hub
Upload fw2-zpg_Latn_removed.duckdb with huggingface_hub
Upload fw2-zpc_Latn_removed.duckdb with huggingface_hub
Upload fw2-zpa_Latn_removed.duckdb with huggingface_hub
Upload fw2-zos_Latn_removed.duckdb with huggingface_hub
Upload fw2-zom_Latn_removed.duckdb with huggingface_hub
Upload fw2-zne_Latn_removed.duckdb with huggingface_hub
Upload fw2-ziw_Latn_removed.duckdb with huggingface_hub
Upload fw2-zia_Latn_removed.duckdb with huggingface_hub
Upload fw2-zgh_Tfng_removed.duckdb with huggingface_hub
Upload fw2-zea_Latn_removed.duckdb with huggingface_hub
Upload fw2-zdj_Latn_removed.duckdb with huggingface_hub
Upload fw2-zca_Latn_removed.duckdb with huggingface_hub
Upload fw2-zaw_Latn_removed.duckdb with huggingface_hub
Upload fw2-zav_Latn_removed.duckdb with huggingface_hub
Upload fw2-zat_Latn_removed.duckdb with huggingface_hub
Upload fw2-zas_Latn_removed.duckdb with huggingface_hub
Upload fw2-zar_Latn_removed.duckdb with huggingface_hub
Upload fw2-zao_Latn_removed.duckdb with huggingface_hub
Upload fw2-zam_Latn_removed.duckdb with huggingface_hub
Upload fw2-zai_Latn_removed.duckdb with huggingface_hub
Upload fw2-zae_Latn_removed.duckdb with huggingface_hub
Upload fw2-zad_Latn_removed.duckdb with huggingface_hub
Upload fw2-zac_Latn_removed.duckdb with huggingface_hub
Upload fw2-zab_Latn_removed.duckdb with huggingface_hub
Upload fw2-zaa_Latn_removed.duckdb with huggingface_hub
Upload fw2-yva_Latn_removed.duckdb with huggingface_hub
Upload fw2-yuz_Latn_removed.duckdb with huggingface_hub
Upload fw2-yuw_Latn_removed.duckdb with huggingface_hub
Upload fw2-yut_Latn_removed.duckdb with huggingface_hub
Upload fw2-yup_Latn_removed.duckdb with huggingface_hub
Upload fw2-yuj_Latn_removed.duckdb with huggingface_hub
Upload fw2-yue_Hani_removed.duckdb with huggingface_hub