Datasetpublic

HPLT/HPLT2.0_cleaned

NB: HPLT2.0 is now superseded by a newer release: HPLT3.0 We recommed switching to v3.0, unless you have a compelling reason to stay on 2.0. This is a large-scale collection of web-crawled documents in 191 world languages, produced by the HPLT project. The source of the data is mostly Internet Archive with some additions from Common Crawl. For a detailed description of the dataset, please refer to our website and our pre-print. The Cleaned variant of HPLT Datasets v2.0 This is… See the full description on the dataset page: https://huggingface.co/datasets/HPLT/HPLT2.0_cleaned.

sourceHugging Facecc0-1.0updated 4mo agoView on Hugging Face

46likes187kdownloads

direng_Latn_10/

direng_Latn_100/

direng_Latn_101/

direng_Latn_102/

direng_Latn_103/

direng_Latn_104/

direng_Latn_105/

direng_Latn_106/

direng_Latn_107/

direng_Latn_108/

direng_Latn_109/

direng_Latn_11/

direng_Latn_110/

direng_Latn_111/

direng_Latn_112/

direng_Latn_113/

direng_Latn_114/

direng_Latn_115/

direng_Latn_116/

direng_Latn_117/

direng_Latn_118/

direng_Latn_119/

direng_Latn_12/

direng_Latn_120/

direng_Latn_121/

direng_Latn_122/

direng_Latn_123/

direng_Latn_124/

direng_Latn_125/

direng_Latn_126/

direng_Latn_127/

direng_Latn_128/

direng_Latn_129/

direng_Latn_13/

direng_Latn_130/

direng_Latn_131/

direng_Latn_132/

direng_Latn_133/

direng_Latn_134/

direng_Latn_135/

direng_Latn_136/

direng_Latn_137/

direng_Latn_138/

direng_Latn_139/

direng_Latn_14/

direng_Latn_140/

direng_Latn_141/

direng_Latn_142/

direng_Latn_143/

direng_Latn_144/

direng_Latn_145/

direng_Latn_146/

direng_Latn_147/

direng_Latn_148/

direng_Latn_149/

direng_Latn_15/

direng_Latn_150/

direng_Latn_151/

direng_Latn_152/

direng_Latn_153/

direng_Latn_154/

direng_Latn_155/

direng_Latn_156/

direng_Latn_157/

direng_Latn_158/

direng_Latn_159/

direng_Latn_16/

direng_Latn_160/

direng_Latn_17/

direng_Latn_18/

direng_Latn_19/

direng_Latn_20/

direng_Latn_21/

direng_Latn_22/

direng_Latn_23/

direng_Latn_24/

direng_Latn_25/

direng_Latn_26/

direng_Latn_27/

direng_Latn_28/

direng_Latn_29/

direng_Latn_30/

direng_Latn_31/

direng_Latn_32/

direng_Latn_33/

direng_Latn_34/

direng_Latn_35/

direng_Latn_36/

direng_Latn_37/

direng_Latn_38/

direng_Latn_39/

direng_Latn_40/

direng_Latn_41/

direng_Latn_42/

direng_Latn_43/

direng_Latn_44/

direng_Latn_45/

direng_Latn_46/

direng_Latn_47/

direng_Latn_48/

direng_Latn_49/

direng_Latn_50/

direng_Latn_51/

direng_Latn_52/

direng_Latn_53/

direng_Latn_54/

direng_Latn_55/

direng_Latn_56/

direng_Latn_57/

direng_Latn_58/

direng_Latn_59/

direng_Latn_60/

direng_Latn_61/

direng_Latn_62/

direng_Latn_63/

direng_Latn_64/

direng_Latn_65/

direng_Latn_66/

direng_Latn_67/

direng_Latn_68/

direng_Latn_69/

direng_Latn_70/

direng_Latn_71/

direng_Latn_72/

direng_Latn_73/

direng_Latn_74/

direng_Latn_75/

direng_Latn_76/

direng_Latn_77/

direng_Latn_78/

direng_Latn_79/

direng_Latn_80/

direng_Latn_81/

direng_Latn_82/

direng_Latn_83/

direng_Latn_84/

direng_Latn_85/

direng_Latn_86/

direng_Latn_87/

direng_Latn_88/

direng_Latn_89/

direng_Latn_90/

direng_Latn_91/

direng_Latn_92/

direng_Latn_93/

direng_Latn_94/

direng_Latn_95/

direng_Latn_96/

direng_Latn_97/

direng_Latn_98/

direng_Latn_99/

dirjpn_Jpan_10/

dirjpn_Jpan_11/

dirjpn_Jpan_12/

dirjpn_Jpan_13/

dirjpn_Jpan_14/

dirjpn_Jpan_15/

dirjpn_Jpan_16/

dirjpn_Jpan_17/

dirjpn_Jpan_18/

dirjpn_Jpan_19/

dirjpn_Jpan_20/

dirjpn_Jpan_21/

dirrus_Cyrl_10/

dirrus_Cyrl_11/

dirrus_Cyrl_12/

dirrus_Cyrl_13/

dirrus_Cyrl_14/

dirrus_Cyrl_15/

dirrus_Cyrl_16/

dirrus_Cyrl_17/

dirrus_Cyrl_18/

dirrus_Cyrl_19/

dirrus_Cyrl_20/

dirrus_Cyrl_21/

dirrus_Cyrl_22/

dirrus_Cyrl_23/

dirrus_Cyrl_24/

dirrus_Cyrl_25/

dirrus_Cyrl_26/

dirrus_Cyrl_27/

dirrus_Cyrl_28/

dirrus_Cyrl_29/

dirrus_Cyrl_30/

dirrus_Cyrl_31/

dirrus_Cyrl_32/

dirrus_Cyrl_33/

dirrus_Cyrl_34/

dirrus_Cyrl_35/

dirrus_Cyrl_36/

dirrus_Cyrl_37/

dirrus_Cyrl_38/

dirrus_Cyrl_39/

dirzho_Hans_10/

dirzho_Hans_11/

dirzho_Hans_12/

dirzho_Hans_13/

dirzho_Hans_14/

dirzho_Hans_15/

dirzho_Hans_16/

dirzho_Hans_17/

dirzho_Hans_18/

dirzho_Hans_19/

dirzho_Hans_20/

dirzho_Hans_21/

dirzho_Hans_22/

dirzho_Hans_23/

dirzho_Hans_24/

dirzho_Hans_25/

dirzho_Hans_26/

dirzho_Hans_27/

dirzho_Hans_28/

dirzho_Hans_29/

dirzho_Hans_30/

dirzho_Hans_31/

dirzho_Hans_32/

dirzho_Hans_33/

dirzho_Hans_34/

dirzho_Hans_35/

dirzho_Hans_36/

dirzho_Hans_37/

dirzho_Hans_38/

dirzho_Hans_39/

dirzho_Hans_40/

dirzho_Hans_41/

dirzho_Hans_42/

dirzho_Hans_43/

dirzho_Hans_44/

dirzho_Hans_45/

dirzho_Hans_46/

dirzho_Hans_47/

dirzho_Hans_48/

dirzho_Hans_49/

dirzho_Hans_50/

dirzho_Hans_51/

dirzho_Hans_52/

dirzho_Hans_53/

dirzho_Hans_54/

dirzho_Hans_55/

dirzho_Hans_56/

dirzho_Hans_57/

dirzho_Hans_58/

dirzho_Hans_59/

dirzho_Hans_60/

dirzho_Hans_61/

dirzho_Hans_62/

dirzho_Hans_63/

dirzho_Hans_64/

dirzho_Hans_65/

dirzho_Hans_66/

dirzho_Hans_67/

dirzho_Hans_68/

dirzho_Hans_69/

dirzho_Hans_70/

dirzho_Hans_71/

dirzho_Hans_72/

dirzho_Hans_73/

dirzho_Hans_74/

dirzho_Hans_75/

dirzho_Hans_76/

dirzho_Hans_77/

dirzho_Hans_78/

dirzho_Hans_79/

file.gitattributes2 KBdownload

fileablations_english.png98 KBdownload

fileablations_norwegian.jpg43 KBdownload

fileenglish-comparison-datasets-by-HPLT.png1.3 MBdownload

fileLLM_eval_en_scores_norm.png84 KBdownload

filenoreval_ablation_camera_ready.jpg43 KBdownload

fileREADME.md62 KBdownload

HPLT/HPLT2.0_cleaned · main · files are served by the source, never re-hosted here

HPLT/HPLT2.0_cleaned · Team Ai