commoncrawl/statistics
Common Crawl Statistics Number of pages, distribution of top-level domains, crawl overlaps, etc. - basic metrics about Common Crawl Monthly Crawl Archives, for more detailed information and graphs please visit our official statistics page. Here you can find the following statistics files: Charsets The character set or encoding of HTML pages only is identified by Tika's AutoDetectReader. The table shows the percentage how character sets have been used to encode… See the full description on the dataset page: https://huggingface.co/datasets/commoncrawl/statistics.
Update data (CC-MAIN-2026-39) (#3)
Delete bao
Upload bao with huggingface_hub
Update data (CC-MAIN-2026-34)
Update data (CC-MAIN-2026-34)
Update data (CC-MAIN-2026-30)
Update data (CC-MAIN-2026-30)
Update data (CC-MAIN-2026-25)
Update data (CC-MAIN-2026-25)
chore: Update data (CC-MAIN-2026-21)
chore:Update data (CC-MAIN-2026-21)
Update data (CC-MAIN-2026-17)
Update data (CC-MAIN-2026-17)
Update data (CC-MAIN-2026-12)
Update data (CC-MAIN-2026-12)
Update data (CC-MAIN-2026-08)
Update data (CC-MAIN-2026-04)
Update data (CC-MAIN-2026-04)
Update data (CC-MAIN-2025-51)
Update data (CC-MAIN-2025-51)
Update data (CC-MAIN-2025-47)
Update data (CC-MAIN-2025-47)
Update data (CC-MAIN-2025-43)
Update data (CC-MAIN-2025-43)
Upload folder using huggingface_hub
chore: updated statistics with latest data from August 2025 crawl (CC-MAIN-2025-33)
chore: updated staticstics with data from July 2025 crawl (CC-MAIN-2025-30)
chore: updated stats with latest data from June 2025 crawl (CC-MAIN-2025-26)
chore: updated stats with latest data from CC-MAIN-2025-21
chore: updated staticstics with latest data from CC-MAIN-2025-18
chore: updated staticstics with latest data up to January 2025 (CC-MAIN-2025-08)
chore: update statistics with data from September and October 2024 crawls: CC-MAIN-2024-38 and CC-MAIN-2024-42
Updated staticstics with latest data from CC-MAIN-2024-33
chore: Updated staticstics with latest data from CC-MAIN-2024-30
chore: Updated the stats for the latest CC-MAIN-2024-26 stats and completed the documentation
Set `sep="\s+"` for the duplicates file (#1)
fix: fixed YAML config
feat: Added config for dataset viewer
Update README.md
Added the files that we use for the plots
Uploaded more statistics files
Create README.md
Upload languages.csv
initial commit