datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Web_Scraper_Datalegal_scrapers
JusticeDAO Legal Scrapers
Research collectors that download official legislative sources only
(national gazettes and official open-data portals). Intended for building
reproducible legal-text corpora, not for production legal research products.
Not legal advice. The official gazette of each jurisdiction prevails.
License
AGPL-3.0 for the collector scripts in this repository.
Contents
scrapers/ includes country collectors, EU/EUR-Lex, shared helpers… See the full description on the dataset page: https://huggingface.co/datasets/endomorphosis/legal_scrapers.korean_data_scraper_aihub
Korean Data Scraper — AI-Hub Local Corpus
상태: 비공개 (private) 저장소입니다.
Korean_data_scraper 프로젝트의 aihub_local 소스가 생성한 코퍼스입니다. 국립정보화진흥원 AI-Hub(aihub.or.kr)에서 내려받은 여러 데이터셋의 로컬 zip 압축 파일을 압축 해제하고, 그 안의 json 파일에서 본문 텍스트를 추출한 결과입니다. 샘플로 확인한 내용 중에는 뉴스 기사(신문기사) 카테고리의 AI-Hub 데이터셋에서 추출된 텍스트가 포함되어 있습니다.
스키마
파일당 1개 레코드(JSONL)이며, 다음과 같은 필드를 가집니다.
{"id": "aihub_local:<zip 파일명>:<json 파일명>", "source": "aihub_local", "text": "...", "url": null, "license": "per-dataset -- check the… See the full description on the dataset page: https://huggingface.co/datasets/mkd-minju/korean_data_scraper_aihub.finra-brokercheck-scraper
FINRA BrokerCheck Scraper · Advisors, Firms & Disclosures
Scrape financial advisors, firm affiliations, CRDs, registration scope, and disclosure histories directly from FINRA BrokerCheck API into clean dataset rows.
Rows in this dataset
1,430
Fields
22
Collector runs behind it
50
Most recent observation
2026-08-03
What this is
Every row here was returned by a real run of a public collector. Nothing is generated from a
template over a… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/finra-brokercheck-scraper.neptun.scraper
Data in this dataset
Docker & NPM
Scraped using crawl4ai.
The NPM and Docker data was scraped from docs.docker.com and docs.npmjs.com and processed using GPT-4 resulting in docker_documentation.jsonl and npm_documentation.jsonl.
The file training-data-v1.jsonl also includes Titanium, dockerNLcommands and docker_ps.
GitHub
Scraped using firecrawl.
The GitHub data was scraped from docs.github.com/en using firecrawl.A few pages might be missing in the… See the full description on the dataset page: https://huggingface.co/datasets/neptun-org/neptun.scraper.tvmaze-scraper
TVmaze Scraper · TV Shows, Episodes, Casts & Networks
Scrape TV shows, episode details, cast members, ratings, genres, and network broadcast data from TVmaze's public database. Pay-per-event pricing per show record.
Rows in this dataset
1,205
Fields
32
Collector runs behind it
50
Most recent observation
2026-08-03
What this is
Every row here was returned by a real run of a public collector. Nothing is generated from a
template over a… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/tvmaze-scraper.shopify-store-products-scraper
Shopify Store Scraper
Scrape products, prices, discounts, variants and stock from any Shopify store's public product JSON. No login, no API key, no headless browser.
Rows in this dataset
11,720
Fields
33
Collector runs behind it
72
Most recent observation
2026-08-04
Browsable presentation
https://reapx.dev/data/shopify-store-products-scraper/ — 1,271 entity pages
Run the collector yourself
https://apify.com/reapx/shopify-store-products-scraper… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/shopify-store-products-scraper.greenhouse-jobs-scraper
Greenhouse Jobs Scraper
Scrape every public job posting from any Greenhouse company job board: title, department, location, remote flag, seniority, advertised salary, full description and apply URL.
Rows in this dataset
14,091
Fields
36
Collector runs behind it
61
Most recent observation
2026-08-04
Browsable presentation
https://reapx.dev/data/greenhouse-jobs-scraper/ — 92 entity pages
Run the collector yourself… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/greenhouse-jobs-scraper.kalshi-scraper
Kalshi Scraper · Event Contracts, Markets, Prices & Volume
Scrape Kalshi prediction markets, event contracts, option pricing, order book quotes, trading volume, open interest, and resolution rules. Export structured JSON, CSV, or Excel data.
Rows in this dataset
7,649
Fields
34
Collector runs behind it
49
Most recent observation
2026-08-03
What this is
Every row here was returned by a real run of a public collector. Nothing is generated… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/kalshi-scraper.Scraped-Data
Scraped-Data — Building to 1 TB
A continuously-growing, cleaned text corpus scraped, parsed, cleaned,
losslessly-zipped, and auto-pushed from a CPU-only Google Colab pipeline.
Current repo size: ~26 GB → target: 1 TB (incremental batch uploads).
Current files
File
Size
Contents
corpus_batch1.clean.zip
4.80 GB
~2.9M Wikipedia docs (stored, from first scrape)
corpus_batch2.clean.zip
4.85 GB
~2.5M docs
corpus_batch3.clean.zip
4.91 GB
~2.3M docs… See the full description on the dataset page: https://huggingface.co/datasets/Gugu8/Scraped-Data.cookpad-scrape-recipes
Cookpad India Recipe Archive
Request More ScrapesOrder Private Scrapes
Overview
This repository contains a dataset scraped from cookpad.com/in, a popular community-driven recipe sharing platform. The dataset serves as an extensive archive of diverse, human-created culinary data, capturing home-cooked recipes, ingredient lists, step-by-step instructions, and related web metadata.
Purpose and Usage
This dataset is published publicly and strictly for… See the full description on the dataset page: https://huggingface.co/datasets/sayurio/cookpad-scrape-recipes.lever-jobs-scraper
Lever Jobs Scraper · Job Postings, Teams, Locations & Salary
Scrape open job postings, departments, locations, remote status, compensation and application URLs from any Lever company job board.
Rows in this dataset
8,716
Fields
20
Collector runs behind it
50
Most recent observation
2026-08-03
What this is
Every row here was returned by a real run of a public collector. Nothing is generated from a
template over a keyword list: a row exists… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/lever-jobs-scraper.scraped-web-databoardgamegeek-scraper
BoardGameGeek Scraper · Games, Ratings, Designers & Mechanics
Scrape board games, release years, player counts, categories, mechanics, designers, artists, and publishers from BoardGameGeek. HTTP only, pay-per-event pricing.
Rows in this dataset
437
Fields
21
Collector runs behind it
50
Most recent observation
2026-08-03
What this is
Every row here was returned by a real run of a public collector. Nothing is generated from a
template over a… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/boardgamegeek-scraper.grants-gov-scraper
Grants.gov Scraper · Grant Opportunities, Agencies & Awards
Scrape US federal grant opportunities, funding announcements, and agency award notices from Grants.gov by keyword, agency, category, eligibility, and status.
Rows in this dataset
1,488
Fields
12
Collector runs behind it
50
Most recent observation
2026-08-03
What this is
Every row here was returned by a real run of a public collector. Nothing is generated from a
template over a… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/grants-gov-scraper.open-library-scraper
Open Library Scraper · Books, Authors, Editions & Subjects
Scrape Open Library books, authors, subjects, editions, and metadata via Open Library API. Fast HTTP scraper charging per returned record with tiered pricing.
Rows in this dataset
3,880
Fields
21
Collector runs behind it
50
Most recent observation
2026-08-04
Browsable presentation
https://reapx.dev/data/open-library-scraper/ — 3,880 entity pages
Run the collector yourself… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/open-library-scraper.sfia-9-scraped
SFIA-9-Scraped Dataset
This repository contains the SFIA-9-Scraped dataset, a JSON collection of the Skills Framework for the Information Age (SFIA) version 9 categories and levels, scraped for non-commercial research use.
🚀 Dataset Overview
Name: SFIA-9-Scraped
Hugging Face: Programmer-RD-AI/sfia-9-scraped
DOI: 10.57967/hf/5746
Author: Ranuga Disansa Gamage
Revision: 89feeb8
Publisher: Hugging Face
Year: 2025
Use this dataset to build RAG systems, taxonomy-driven… See the full description on the dataset page: https://huggingface.co/datasets/Programmer-RD-AI/sfia-9-scraped.doaj-scraper
DOAJ Scraper · Open Access Journals, Articles & Publishers
Scrape open access research articles, DOIs, authors, subjects, publishers, and abstracts from the Directory of Open Access Journals (DOAJ) API. Features pay-per-event pricing and automatic backoff.
Rows in this dataset
1,720
Fields
22
Collector runs behind it
50
Most recent observation
2026-08-03
What this is
Every row here was returned by a real run of a public collector. Nothing… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/doaj-scraper.shopify-app-store-scraper
Shopify App Store Scraper
List Shopify App Store apps and get one structured row per app, with developer, star rating, review count, every advertised pricing plan and the app's rank in its category.
Rows in this dataset
3,880
Fields
26
Collector runs behind it
87
Most recent observation
2026-08-04
Browsable presentation
https://reapx.dev/data/shopify-app-store-scraper/ — 1,625 entity pages
Run the collector yourself… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/shopify-app-store-scraper.github-repo-scraper
GitHub Repo Scraper · Repositories, Stars, Topics & Languages
Scrape GitHub repositories by language, topic, star count, license, organization, and pushed date window. Returns clean structured repo metrics and metadata without authentication.
Rows in this dataset
2,481
Fields
27
Collector runs behind it
50
Most recent observation
2026-08-04
Browsable presentation
https://reapx.dev/data/github-repo-scraper/ — 2,071 entity pages
Run the collector yourself… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/github-repo-scraper.app-store-reviews-scraper
App Store Reviews Scraper
Scrape Apple App Store reviews, star ratings and app version history for any iOS app in any country storefront. No login, no API key.
Rows in this dataset
23,048
Fields
43
Collector runs behind it
88
Most recent observation
2026-08-04
Browsable presentation
https://reapx.dev/data/app-store-reviews-scraper/ — 176 entity pages
Run the collector yourself
https://apify.com/reapx/app-store-reviews-scraper
What this is… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/app-store-reviews-scraper.sec-edgar-scraper
SEC EDGAR Scraper
Search SEC EDGAR by ticker, CIK, SIC industry code, or full-text phrase and get one structured row per filing, with company classification, period dates, direct document links and optional XBRL financials.
Rows in this dataset
2,691
Fields
27
Collector runs behind it
64
Most recent observation
2026-08-04
Browsable presentation
https://reapx.dev/data/sec-edgar-scraper/ — 771 entity pages
Run the collector yourself… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/sec-edgar-scraper.arxiv-papers-scraper
arXiv Papers Scraper
Search arXiv and export papers with full abstracts, author lists, subject categories, DOIs, journal references and PDF links. Filter by subject class, keyword, author, affiliation or date window.
Rows in this dataset
21,722
Fields
26
Collector runs behind it
92
Most recent observation
2026-08-04
Browsable presentation
https://reapx.dev/data/arxiv-papers-scraper/ — 10,624 entity pages
Run the collector yourself… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/arxiv-papers-scraper.Nifty-Authoritarian-ScrapeData Scrape from LGBT Literature Archive Nifty.Org
-Category: Authoritarian
Connor-Data-Scraped-v1crossref-scraper
Crossref Scraper · DOI Metadata, Authors, Journals & Citations
Scrape scholarly DOI metadata, works, journal articles, authors, citations, funding, and licenses from the Crossref REST API. Fast HTTP scraper with pay-per-event pricing.
Rows in this dataset
2,492
Fields
29
Collector runs behind it
50
Most recent observation
2026-08-04
Browsable presentation
https://reapx.dev/data/crossref-scraper/ — 2,492 entity pages
Run the collector yourself… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/crossref-scraper.pypi-scraper
PyPI Scraper · Python Packages, Releases, Authors & Licences
Scrape Python packages, release histories, dependencies, authors, maintainers, licenses, and download statistics from PyPI. HTTP only, pay-per-event pricing.
Rows in this dataset
451
Fields
27
Collector runs behind it
50
Most recent observation
2026-08-03
What this is
Every row here was returned by a real run of a public collector. Nothing is generated from a
template over a… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/pypi-scraper.cypherhackz-scraped
website: cypherhackz
Number of pages scraped: 9
Number of posts scraped: 805
Link to dataset on Huggingface
startech.com.bd-web-scrape
Star Tech Product Archive
Overview
This repository contains a comprehensive dataset scraped from startech.com.bd, a major retailer of computers, hardware components, and consumer electronics in Bangladesh. The dataset serves as a structured archive of product catalogs, technical specifications, pricing, and descriptions.
Purpose and Usage
This dataset is published publicly and strictly for educational, research, and analytical purposes. It is an excellent… See the full description on the dataset page: https://huggingface.co/datasets/sayurio/startech.com.bd-web-scrape.storymirror.com-web-scrape
StoryMirror Multilingual Literature Archive
Overview
This repository contains a large-scale text dataset scraped from storymirror.com, a prominent digital platform for Indian literature. The primary goal of this archive is to preserve a massive, multilingual collection of purely human-written stories, poems, and quotes, creating a distinct record of human creativity and storytelling across various Indian languages.
Purpose and Usage
This dataset is… See the full description on the dataset page: https://huggingface.co/datasets/sayurio/storymirror.com-web-scrape.