datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
document-haystack
Document Haystack Dataset
This repository contains the dataset for the paper “Document Haystack: A Long Context Multimodal Image/Document Understanding Vision LLM Benchmark”.
📑 Abstract Paper
The proliferation of multimodal Large Language Models has significantly advanced the ability to analyze and understand complex data inputs from different modalities. However, the processing of long documents remains under-explored, largely due to a lack of suitable benchmarks. To… See the full description on the dataset page: https://huggingface.co/datasets/AmazonScience/document-haystack.Amazon-Reviews-2023Amazon Review 2023 is an updated version of the Amazon Review 2018 dataset.
This dataset mainly includes reviews (ratings, text) and item metadata (desc-
riptions, category information, price, brand, and images). Compared to the pre-
vious versions, the 2023 version features larger size, newer reviews (up to Sep
2023), richer and cleaner meta data, and finer-grained timestamps (from day to
milli-second).MultilingualMultiModalClassification
Additional Information
To load the dataset,
import datasets
ds = datasets.load_dataset("AmazonScience/MultilingualMultiModalClassification", data_dir="wiki-doc-ar-merged")
print(ds)
DatasetDict({
train: Dataset({
features: ['image', 'filename', 'words', 'ocr_bboxes', 'label'],
num_rows: 8129
})
validation: Dataset({
features: ['image', 'filename', 'words', 'ocr_bboxes', 'label'],
num_rows: 1742
})
test: Dataset({
features:… See the full description on the dataset page: https://huggingface.co/datasets/AmazonScience/MultilingualMultiModalClassification.amazon_massive_intent
MassiveIntentClassification
An MTEB dataset
Massive Text Embedding Benchmark
MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages
Task category
t2c
Domains
Spoken
Reference
https://arxiv.org/abs/2204.08582
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["MassiveIntentClassification"])
evaluator =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/amazon_massive_intent.massive MASSIVE is a parallel dataset of > 1M utterances across 51 languages with annotations
for the Natural Language Understanding tasks of intent prediction and slot annotation.
Utterances span 60 intents and include 55 slot types. MASSIVE was created by localizing
the SLURP dataset, composed of general Intelligent Voice Assistant single-shot interactions.amazon_polarity
Dataset Card for Amazon Review Polarity
Dataset Summary
The Amazon reviews dataset consists of reviews from amazon.
The data span a period of 18 years, including ~35 million reviews up to March 2013.
Reviews include product and user information, ratings, and a plaintext review.
Supported Tasks and Leaderboards
text-classification, sentiment-classification: The dataset is mainly used for text classification: given the content and the title, predict the correct… See the full description on the dataset page: https://huggingface.co/datasets/fancyzhx/amazon_polarity.amazon_massive_scenario
MassiveScenarioClassification
An MTEB dataset
Massive Text Embedding Benchmark
MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages
Task category
t2c
Domains
Spoken
Reference
https://arxiv.org/abs/2204.08582
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["MassiveScenarioClassification"])
evaluator =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/amazon_massive_scenario.SWE-PolyBench
SWE-PolyBench
SWE-PolyBench is a multi language repo level software engineering benchmark. Currently it includes 4 languages: Python, Java, Javascript, and Typescript. The number of instances in each language is:
Javascript: 1017
Typescript: 729
Python: 199
Java: 165
Datasets
There are total three datasets available under SWE-PolyBench. AmazonScience/SWE-PolyBench is the full dataset, AmazonScience/SWE-PolyBench_500 is the stratified sampled dataset with 500 instances and… See the full description on the dataset page: https://huggingface.co/datasets/AmazonScience/SWE-PolyBench.amazon_counterfactual
AmazonCounterfactualClassification
An MTEB dataset
Massive Text Embedding Benchmark
A collection of Amazon customer reviews annotated for counterfactual detection pair classification.
Task category
t2c
Domains
Reviews, Written
Reference
https://arxiv.org/abs/2104.06893
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["AmazonCounterfactualClassification"])… See the full description on the dataset page: https://huggingface.co/datasets/mteb/amazon_counterfactual.amazon-berkeley-objects
Amazon Berkeley Objects (ABO)
A Hugging Face packaging of the Amazon Berkeley Objects (ABO) dataset. The
data content is the official CC BY 4.0 release from
https://amazon-berkeley-objects.s3.amazonaws.com/index.html. This mirror
changes only the packaging: files are grouped into typed Parquet shards, and
every original media file is preserved byte-for-byte and never transcoded.
Images use the datasets Image() feature, 3D product models use the native
Mesh() feature (original… See the full description on the dataset page: https://huggingface.co/datasets/suvadityamuk/amazon-berkeley-objects.amazon_reviews_multi_enmigration-bench-java-full
MigrationBench
1. 📖 Overview
🤗 MigrationBench
is a large-scale code migration benchmark dataset at the repository level,
across multiple programming languages.
Current and initial release includes java 8 repositories with the maven build system… See the full description on the dataset page: https://huggingface.co/datasets/AmazonScience/migration-bench-java-full.amazon_reviews_multiWe provide an Amazon product reviews dataset for multilingual text classification. The dataset contains reviews in English, Japanese, German, French, Chinese and Spanish, collected between November 1, 2015 and November 1, 2019. Each record in the dataset contains the review text, the review title, the star rating, an anonymized reviewer ID, an anonymized product ID and the coarse-grained product category (e.g. ‘books’, ‘appliances’, etc.) The corpus is balanced across stars, so each star rating constitutes 20% of the reviews in each language.
For each language, there are 200,000, 5,000 and 5,000 reviews in the training, development and test sets respectively. The maximum number of reviews per reviewer is 20 and the maximum number of reviews per product is 20. All reviews are truncated after 2,000 characters, and all reviews are at least 20 characters long.
Note that the language of a review does not necessarily match the language of its marketplace (e.g. reviews from amazon.de are primarily written in German, but could also be written in English, etc.). For this reason, we applied a language detection algorithm based on the work in Bojanowski et al. (2017) to determine the language of the review text and we removed reviews that were not written in the expected language.amazon_counterfactual_en
Amazon Counterfactual Statements
This dataset is the en-ext split from SetFit/amazon_counterfactual. As the original test set is rather small (1333 examples), a different split was created with 50-50 for training & testing.
The dataset is described in amazon-multilingual-counterfactual-dataset / Paper
It contains statements from Amazon reviews about events that did not or cannot take place.
kaputt
Kaputt: A Large-Scale Dataset for Visual Defect Detection
Abstract
We present a novel large-scale dataset for defect detection in a logistics
setting. Recent work on industrial anomaly detection has primarily focused on
manufacturing scenarios with highly controlled poses and a limited number of
object categories. Existing benchmarks like MVTec-AD (Bergmann et al., 2021) and
VisA (Zou et al., 2022) have reached saturation, with state-of-the-art methods
achieving… See the full description on the dataset page: https://huggingface.co/datasets/amazon/kaputt.amazon-reviews-2023-with-asin
Amazon Reviews 2023 (with ASIN)
A trimmed version of the McAuley-Lab/Amazon-Reviews-2023 dataset, retaining only the fields most relevant for NLP tasks while adding explicit product identification via parent_asin.
What's Different from the Original
The original dataset includes 10+ fields per review and requires a legacy loading script that is no longer supported by HuggingFace. This version:
Keeps only 4 fields: rating, title, text, and parent_asin
Is stored in… See the full description on the dataset page: https://huggingface.co/datasets/BarbaDLuca/amazon-reviews-2023-with-asin.SWE-PolyBench_Verified
SWE-PolyBench
SWE-PolyBench is a multi language repo level software engineering benchmark. Currently it includes 4 languages: Python, Java, Javascript, and Typescript. The number of instances in the verified split is:
Javascript: 100
Typescript: 100
Python: 113
Java: 69
Datasets
There are total three datasets available under SWE-PolyBench. AmazonScience/SWE-PolyBench is the full dataset, AmazonScience/SWE-PolyBench_500 is the stratified sampled dataset with 500 instances… See the full description on the dataset page: https://huggingface.co/datasets/AmazonScience/SWE-PolyBench_Verified.amazon_reviews_mcauley_1and5amazon_massive_intent_en-USamazon-benchmark
Amazon query–bundle benchmark
Canonical, category-organized query and reference-positive data. Experiment traces should reference this repository by commit SHA, category, split, and candidate_id, rather than republishing the dataset.
Musical Instruments
Split
Examples
agent_dev
2,028
agent_hidden
1,960
Each record contains a query and 3–7 reference product IDs. These are observed reference positives, not exhaustive labels for all valid… See the full description on the dataset page: https://huggingface.co/datasets/iaouali/amazon-benchmark.amazon_reviews_multiWe provide an Amazon product reviews dataset for multilingual text classification. The dataset contains reviews in English, Japanese, German, French, Chinese and Spanish, collected between November 1, 2015 and November 1, 2019. Each record in the dataset contains the review text, the review title, the star rating, an anonymized reviewer ID, an anonymized product ID and the coarse-grained product category (e.g. ‘books’, ‘appliances’, etc.) The corpus is balanced across stars, so each star rating constitutes 20% of the reviews in each language.
For each language, there are 200,000, 5,000 and 5,000 reviews in the training, development and test sets respectively. The maximum number of reviews per reviewer is 20 and the maximum number of reviews per product is 20. All reviews are truncated after 2,000 characters, and all reviews are at least 20 characters long.
Note that the language of a review does not necessarily match the language of its marketplace (e.g. reviews from amazon.de are primarily written in German, but could also be written in English, etc.). For this reason, we applied a language detection algorithm based on the work in Bojanowski et al. (2017) to determine the language of the review text and we removed reviews that were not written in the expected language.repository-metadataConfBench
FCC Invoices Verified Augmented
Dataset Description
FCC Invoices Verified Augmented is a document understanding benchmark dataset consisting of 75 real-world Federal Communications Commission (FCC) invoice documents, each augmented with up to 18 distinct document degradation pipelines. The dataset is designed to support confidence calibration research, OCR robustness evaluation, and key information extraction (KIE) under realistic noise conditions.
Each document… See the full description on the dataset page: https://huggingface.co/datasets/amazon/ConfBench.bold
Dataset Card for Bias in Open-ended Language Generation Dataset (BOLD)
Dataset Description
Bias in Open-ended Language Generation Dataset (BOLD) is a dataset to evaluate fairness in open-ended language generation in English language. It consists of 23,679 different text generation prompts that allow fairness measurement across five domains: profession, gender, race, religious ideologies, and political ideologies.
Some examples of prompts in BOLD are as follows:
Many… See the full description on the dataset page: https://huggingface.co/datasets/AmazonScience/bold.Amazon-Reviews-2023Amazon Review 2023 is an updated version of the Amazon Review 2018 dataset.
This dataset mainly includes reviews (ratings, text) and item metadata (desc-
riptions, category information, price, brand, and images). Compared to the pre-
vious versions, the 2023 version features larger size, newer reviews (up to Sep
2023), richer and cleaner meta data, and finer-grained timestamps (from day to
milli-second).model-mappingRealKIE-FCC-Verified
RealKIE-FCC-Verified
It is a test set with single and multi-page invoices sourced from the Federal Communications Commission (FCC) to evaluate key information extraction (KIE) performance.
Task
Extract information from the document in JSON format given the corresponding JSON schema. It contains 75 documents, with:
a) image_files: Each document has multiple pages
b) json_schema: A common JSON schema requiring extraction of specified information including line… See the full description on the dataset page: https://huggingface.co/datasets/amazon-agi/RealKIE-FCC-Verified.SWE-PolyBench_500
SWE-PolyBench
SWE-PolyBench is a multi language repo level software engineering benchmark. Currently it includes 4 languages: Python, Java, Javascript, and Typescript. The number of instances in each language is:
Javascript: 1017
Typescript: 729
Python: 199
Java: 165
Datasets
There are total three datasets available under SWE-PolyBench. AmazonScience/SWE-PolyBench is the full dataset, AmazonScience/SWE-PolyBench_500 is the stratified sampled dataset with 500 instances and… See the full description on the dataset page: https://huggingface.co/datasets/AmazonScience/SWE-PolyBench_500.RobustAD
RobustAD Dataset
About the Dataset
RobustAD, specifically designed to evaluate the robustness of anomaly detection models in real-world scenarios. RobustAD features a curated dataset of defect detection images with meticulously controlled distribution shifts across multiple dimensions relevant to practical applications and more closely mirrors real-world deployment scenarios.
RobustAD is designed to cover inspection challenges across multiple industries to ensure the… See the full description on the dataset page: https://huggingface.co/datasets/AmazonScience/RobustAD.SIFT-50M
Dataset Card for SIFT-50M
SIFT-50M (Speech Instruction Fine-Tuning) is a 50-million-example dataset designed for instruction fine-tuning and pre-training of speech-text large language models (LLMs). It is built from publicly available speech corpora containing a total of 14K hours of speech and leverages LLMs and off-the-shelf expert models. The dataset spans five languages, covering diverse aspects of speech understanding and controllable speech generation instructions. SIFT-50M… See the full description on the dataset page: https://huggingface.co/datasets/amazon-agi/SIFT-50M.