datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Athar-EmbeddingsFinQAFinQA is a large-scale dataset with 2.8k financial reports for 8k Q&A pairs to study numerical reasoning with structured and unstructured evidence.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/FinQA", "queries")
documents = datasets.load_dataset("embedding-benchmark/FinQA", "corpus")
pair_labels = datasets.load_dataset("embedding-benchmark/FinQA", "default")
FinanceBenchThe FinanceBench dataset is derived from the PatronusAI/financebench-test dataset, containing only the PASS examples processed into a clean format for question-answering tasks in the financial domain. FinanceBench-rtl has been repurposed for retrieval.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/FinanceBench", "queries")
documents = datasets.load_dataset("embedding-benchmark/FinanceBench", "corpus")
pair_labels =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/FinanceBench.LMEB
🚀 LMEB is now officially supported by MTEB.You can evaluate embedding models on LMEB through the standard MTEB evaluation framework. After running the benchmark, results can be submitted to the MTEB results repository. Once merged, the LMEB leaderboard will be updated on the MTEB Leaderboard.
🔥 Why LMEB?
LMEB fills a crucial gap in current embedding benchmarks, offering a standardized and reproducible… See the full description on the dataset page: https://huggingface.co/datasets/KaLM-Embedding/LMEB.stackexchange_title_body_jsonljsonl.gz format from https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_xml
Each line contains a dict in the format: {"text": ["title", "body"], "tags": ["tag1", "tag2"]}
The following parameters have been used for filtering: min_title_len = 20 min_body_len = 20 max_body_len = 4096 min_score = 0
If a stackexchange contained less than 10k questions (after filtering), it is written to the small_stackexchanges.jsonl.gz file.
This is a dump of the files from… See the full description on the dataset page: https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_title_body_jsonl.HC3FinanceThe HC3 dataset comprises tens of thousands of comparison responses from both human experts and ChatGPT across various domains, including open-domain, financial, medical, legal, and psychological areas. The data collection process involved sourcing publicly available question-answering datasets and wiki texts, ensuring that the human answers were either expert-provided or high-quality user responses, thereby minimizing mislabeling and enhancing the dataset's reliability.
Usage
import datasets… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/HC3Finance.ChatDoctor_HealthCareMagicThe ChatDoctor-HealthCareMagic-100k dataset comprises 112,000 real-world medical question-and-answer pairs, providing a substantial and diverse collection of authentic medical dialogues. There is a slight risk to this dataset since there are grammatical inconsistencies in many of the questions and answers, but this can potentially help separate strong healthcare retrieval models from weak ones.
Usage
import datasets
# Download the dataset
queries =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/ChatDoctor_HealthCareMagic.simple-wiki
Dataset Card for "simple-wiki"
Dataset Summary
This dataset contains pairs of equivalent sentences obtained from Wikipedia.
Supported Tasks
Sentence Transformers training; useful for semantic search and sentence similarity.
Languages
English.
Dataset Structure
Each example in the dataset contains pairs of equivalent sentences and is formatted as a dictionary with the key "set" and a list with the sentences as "value".
{"set":… See the full description on the dataset page: https://huggingface.co/datasets/embedding-data/simple-wiki.gemma-2b-dictionary-embeddings-all-layers
Gemma-2B Dictionary Embeddings - All Layers
This dataset contains pre-computed embeddings for 77,477 English words from WordNet using the Gemma-2B model across all 27 layers.
Dataset Structure
metadata.json: Contains dataset metadata (model info, dimensions, word count)
embeddings_layer_X.pkl: Pickle files containing embeddings for layer X (0-26)
Usage
import pickle
from huggingface_hub import hf_hub_download
# Download a specific layer
layer_0_path =… See the full description on the dataset page: https://huggingface.co/datasets/LeeHarrold/gemma-2b-dictionary-embeddings-all-layers.terminal-bench-4-qwen3-8b-embeddings-train20-seed42-successes
Terminal trajectory embedding task subset
20 selected tasks, 1,281 trajectories,
and 123,849 state/action pairs. Only trajectories with numeric reward == 1 are retained.
The saved vectors are exact selected rows of the existing embeddings; the encoder
was not rerun. train/metadata.json matches both tensor row orders.
source_row_indices.json records the original row indices; selection.json
records selection parameters, source checksums and output checksums.
selected_tasks.json… See the full description on the dataset page: https://huggingface.co/datasets/Hkang/terminal-bench-4-qwen3-8b-embeddings-train20-seed42-successes.MBPPThe MBPP dataset consists of around 1,000 crowd-sourced Python programming problems, designed to be solvable by entry level programmers, covering programming fundamentals, standard library functionality, and so on. Each problem consists of a task description, code solution and 3 automated test cases. As described in the paper, a subset of the data has been hand-verified by the dataset authors to ensure quality.
Usage
import datasets
# Download the dataset
queries =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/MBPP.HumanEvalThe HumanEval dataset released by OpenAI includes 164 programming problems with a handwritten function signature, docstring, body, and several unit tests for each problem. The dataset was handcrafted by engineers and researchers at OpenAI.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/HumanEval", "queries")
documents = datasets.load_dataset("embedding-benchmark/HumanEval", "corpus")
pair_labels =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/HumanEval.FreshStack_mtebUsage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/MBPP", "queries")
documents = datasets.load_dataset("embedding-benchmark/MBPP", "corpus")
pair_labels = datasets.load_dataset("embedding-benchmark/MBPP", "default")
DS1000DS-1000 is a code generation benchmark with a thousand data science problems spanning seven Python libraries, such as NumPy and Pandas. It employs multi-criteria evaluation metrics, including functional correctness and surface-form constraints, resulting in a high-quality dataset with only 1.8% incorrect solutions among accepted Codex-002 predictions.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/DS1000", "queries")
documents =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/DS1000.WikiSQL_mtebWikiSQL is a dataset comprising 80,654 hand-annotated examples of natural language questions and corresponding SQL queries across 24,241 tables from Wikipedia.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/MBPP", "queries")
documents = datasets.load_dataset("embedding-benchmark/MBPP", "corpus")
pair_labels = datasets.load_dataset("embedding-benchmark/MBPP", "default")
embedding_dataairbnb_embeddings
Overview
This dataset consists of AirBnB listings with property descriptions, reviews, and other metadata.
It also contains text embeddings of the property descriptions as well as image embeddings of the listing image. The text embeddings were created using OpenAI's text-embedding-3-small model and the image embeddings using OpenAI's clip-vit-base-patch32 model available on Hugging Face.
The text embeddings have 1536 dimensions, while the image embeddings have 512 dimensions.… See the full description on the dataset page: https://huggingface.co/datasets/MongoDB/airbnb_embeddings.kilt-qwen-embeddingThe corpus refers to https://huggingface.co/datasets/corag/kilt-corpus
CPRet-Embeddings
CPRet-Embeddings
This repository provides the problem descriptions and their corresponding precomputed embeddings used by the CPRet retrieval server.
You can explore the retrieval server via the online demo at https://cpret.online/.
📦 Files
probs_2609.jsonlA JSONL file containing natural language descriptions of competitive programming problems.Each line is a JSON object with metadata such as problem title, platform/source OJ, URL, and full description.… See the full description on the dataset page: https://huggingface.co/datasets/coldchair16/CPRet-Embeddings.WikiAnswers
Dataset Card for "WikiAnswers"
Dataset Summary
The WikiAnswers corpus contains clusters of questions tagged by WikiAnswers users as paraphrases.
Each cluster optionally contains an answer provided by WikiAnswers users. There are 30,370,994 clusters containing an average of 25 questions per cluster. 3,386,256 (11%) of the clusters have an answer.
Supported Tasks
Sentence Transformers training; useful for semantic search and sentence similarity.… See the full description on the dataset page: https://huggingface.co/datasets/embedding-data/WikiAnswers.openshape-objaverse-embeddingssentence-compression
Dataset Card for "sentence-compression"
Dataset Summary
Dataset with pairs of equivalent sentences.
The dataset is provided "AS IS" without any warranty, express or implied.
Google disclaims all liability for any damages, direct or indirect, resulting from using the dataset.
Disclaimer: The team releasing sentence-compression did not upload the dataset to the Hub and did not write a dataset card. These steps were done by the Hugging Face team.
Supported Tasks… See the full description on the dataset page: https://huggingface.co/datasets/embedding-data/sentence-compression.PAQ_pairs
Dataset Card for "PAQ_pairs"
Dataset Summary
Pairs questions and answers obtained from Wikipedia.
Disclaimer: The team releasing PAQ QA pairs did not upload the dataset to the Hub and did not write a dataset card.
These steps were done by the Hugging Face team.
Supported Tasks
Sentence Transformers training; useful for semantic search and sentence similarity.
Languages
English.
Dataset Structure
Each example in the dataset contains… See the full description on the dataset page: https://huggingface.co/datasets/embedding-data/PAQ_pairs.canva-visual-search-embeddings
Visual Search Embedding Benchmark: Extending Canva's DINOv2 Evaluation
Executive Summary
This benchmark extends Canva's January 2025 engineering evaluation
which chose DINOv2 for production image replacement. We test three newer models released since then
against DINOv2 on 500 design-domain images (advertising posters from the CGL-Dataset).
Key Findings
Metric
Winner
Score
vs DINOv2
Recall@1
facebook/dinov2-base
1.0000
—
Recall@5… See the full description on the dataset page: https://huggingface.co/datasets/prometheus04/canva-visual-search-embeddings.coco_captions_quintets
Dataset Card for "coco_captions"
Dataset Summary
COCO is a large-scale object detection, segmentation, and captioning dataset. This repo contains five captions per image; useful for sentence similarity tasks.
Disclaimer: The team releasing COCO did not upload the dataset to the Hub and did not write a dataset card.
These steps were done by the Hugging Face team.
Supported Tasks
Sentence Transformers training; useful for semantic search and sentence… See the full description on the dataset page: https://huggingface.co/datasets/embedding-data/coco_captions_quintets.QQP_triplets
Dataset Card for "QQP_triplets"
Dataset Summary
This dataset will give anyone the opportunity to train and test models of semantic equivalence, based on actual Quora data. The data is organized as triplets (anchor, positive, negative).
Disclaimer: The team releasing Quora data did not upload the dataset to the Hub and did not write a dataset card.
These steps were done by the Hugging Face team.
Supported Tasks
Sentence Transformers training; useful for… See the full description on the dataset page: https://huggingface.co/datasets/embedding-data/QQP_triplets.flickr30k_captions_quintets
Dataset Card for "flickr30k-captions"
Dataset Summary
We propose to use the visual denotations of linguistic expressions (i.e. the set of images they describe) to define novel denotational similarity metrics, which we show to be at least as beneficial as distributional similarities for two tasks that require semantic inference. To compute these denotational similarities, we construct a denotation graph, i.e. a subsumption hierarchy over constituents and their denotations… See the full description on the dataset page: https://huggingface.co/datasets/embedding-data/flickr30k_captions_quintets.SPECTER
Dataset Card for "SPECTER"
Dataset Summary
Dataset containing triplets (three sentences): anchor, positive, and negative. Contains titles of papers.
Disclaimer: The team releasing SPECTER did not upload the dataset to the Hub and did not write a dataset card.
These steps were done by the Hugging Face team.
Dataset Structure
Each example in the dataset contains triplets of equivalent sentences and is formatted as a dictionary with the key "set" and a list with… See the full description on the dataset page: https://huggingface.co/datasets/embedding-data/SPECTER.korean-medicine-embedding-dataset
Korean Medicine Embedding Dataset (한의학 임베딩 데이터셋)
A contrastive-learning dataset for training and evaluating domain-adapted text embedding models for Korean Medicine (KM, 한의학). Each example is a query–positive–negatives triplet built from KM terminology resources and a curated KM ontology, capturing semantic relations such as disease–symptom, disease–prescription, prescription–indication, prescription–herb, and herb–indication.
This dataset accompanies the study "A Domain-Adapted… See the full description on the dataset page: https://huggingface.co/datasets/cnupo23/korean-medicine-embedding-dataset.ETP-Eval26-embeddings
ETP-Eval26 Embeddings
Vorberechnete Vektoren zum Datensatz
ETP-Eval26.
Inhalt
Eine .npz-Datei je Modell, dazu meta.json mit den Zeilenbeschriftungen.
Datei
Modell
Dimension
bge-m3.npz
BAAI/bge-m3
1024
mE5-large.npz
intfloat/multilingual-e5-large
1024
labse.npz
sentence-transformers/LaBSE
768
sphilberta.npz
bowphs/SPhilBerta
768
qwen3-emb-0.6b.npz
Qwen/Qwen3-Embedding-0.6B
1024
qwen3-emb-4b.npz
Qwen/Qwen3-Embedding-4B
2560
xlmr.npz… See the full description on the dataset page: https://huggingface.co/datasets/mschonhardt/ETP-Eval26-embeddings.