dpr
Datasets
All datasets matching “dpr”wiki_dprThis is the wikipedia split used to evaluate the Dense Passage Retrieval (DPR) model.
It contains 21M passages from wikipedia along with their DPR embeddings.
The wikipedia articles were split into multiple, disjoint text blocks of 100 words as passages.dprobe-resultswiki_dpr_dummyThis dummy dataset is used for testing purpose for rag model in transformers. It is proudced via the following steps:
dataset = datasets.load_dataset("wiki_dpr", with_embeddings=True, with_index=True, index_name="exact", embeddings_name="nq", dummy=True, revision=None)
dataset["train"].drop_index("embeddings")
dataset.push_to_hub("hf-internal-testing/wiki_dpr_dummy", token="...")
The index file `index.faiss` (after being renamed locally) is then uploaded manually.
wiki_dprnq_corpus_dpr
Dataset Card for "nq_corpus_dpr"
More Information needed
wiki_dpr_e5wiki_dpr encoded with intfloat/e5-base-v2