Datasetpublic

pyterrier/hotpotqa.pisa

hotpotqa.pisa Description A PISA index for the Hotpot QA dataset Usage # Load the artifact import pyterrier as pt index = pt.Artifact.from_hf('pyterrier/hotpotqa.pisa') index.bm25() # returns a BM25 retriever Benchmarks hotpotqa/dev name nDCG@10 R@1000 bm25 0.6525 0.8909 dph 0.6445 0.8888 hotpotqa/test name nDCG@10 R@1000 bm25 0.6318 0.8851 dph 0.6246 0.8837 Reproduction import… See the full description on the dataset page: https://huggingface.co/datasets/pyterrier/hotpotqa.pisa.

sourceHugging Faceupdated 2y agoView on Hugging Face

0likes37downloads

Dataset Card

hotpotqa.pisa

Description

A PISA index for the Hotpot QA dataset

Usage

python

# Load the artifact
import pyterrier as pt
index = pt.Artifact.from_hf('pyterrier/hotpotqa.pisa')
index.bm25() # returns a BM25 retriever

Benchmarks

hotpotqa/dev

name	nDCG@10	R@1000
bm25	0.6525	0.8909
dph	0.6445	0.8888

hotpotqa/test

name	nDCG@10	R@1000
bm25	0.6318	0.8851
dph	0.6246	0.8837

Reproduction

python

import pyterrier as pt
from tqdm import tqdm
import ir_datasets
from pyterrier_pisa import PisaIndex
index = PisaIndex("hotpotqa.pisa", threads=16)
dataset = ir_datasets.load('beir/hotpotqa')
docs = ({'docno': d.doc_id, 'text': d.default_text()} for d in tqdm(dataset.docs))
index.index(docs)

Metadata

{
  "type": "sparse_index",
  "format": "pisa",
  "package_hint": "pyterrier-pisa",
  "stemmer": "porter2"
}