reranking
scidocs-rerankingaskubuntudupquestions-rerankingCMedQAv1-reranking
CMedQAv1-reranking
An MTEB dataset
Massive Text Embedding Benchmark
Chinese community medical question answering
Task category
t2t
Domains
Medical, Written
Reference
https://github.com/zhangsheng93/cMedQA
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["CMedQAv1-reranking"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/CMedQAv1-reranking.CMedQAv2-reranking
CMedQAv2-reranking
An MTEB dataset
Massive Text Embedding Benchmark
Chinese community medical question answering
Task category
t2t
Domains
Medical, Written
Reference
https://github.com/zhangsheng93/cMedQA2
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["CMedQAv2-reranking"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/CMedQAv2-reranking.juristcu-reranking
JurisTCUReranking
Genuine legal-domain reranking: for each of 150 queries, rerank the ~100 first-stage candidates (BM25 top-100 over the full 16k-document TCU jurisprudence corpus, unioned with human-judged docs) under graded relevance 0-3. Candidates include lexical hard negatives, so the task is distinct from first-stage retrieval. Complements QuatiReranking (web) with a legal-domain reranking probe.
Part of MTEB-BR — the native Brazilian-Portuguese MTEB sub-benchmark. Task… See the full description on the dataset page: https://huggingface.co/datasets/MTEB-BR/juristcu-reranking.quati-reranking
QuatiReranking
Genuine web-domain reranking: for each of 50 PT-BR queries, rerank the ~100 first-stage candidates (BM25 top-100 over the full 1M-passage Quati Brazilian web corpus, unioned with human-judged passages) under graded relevance 0-3. Candidates include lexical hard negatives, so the task is distinct from first-stage retrieval. Complements JurisTCUReranking (legal) with a web-domain probe.
Part of MTEB-BR — the native Brazilian-Portuguese MTEB sub-benchmark. Task type:… See the full description on the dataset page: https://huggingface.co/datasets/MTEB-BR/quati-reranking.