Angshul/SparseGeometricRAG
SparseGeometricRAG CPU-first sparse geometric retrieval for practical top-10 RAG No transformer inference at retrieval time. No retrieval GPU requirement. No dense document-vector dot products. No external API. SparseGeometricRAG is a retrieval system built around one systems objective: make the retrieval layer cheap enough to run on ordinary multicore CPU hardware without turning the corpus into a dense embedding database. It uses sparse TF-IDF geometry, fuzzy… See the full description on the dataset page: https://huggingface.co/datasets/Angshul/SparseGeometricRAG.
0331
1# Reproducibility2 3## Environment4 5```bash6python -m venv .venv7source .venv/bin/activate8pip install -U pip9pip install -e .10```11 12Optional ANN/neural baselines:13 14```bash15pip install -r requirements-baselines.txt16```17 18## SciFact19 20Place a standard BEIR archive at `data/scifact.zip` and run:21 22```bash23./scripts/reproduce_scifact.sh data/scifact.zip artifacts/scifact_index24```25 26## TREC-COVID27 28Place a standard BEIR archive at `data/trec-covid.zip` and run:29 30```bash31./scripts/reproduce_treccovid.sh data/trec-covid.zip artifacts/treccovid_index32```33 34## One configuration35 36```bash37python experiments/beir/run_rag_top10.py data/trec-covid.zip artifacts/treccovid_index --pool 100 --hq-branches 10 --lambda-diversity 0.1 --output results/reproduced/treccovid_p100.json38```39 40## Full MS MARCO41 42The exact historical full-scale scripts are preserved under `experiments/msmarco_scale/`. They are intentionally kept close to the scripts that produced the recorded JSON files. The 8.84M corpus shards and multi-GB generated arrays are not in this repository. Use `manifests/msmarco_manifest.json` to verify the shard set, then follow the build order described in the root README.43 44## Exact experiment history vs cleaned runner45 46`experiments/beir/*exact_history.py` contains the scripts as executed in the current session, including their original local paths. `experiments/beir/run_rag_top10.py` and `run_pool_sweep.py` are cleaned path-independent runners using the same formulas.47