replication
replicationbenchReplicationBench
ReplicationBench
arXiv: ReplicationBench: Can AI Agents Replicate Astrophysics Research Papers?
GitHub: https://github.com/Christine8888/replicationbench-release
Dataset Description
The ReplicationBench dataset contains 111 astrophysics research replication tasks, spanning complete replications of 20 research papers. The dataset includes:
Original and masked manuscript text
Metadata (title, abstract, publication info, etc.)
Pointers to datasets and dataset access… See the full description on the dataset page: https://huggingface.co/datasets/ChristineYe8/ReplicationBench.jmh-grpo-replication-data
JMH-Bench GRPO replication data
Data for replicating the BSc thesis Reinforcement Learning with Verifiable Execution Rewards
for Synthesizing Regression-Detecting Microbenchmarks. The code is in the replication repository
RL4JMH-thesis-submission. The trained models are
run 9 (evaluated in the
thesis) and run 8.
Path
Contents
classpaths/
Mutant-bearing jars and dependency jars of the 246 training libraries, as used by GRPO run 9 (1,764 jars, 1.2 GB)… See the full description on the dataset page: https://huggingface.co/datasets/bookxd/jmh-grpo-replication-data.replicationbench
ReplicationBench Dataset
Dataset Description
A benchmark to evaluate AI agents in astrophysics research through replicating existing research papers.
Dataset Structure
Data Splits
ReplicationBench (source: epxert): Core expert-written benchmark
ReplicationBench-Plus (source: showyourwork): Extension dataset generated through hybrid LLM-expert system
Data Configurations
Each split contains two types of data:
metadata: Paper metadata and… See the full description on the dataset page: https://huggingface.co/datasets/replicationbench-submission/replicationbench.lifemem-replication-src
LifeMem replication (arXiv:2608.19621)
Replication of Mitigating Identity Essentialism in LLM Agents with Longitudinal Life Trajectories (Wang, Zhou, Du, Su, Cao, Pan, Ai, Wu, Zhang, Liu). Official code: halsayxi/LifeMem.
The paper's claim: static demographic prompts make LLM survey agents essentialist — within-group answers collapse, SES clusters separate. LifeMem stores life events in a hippocampal retriever (top-K=5, α=0.9) and consolidates them into a per-agent LoRA adapter.… See the full description on the dataset page: https://huggingface.co/datasets/mtorres98/lifemem-replication-src.dpst-replication
DPST Replication
Replication package for the EMNLP 2025 paper:
Leveraging Semantic Triples for Private Document Generation with Local Differential Privacy GuaranteesStephen Meisenbacher, Maulik Chevli, Florian Mattheshttps://aclanthology.org/2025.emnlp-main.455/
All credit goes to the original authors. This repository provides a replication environment for running the method, based on the official released code: https://github.com/sjmeis/DPST.… See the full description on the dataset page: https://huggingface.co/datasets/weijunl/dpst-replication.