OpenMOSS-Team/SWE-bench-Science
SWE-bench Science SWE-bench Science evaluates coding agents on software-engineering tasks drawn from scientific-computing repositories. The release contains 119 tasks across 20 scientific domains, with isolated environments and separate programmatic verifiers. GitHub release repository: OpenMOSS/SWE-bench-Science Runtime images: Docker Hub, pinned by immutable linux/amd64 digests Evaluation framework: Pier, compatible with Harbor task format Dataset Summary… See the full description on the dataset page: https://huggingface.co/datasets/OpenMOSS-Team/SWE-bench-Science.
Sync model-independent offline Docker networking
Remove stale selection snapshots
Align runner paths with GitHub release repository
Sync task 118 immutable image digest metadata (part 5)
Sync task 118 immutable image digest metadata (part 4)
Sync task 118 immutable image digest metadata (part 3)
Sync task 118 immutable image digest metadata (part 2)
Sync task 118 immutable image digest metadata
Replace release task 001 with audited source task 120
Publish canonical image provenance and verifier fixes (part 5)
Publish canonical image provenance and verifier fixes (part 4)
Publish canonical image provenance and verifier fixes (part 3)
Publish canonical image provenance and verifier fixes (part 2)
Publish canonical image provenance and verifier fixes
Add task category and paper/project links (#2)
Align public domain count and license wording
Discover private tests by directory (part 5)
Discover private tests by directory (part 4)
Discover private tests by directory (part 3)
Discover private tests by directory (part 2)
Discover private tests by directory
Fix baseline patch capture before verifier (part 4)
Fix baseline patch capture before verifier (part 3)
Fix baseline patch capture before verifier (part 2)
Fix baseline patch capture before verifier
Expose the dataset viewer table as the test split
Add science ablation labels and batch runner documentation
Classify the dataset as a software engineering benchmark
Handle empty material flags in the CSV viewer
Preserve task ID types in the CSV viewer
Gate task 035 government report materials
Configure tasks.csv as the Dataset Viewer train split
Use CSV as the single Dataset Viewer source
Complete upstream repository metadata for task 005
Publish remediated task image digests 091-105 (part 3)
Publish remediated task image digests 091-105 (part 2)
Publish remediated task image digests 091-105
Publish audited SWE-bench Science snapshot (part 2)
Publish audited SWE-bench Science snapshot
Publish audited task materials and Docker Hub digests (part 4)
Publish audited task materials and Docker Hub digests (part 3)
Publish audited task materials and Docker Hub digests (part 2)
Publish audited task materials and Docker Hub digests
Refresh dataset card and usage guide
Fix task 119 environment image digest (part 2)
Fix task 119 environment image digest
Publish audited task images 106-119 (part 2)
Publish audited task images 106-119
Add Parquet sample for Dataset Viewer
Add explicit train split for Dataset Viewer