SamuelChien821/devopsbench-100
DevOpsBench-100 DevOpsBench-100 is a synthetic long-horizon software-engineering / SRE agent benchmark: 100 tasks over one executable world ("NovaCart", a mid-size e-commerce SaaS) with 72 SQLite tables, 1451 seeded rows, a 38-file monorepo with 417 commits, and 97 MCP tools spanning a first-party engineering stack (tickets, PRs, CI, deployments, canaries, migrations, feature flags, metrics, alerts, incidents, chat, knowledge base) plus deliberately disagreeing vendor-shaped… See the full description on the dataset page: https://huggingface.co/datasets/SamuelChien821/devopsbench-100.
02.5k
1-- case_id: DOB-0332-- {"environment": "production", "migration_id": 2, "name": "0041_settlement_batches", "service": "payments", "status": "applied"}3-- {"environment": "staging", "migration_id": 1, "name": "0041_settlement_batches", "service": "payments", "status": "applied"}4 