ByteDance-Seed/EdgeBench
Overview EdgeBench is a benchmark of 134 real-world tasks for evaluating how autonomous AI agents learn from real-world environments. Instead of measuring one-shot performance, EdgeBench places agents in executable task environments with realistic, multi-level feedback and lets them iterate for 12+ hours per task — tracking the full trajectory of improvement, not just the final score. We publicly release 51 tasks… See the full description on the dataset page: https://huggingface.co/datasets/ByteDance-Seed/EdgeBench.
864.7k
1{2 "task_id": "ad_placement_optimization",3 "name": "Ad Placement Optimization",4 "category": "Combinatorial Optimization",5 "base_image": "cpp",6 "platform": "linux/amd64",7 "internet": false,8 "cwd": "/home/workspace/ad-placement",9 "submit_paths": [10 "solution.cpp"11 ],12 "work": {13 "image_tag": "49747cad3ebd",14 "specs_dir": "/home/workspace/ad-placement",15 "agent_query": "## Ad Placement Optimization\n\nYou need to place rectangular ads for n companies on a 10000x10000 grid.\n\nEach company i wants an ad space containing point (x_i+0.5, y_i+0.5) with area as close to r_i as possible. Maximize the total satisfaction.\n\nFull problem description, constraints, scoring formula, and input/output format are in `README.md`.\n\n## Local Testing Tools\n\n`tools/` provides an input generator and scoring program. The generator reads a seed file and writes generated cases into a directory; it does not accept a raw seed directly.\n\n```bash\n# Generate one test input\nprintf '0\\n' > /tmp/seeds.txt\nrm -rf /tmp/ad_cases && mkdir -p /tmp/ad_cases\n./tools/bin/gen /tmp/seeds.txt -d /tmp/ad_cases\ncp /tmp/ad_cases/0000.txt input.txt\n\n# Run your solution\n./my_solution < input.txt > output.txt\n\n# Score it\n./tools/bin/tester input.txt output.txt\n# stderr: Score = <number>\n```\n\nFor multiple local cases, put one seed per line in the seed file, for example `seq 0 9 > /tmp/seeds.txt`, then run `./tools/bin/gen /tmp/seeds.txt -d /tmp/ad_cases`.\n\nYou can generate unlimited test data with any seed value. Use this extensively for local testing and optimization.\n\n## Compilation\n\nRecommended: C++17 with `g++ -std=c++17 -O2`.\n\nTime limit: 5 seconds per test case. Memory limit: 1 GB. No GPU.\n\n## Rules\n\n- Write your solution as a single C++ file in the project root directory\n- Do NOT modify files in `tools/`\n- Use `tools/bin/gen` with a seed file and `tools/bin/tester` for local testing\n- Your program should read from stdin and write to stdout\n- Run your solution to completion and verify with the tester before finishing"16 },17 "judge": {18 "image_tag": "56cbfc81cfa1",19 "eval_cmd": "cd /home/workspace/ad-placement && bash /tmp/eval_ahc.sh",20 "eval_timeout": 600,21 "parser": "score_sum",22 "score_direction": "maximize",23 "selection": "score_first",24 "rescale": {25 "kind": "piecewise_max",26 "baseline": 0.0,27 "rank30": 47169682940.0,28 "rank1": 49702568341.0,29 "super_anchor": 50000000000.030 }31 }32}33