tokenintelligence/LiveCodeBench-SnapShot-0406
LiveCodeBench Official repository for the paper "LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code" 🏠 Home Page • 💻 Data • 🏆 Leaderboard • 🔍 Explorer Introduction LiveCodeBench provides holistic and contamination-free evaluation of coding capabilities of LLMs. Particularly, LiveCodeBench continuously collects new problems over time from contests across three competition platforms -- LeetCode… See the full description on the dataset page: https://huggingface.co/datasets/tokenintelligence/LiveCodeBench-SnapShot-0406.
0106
1#!/bin/bash2set -e3 4BASE="/data/repos/agentica-rllm/scripts/train/deepcoder/checkpoints/deepcoder-1.7b"5 6run_eval() {7 local model_path="$1"8 local nickname="$2"9 echo "Running eval: $nickname"10 CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m lcb_runner.runner.main \11 --model "$model_path" \12 --nickname "$nickname" \13 --scenario codegeneration \14 --evaluate \15 --release_version v4_v6 \16 --n 256 \17 --temperature 0.6 \18 --top_p 0.95 \19 --tensor_parallel_size 8 \20 --num_process_evaluate 6021}22 23 24 25# run_eval "$BASE/adv:maxrl-cont:false/global_step_100/actor/checkpoint" maxrl_binary_10026# run_eval "$BASE/adv:tailrl-cont:true/global_step_100/actor/checkpoint" tailrl_cont_10027# run_eval "$BASE/adv:maxrl-cont:false/global_step_200/actor/checkpoint" maxrl_binary_20028# run_eval "$BASE/adv:tailrl-cont:true/global_step_200/actor/checkpoint" tailrl_cont_20029# run_eval "$BASE/adv:maxrl-cont:false/global_step_300/actor/checkpoint" maxrl_binary_30030# run_eval "$BASE/adv:tailrl-cont:true/global_step_300/actor/checkpoint" tailrl_cont_30031run_eval "$BASE/adv:maxrl-cont:false/global_step_400/actor/checkpoint" maxrl_binary_40032run_eval "$BASE/adv:tailrl-cont:true/global_step_400/actor/checkpoint" tailrl_cont_40033