Inferless/LLM-Inference-Benchmark

1---2title: LLM Inference Benchmark3emoji: 🥇4colorFrom: green5colorTo: indigo6sdk: gradio7app_file: app.py8pinned: true9license: apache-2.010---11 12# Start the configuration13 14Most of the variables to change for a default leaderboard are in `src/env.py` (replace the path for your leaderboard) and `src/about.py` (for tasks).15 16Results files should have the following format and be stored as json files:17```json18{19    "config": {20        "model_dtype": "torch.float16", # or torch.bfloat16 or 8bit or 4bit21        "model_name": "path of the model on the hub: org/model",22        "model_sha": "revision on the hub",23    },24    "results": {25        "task_name": {26            "metric_name": score,27        },28        "task_name2": {29            "metric_name": score,30        }31    }32}33```34 35Request files are created automatically by this tool.36 37If you encounter problem on the space, don't hesitate to restart it to remove the create eval-queue, eval-queue-bk, eval-results and eval-results-bk created folder.38 39# Code logic for more complex edits40 41You'll find 42- the main table' columns names and properties in `src/display/utils.py`43- the logic to read all results and request files, then convert them in dataframe lines, in `src/leaderboard/read_evals.py`, and `src/populate.py`44- the logic to allow or filter submissions in `src/submission/submit.py` and `src/submission/check_validity.py`