Vaishnavi-279/sql-debugger-env

1---2title: SQL Debugger Environment3emoji: 🛠️4colorFrom: blue5colorTo: indigo6sdk: docker7pinned: false8tags:9  - openenv10  - rl11  - sql12  - debugging13  - agent14---15 16# SQL Debugger — OpenEnv RL Environment17 18An OpenEnv reinforcement learning environment where an AI agent must debug and fix broken SQL queries.19 20The agent receives a broken SQL query and database schema each episode and iteratively submits corrected queries until the result exactly matches the expected output — or it runs out of attempts.21 22## Motivation23 24SQL debugging is a genuine, high-value daily task for developers and data engineers. Production queries silently return wrong data, new engineers inherit broken legacy code, and LLMs frequently generate subtly wrong SQL. This environment trains and evaluates agents on exactly this skill, with deterministic grading and 5-tier partial reward signals at every step.25 26## Environment Summary27 28| Property              | Value                                 |29| --------------------- | ------------------------------------- |30| Max steps per episode | 5                                     |31| Reward range          | 0.0 – 1.0                             |32| Tasks                 | 3 (easy → medium → hard)              |33| Database              | In-memory SQLite (zero deps)          |34| Server port           | 7860                                  |35| Termination           | Perfect match OR max attempts reached |36 37## Action Space38 39```json40{41  "sql": "SELECT name, salary FROM employees WHERE dept='Engineering' ORDER BY salary DESC"42}43```44 45| Field | Type   | Description                                 |46| ----- | ------ | ------------------------------------------- |47| `sql` | string | A corrected SQL SELECT statement to execute |48 49## Observation Space50 51| Field              | Type           | Description                        |52| ------------------ | -------------- | ---------------------------------- |53| `task_id`          | string         | Task identifier                    |54| `difficulty`       | string         | easy / medium / hard               |55| `task_description` | string         | What the query must return         |56| `schema_ddl`       | string         | CREATE TABLE + INSERT statements   |57| `broken_query`     | string         | The original broken query to fix   |58| `error_message`    | string         | SQLite error from last attempt     |59| `execution_result` | list or null   | Rows from last submitted query     |60| `expected_result`  | list           | Ground-truth rows to match exactly |61| `attempt`          | int            | Attempts made this episode         |62| `max_attempts`     | int            | Max allowed (5)                    |63| `hint`             | string or null | Optional bug-type hint             |64| `done`             | bool           | Episode ended                      |65| `reward`           | float          | Reward from last step              |66 67## Reward Function68 69| Condition                             | Reward              |70| ------------------------------------- | ------------------- |71| Query errored / no output             | 0.0                 |72| Wrong row count                       | 0.3                 |73| Right count, wrong values             | 0.6                 |74| Right values, wrong order             | 0.8                 |75| Perfect match                         | 1.0                 |76| Efficiency penalty per wasted attempt | -0.05 x (attempt-1) |77 78## Tasks79 80### Easy — task_easy_syntax81 82Bug: Three keyword typos (SELEC, FORM, DESK)83Schema: employees(id, name, dept, salary)84Goal: Return Engineering employees ordered by salary DESC85 86### Medium — task_medium_join87 88Bug: Wrong JOIN column + wrong aggregation expression89Schema: customers(id, name) + orders(id, customer_id, quantity, unit_price)90Goal: Return customer names with total order value (quantity x unit_price)91 92### Hard — task_hard_complex93 94Bug: 3 simultaneous bugs — NULL not excluded in subquery, LEFT JOIN skews averages, ORDER BY ASC should be DESC95Schema: products(id, name) + reviews(id, product_id, score) — score is nullable96Goal: Return products with above-average review scores97 98## Setup99 100### Build and run with Docker101 102```bash103docker build -t sql_debugger_env-env:latest .104docker run -p 7860:7860 sql_debugger_env-env:latest105```106 107### Test the server108 109```bash110curl http://localhost:7860/health111curl -X POST http://localhost:7860/reset -H "Content-Type: application/json" -d "{}"112```113 114### Run inference115 116```bash117export HF_TOKEN=your_token_here118export IMAGE_NAME=sql_debugger_env-env:latest119export API_BASE_URL=https://router.huggingface.co/v1120export MODEL_NAME=Qwen/Qwen2.5-72B-Instruct121python inference.py122```123 124## Baseline Scores125 126| Task              | Difficulty | Expected Score |127| ----------------- | ---------- | -------------- |128| task_easy_syntax  | Easy       | ~0.85          |129| task_medium_join  | Medium     | ~0.65          |130| task_hard_complex | Hard       | ~0.40          |131| Overall           |            | ~0.63          |132 133## Project Structure134 135my-openenv/136├── inference.py137├── README.md138└── sql_debugger_env/139├── models.py140├── tasks.py141├── executor.py142├── client.py143├── openenv.yaml144├── pyproject.toml145├── uv.lock146├── Dockerfile147└── server/148├── app.py149├── sql_debugger_env_environment.py150└── requirements.txt151 152## API Endpoints153 154| Method | Endpoint  | Description                      |155| ------ | --------- | -------------------------------- |156| GET    | /health   | Returns healthy status           |157| GET    | /metadata | Environment name and description |158| GET    | /schema   | Action and observation schemas   |159| POST   | /reset    | Start new episode                |160| POST   | /step     | Submit corrected SQL query       |161| GET    | /state    | Current episode state            |162 163## Environment Variables164 165| Variable          | Required | Default                          | Description         |166| ----------------- | -------- | -------------------------------- | ------------------- |167| HF_TOKEN          | yes      | —                                | HuggingFace token   |168| API_BASE_URL      | no       | https://router.huggingface.co/v1 | LLM endpoint        |169| MODEL_NAME        | no       | Qwen/Qwen2.5-72B-Instruct        | Model identifier    |170| IMAGE_NAME        | no       | sql_debugger_env-env:latest      | Docker image        |171| SQL_DEBUGGER_TASK | no       | task_easy_syntax                 | Which task to serve |172