gabrielpondc/NL2SQL-StarCoder-15B

1---2 3 4frameworks:5 6- Pytorch7license: other8tasks:9- text-generation10 11---12 13# Model Card for NL2SQL-StarCoder-15B14 15## Model Inro16 17NL2SQL-StarCoder-15B is a NLP-SQL model fintuned by QLoRA based on StarCoder 15B Code-LLM。 18 19## Requirements20 21- python>=3.822- pytorch>=2.0.023- transformers==4.32.024- CUDA 11.425## Data Format26 27The data is in the form of a string spliced by the model in the training data format, which is also how the input PROMPT is spliced during inference:28```python29"""30<|user|>31/* Given the following database schema: */32CREATE TABLE "table_name" (33"col1" int,34...35...36)37 38/* Write a sql to answer the following question: {Question} */39<|assistant|>40```sql41{Output SQL}42```<|end|>43"""44```45 46But from test we recomended using the promt what sqlcoder was given:47 48```49### Instructions:50Your task is to convert a question into a SQL query, given a Postgres database schema.51Adhere to these rules:52- **Deliberately go through the question and database schema word by word** to appropriately answer the question53- **Use Table Aliases** to prevent ambiguity. For example, `SELECT table1.col1, table2.col1 FROM table1 JOIN table2 ON table1.id = table2.id`.54- When creating a ratio, always cast the numerator as float55### Input:56Generate a SQL query that answers the question `{question}`.57This query will run on a database whose schema is represented in this string:58CREATE TABLE "table_name" (59"col1" int,60...61...62)63### Response:64Based on your instructions, here is the SQL query I have generated to answer the question `{question}`:65```sql66 67 68```69 70## Quick Start71 72```python73import torch74from transformers import AutoModelForCausalLM, AutoTokenizer75model_dir = "gabrielpondc/NL2SQL-StarCoder-15B"76tokenizer = AutoTokenizer.from_pretrained(model_dir, device_map="auto", 77                              trust_remote_code=True, torch_dtype=torch.float16)78tokenizer.padding_side = "left"79tokenizer.pad_token_id = tokenizer.convert_tokens_to_ids("<fim_pad>")80tokenizer.eos_token_id = tokenizer.convert_tokens_to_ids("<|endoftext|>")81tokenizer.pad_token = "<fim_pad>"82tokenizer.eos_token = "<|endoftext|>"83 84model = AutoModelForCausalLM.from_pretrained(model_dir, device_map="auto", 85                              trust_remote_code=True, torch_dtype=torch.float16)86model.eval()87 88text = '<|user|>\n/* Given the following database schema: */\nCREATE TABLE "singer" (\n"Singer_ID" int,\n"Name" text,\n"Country" text,\n"Song_Name" text,\n"Song_release_year" text,\n"Age" int,\n"Is_male" bool,\nPRIMARY KEY ("Singer_ID")\n)\n\n/* Write a sql to answer the following question: Show countries where a singer above age 40 and a singer below 30 are from. */<|end|>\n'89inputs = tokenizer(text, return_tensors='pt', padding=True, add_special_tokens=False).to("cuda")90outputs = model.generate(91        inputs=inputs["input_ids"],92        attention_mask=inputs["attention_mask"],93        max_new_tokens=512,94        top_p=0.95,95        temperature=0.1,96        do_sample=False,97        eos_token_id=tokenizer.eos_token_id,98        pad_token_id=tokenizer.pad_token_id99    )100gen_text = tokenizer.batch_decode(outputs[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)101print(gen_text)102```103 104