Successmove/tinyllama-function-calling-cpu-optimized

1---2license: apache-2.03tags:4- llm5- tinyllama6- function-calling7- cpu-optimized8- low-resource9---10 11# TinyLlama Function Calling (CPU Optimized)12 13This is a CPU-optimized version of TinyLlama that has been fine-tuned for function calling capabilities.14 15## Model Details16 17- **Base Model**: TinyLlama-1.1B-Chat-v1.018- **Parameters**: 1.1 billion19- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)20- **Training Data**: Function calling examples from Glaive Function Calling v2 dataset21- **Optimization**: Merged LoRA weights, converted to float32 for CPU deployment22 23## Key Features24 251. **Function Calling Capabilities**: The model can identify when functions should be called and generate appropriate function call syntax262. **CPU Optimized**: Ready to run efficiently on low-end hardware without GPUs273. **Lightweight**: Only 1.1B parameters, making it suitable for older hardware284. **Low Resource Requirements**: Requires only 4-6 GB RAM for loading29 30## Usage31 32```python33from transformers import AutoModelForCausalLM, AutoTokenizer34import torch35 36# Load the model37model = AutoModelForCausalLM.from_pretrained("tinyllama-function-calling-cpu-optimized")38tokenizer = AutoTokenizer.from_pretrained("tinyllama-function-calling-cpu-optimized")39 40# Example prompt for function calling41prompt = """### Instruction:42Given the available functions and the user query, determine which function(s) to call and with what arguments.43 44Available functions:45{46    "name": "get_exchange_rate",47    "description": "Get the exchange rate between two currencies",48    "parameters": {49        "type": "object",50        "properties": {51            "base_currency": {52                "type": "string",53                "description": "The currency to convert from"54            },55            "target_currency": {56                "type": "string",57                "description": "The currency to convert to"58            }59        },60        "required": [61            "base_currency",62            "target_currency"63        ]64    }65}66 67User query: What is the exchange rate from USD to EUR?68 69### Response:"""70 71# Tokenize and generate response72inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=512)73with torch.no_grad():74    outputs = model.generate(75        **inputs,76        max_new_tokens=150,77        do_sample=True,78        temperature=0.7,79        top_k=50,80        top_p=0.9581    )82 83response = tokenizer.decode(outputs[0], skip_special_tokens=True)84print(response)85```86 87## Performance on Low-End Hardware88 89The CPU-optimized model requires approximately:90- 4-6 GB RAM for loading91- 2-4 CPU cores for inference92- No GPU required93 94This makes it suitable for:95- Older laptops (2018 and newer)96- Low-end desktops97- Edge devices with ARM processors98 99## Training Process100 101The model was fine-tuned using LoRA (Low-Rank Adaptation) on the Glaive Function Calling v2 dataset. Only a subset of 50 examples was used for demonstration purposes.102 103## License104 105This model is licensed under the Apache 2.0 license.