jaothan/podman_llamacpp_python

1#2. **Set the `MODEL_ENDPOINT` Environment Variable**2#In your chat application, ensure the `MODEL_ENDPOINT` environment variable is set to the correct URL of the model server. For example:3#```python4import os5import requests6 7# Get the model endpoint from the environment variable8model_service = os.getenv("MODEL_ENDPOINT", "http://127.0.0.1:8000")9 10# Example function to call the model server11def call_model(prompt):12    url = f"{model_service}/generate"13    payload = {"prompt": prompt}14    response = requests.post(url, json=payload)15    if response.status_code == 200:16        return response.json().get("response", "No response from model")17    else:18        return f"Error: {response.status_code}"19 20# Test the connection21if __name__ == "__main__":22    prompt = "Hello, model!"23    result = call_model(prompt)24    print(result)25#### 3. **Test the Connection**26#Run the chat application and test the connection to the model server:27 28#```bash29# Set the MODEL_ENDPOINT environment variable30#export MODEL_ENDPOINT="http://localhost:8001"31 32# Run the chat application33#python chat_app.py34#```35 36#If everything is set up correctly, the chat application should be able to call #the model server and receive a response.37 38 39