Felipe97/llama-cpp-compiled

1# Gemma 3 vision2 3> [!IMPORTANT]4>5> This is very experimental, only used for demo purpose.6 7## Quick started8 9You can use pre-quantized model from [ggml-org](https://huggingface.co/ggml-org)'s Hugging Face account10 11```bash12# build13cmake -B build14cmake --build build --target llama-mtmd-cli15 16# alternatively, install from brew (MacOS)17brew install llama.cpp18 19# run it20llama-mtmd-cli -hf ggml-org/gemma-3-4b-it-GGUF21llama-mtmd-cli -hf ggml-org/gemma-3-12b-it-GGUF22llama-mtmd-cli -hf ggml-org/gemma-3-27b-it-GGUF23 24# note: 1B model does not support vision25```26 27## How to get mmproj.gguf?28 29Simply to add `--mmproj` in when converting model via `convert_hf_to_gguf.py`:30 31```bash32cd gemma-3-4b-it33python ../llama.cpp/convert_hf_to_gguf.py --outfile model.gguf --outtype f16 --mmproj .34# output file: mmproj-model.gguf35```36 37## How to run it?38 39What you need:40- The text model GGUF, can be converted using `convert_hf_to_gguf.py`41- The mmproj file from step above42- An image file43 44```bash45# build46cmake -B build47cmake --build build --target llama-mtmd-cli48 49# run it50./build/bin/llama-mtmd-cli -m {text_model}.gguf --mmproj mmproj.gguf --image your_image.jpg51```52