Team Ai
Modelpublic

aifoundry-org/FLUX.1-schnell-Quantized

sourceHugging Faceapache-2.0updated 2y agoView on Hugging Face
16likes793downloads
Model Card

Quantized versions of https://huggingface.co/black-forest-labs/FLUX.1-schnell

Tools used for quantization: modded stable-diffusion.cpp, LlamaQuantizer

Work in progress, use at your own risk

How to:

[WIP]

  1. 1.Dowload and build stable-diffusion.cpp
  2. 2.Download one of the models from this repo and
  3. 3.Autoencoder https://huggingface.co/black-forest-labs/FLUX.1-schnell/resolve/main/ae.safetensors
  4. 4.CLIPL https://huggingface.co/comfyanonymous/fluxtextencoders/blob/main/clipl.safetensors
  5. 5.T5XXL https://huggingface.co/comfyanonymous/fluxtextencoders/blob/main/t5xxl_fp16.safetensors
  6. 6.Enter your stable-diffusion.cpp dir
  7. 7.Run the following command:
   ./build/bin/sd --diffusion-model [path to gguf] --vae [path to ae.safetensors] --clip_l [path to clip_l.safetensors] --t5xxl [path to t5xxl_fp16.safetensors] -p "a frog holding a sign saying 'hi' " -o ../frog.png -v --cfg-scale 1.0 --sampling-method euler -v --seed 42 --steps 4

Results:

<table style="border-collapse: collapse; width: 100%;"> <tr> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"><strong>Quant type</strong></td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"><strong>Size</strong></td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em; min-width: 256px;"><strong>Result (x0.5)</strong></td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"><strong>Download link</strong></td> </tr> <tr> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong>default</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong>23.8 GB</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/fluxfrogdefault.png"> <img src="./examples/fluxfrogdefault.png" alt="fluxfrogdefault.png" style="display: block; margin: 0 auto; min-width: 256px; width: 256px; height: 256px; aspect-ratio: 1 / 1; object-fit: cover;"> </a> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <a href="https://huggingface.co/black-forest-labs/FLUX.1-schnell/resolve/main/flux1-schnell.safetensors">flux1-schnell.safetensors.gguf</a> </td> </tr> <tr> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong>FP16</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong> 23.8 GB</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/fluxfrogF16.png"> <img src="./examples/fluxfrogF16.png" alt="fluxfrogF16.png" style="display: block; margin: 0 auto; min-width: 256px; width: 256px; height: 256px; aspect-ratio: 1 / 1; object-fit: cover;"> </a> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-F16.gguf">flux1-schnell-F16.gguf</a> </td> </tr> <tr> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong>Q80</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong> 12.6 GB</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/fluxfrogQ80.png"> <img src="./examples/fluxfrogQ80.png" alt="fluxfrogQ80.png" style="display: block; margin: 0 auto; min-width: 256px; width: 256px; height: 256px; aspect-ratio: 1 / 1; object-fit: cover;"> </a> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-Q80.gguf">flux1-schnell-Q80.gguf</a> </td>

<tr> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong>Q50</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong> 8.18 GB</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/fluxfrogQ50.png"> <img src="./examples/fluxfrogQ50.png" alt="fluxfrogQ50.png" style="display: block; margin: 0 auto; min-width: 256px; width: 256px; height: 256px; aspect-ratio: 1 / 1; object-fit: cover;"> </a> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-Q50.gguf">flux1-schnell-Q50.gguf</a> </td> </tr> <tr> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong>Q51</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong> 8.92 GB</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/fluxfrogQ51.png"> <img src="./examples/fluxfrogQ51.png" alt="fluxfrogQ51.png" style="display: block; margin: 0 auto; min-width: 256px; width: 256px; height: 256px; aspect-ratio: 1 / 1; object-fit: cover;"> </a> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-Q51.gguf">flux1-schnell-Q51.gguf</a> </td> </tr> <tr> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong>Q40</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong> 6.69 GB</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/fluxfrogQ40.png"> <img src="./examples/fluxfrogQ40.png" alt="fluxfrogQ40.png" style="display: block; margin: 0 auto; min-width: 256px; width: 256px; height: 256px; aspect-ratio: 1 / 1; object-fit: cover;"> </a> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-Q40.gguf">flux1-schnell-Q40.gguf</a> </td> </tr> <tr> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong>Q41</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong> 7.43 GB</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/fluxfrogQ41.png"> <img src="./examples/fluxfrogQ41.png" alt="fluxfrogQ41.png" style="display: block; margin: 0 auto; min-width: 256px; width: 256px; height: 256px;"> </a> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-Q41.gguf">flux1-schnell-Q41.gguf</a> </td> </tr> <tr> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong>Q4K</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong> 6.69 GB</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/fluxfrogQ4K.png"> <img src="./examples/fluxfrogQ4K.png" alt="fluxfrogQ4K.png" style="display: block; margin: 0 auto; min-width: 256px; width: 256px; height: 256px; aspect-ratio: 1 / 1; object-fit: cover;"> </a> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-Q4K.gguf">flux1-schnell-Q4K.gguf</a> </td> </tr> <tr> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong>Q2K</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <strong> 3.9 GB</strong> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/fluxfrogQ2K.png"> <img src="./examples/fluxfrogQ2K.png" alt="fluxfrogQ2K.png" style="display: block; margin: 0 auto; min-width: 256px; width: 256px; height: 256px; aspect-ratio: 1 / 1; object-fit: cover;"> </a> </td> <td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> <a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-Q2K.gguf">flux1-schnell-Q2K.gguf</a> </td> </tr>

</table>