diffusers/FLUX.1-dev-bnb-8bit

1---2base_model: black-forest-labs/FLUX.1-dev3library_name: diffusers4base_model_relation: quantized5tags:6- quantization7---8 9# Visual comparison of Flux-dev model outputs using BF16 and BnB 8-bit quantization10 11<td style="text-align: center;">12  BF16<br>13  <medium-zoom background="rgba(0,0,0,.7)"><img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/quantization-backends-diffusers/combined_flux-dev_bf16_combined.png" alt="Flux-dev output with BF16: Baroque, Futurist, Noir styles"></medium-zoom>14</td>15<td style="text-align: center;">16  BnB 8-bit<br>17  <medium-zoom background="rgba(0,0,0,.7)"><img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/quantization-backends-diffusers/combined_flux-dev_bnb_8bit_combined.png" alt="Flux-dev output with BnB 8-bit: Baroque, Futurist, Noir styles"></medium-zoom>18</td>19 20# Usage with Diffusers21 22To use this quantized FLUX.1 [dev] checkpoint, you need to install the 🧨 diffusers and bitsandbytes library:23 24```25pip install -U diffusers26pip install -U bitsandbytes27```28 29After installing the required library, you can run the following script: 30 31```python32from diffusers import FluxPipeline33 34pipe = FluxPipeline.from_pretrained(35    "diffusers/FLUX.1-dev-bnb-8bit",36    torch_dtype=torch.bfloat1637)38pipe.to("cuda")39 40prompt = "Baroque style, a lavish palace interior with ornate gilded ceilings, intricate tapestries, and dramatic lighting over a grand staircase."41 42pipe_kwargs = {43    "prompt": prompt,44    "height": 1024,45    "width": 1024,46    "guidance_scale": 3.5,47    "num_inference_steps": 50,48    "max_sequence_length": 512,49}50 51image = pipe(52    **pipe_kwargs, generator=torch.manual_seed(0),53).images[0]54 55image.save("flux.png")56```57 58# How to generate this quantized checkpoint ? 59 60This checkpoint was created with the following script using "black-forest-labs/FLUX.1-dev" checkpoint:61 62```python63 64import torch65from diffusers import FluxPipeline66from diffusers import BitsAndBytesConfig as DiffusersBitsAndBytesConfig67from diffusers.quantizers import PipelineQuantizationConfig68from transformers import BitsAndBytesConfig as TransformersBitsAndBytesConfig69 70pipeline_quant_config = PipelineQuantizationConfig(71    quant_mapping={72        "transformer": DiffusersBitsAndBytesConfig(load_in_8bit=True),73        "text_encoder_2": TransformersBitsAndBytesConfig(load_in_8bit=True),74    }75)76 77pipe = FluxPipeline.from_pretrained(78    "black-forest-labs/FLUX.1-dev",79    quantization_config=pipeline_quant_config,80    torch_dtype=torch.bfloat1681)82 83pipe.save_pretrained("FLUX.1-dev-bnb-8bit")84```