diffusers/FLUX.1-dev-torchao-fp8

1---2base_model: black-forest-labs/FLUX.1-dev3library_name: diffusers4base_model_relation: quantized5tags:6- quantization7---8# Visual comparison of Flux-dev model outputs using BF16 and torchao float8_weight_only quantization9 10<td style="text-align: center;">11  BF16<br>12  <medium-zoom background="rgba(0,0,0,.7)"><img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/quantization-backends-diffusers/combined_flux-dev_bf16_combined.png" alt="Flux-dev output with BF16: Baroque, Futurist, Noir styles"></medium-zoom>13</td>14<td style="text-align: center;">15  torchao fp8_weight_only<br>16  <medium-zoom background="rgba(0,0,0,.7)"><img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/quantization-backends-diffusers/combined_flux-dev_torchao_fp8_combined.png" alt="torchao fp8_weight_only Output">17  </td>18 19# Usage with Diffusers20 21To use this quantized FLUX.1 [dev] checkpoint, you need to install the 🧨 diffusers and torchao library:22 23```24pip install -U diffusers25pip install -U torchao26```27 28After installing the required library, you can run the following script: 29 30```python31from diffusers import FluxPipeline32pipe = FluxPipeline.from_pretrained(33    "diffusers/FLUX.1-dev-torchao-fp8",34    torch_dtype=torch.bfloat16,35    use_safetensors=False,36    device_map="balanced"37)38prompt = "Baroque style, a lavish palace interior with ornate gilded ceilings, intricate tapestries, and dramatic lighting over a grand staircase."39pipe_kwargs = {40    "prompt": prompt,41    "height": 1024,42    "width": 1024,43    "guidance_scale": 3.5,44    "num_inference_steps": 50,45    "max_sequence_length": 512,46}47image = pipe(48    **pipe_kwargs, generator=torch.manual_seed(0),49).images[0]50image.save("flux.png")51```52 53# How to generate this quantized checkpoint ? 54 55This checkpoint was created with the following script using "black-forest-labs/FLUX.1-dev" checkpoint:56 57```python58import torch59from diffusers import FluxPipeline60from diffusers.quantizers import PipelineQuantizationConfig61from diffusers import TorchAoConfig as DiffusersTorchAoConfig62from transformers import TorchAoConfig as TransformersTorchAoConfig63 64from torchao.quantization import Float8WeightOnlyConfig65 66pipeline_quant_config = PipelineQuantizationConfig(67    quant_mapping={68        "transformer": DiffusersTorchAoConfig("float8_weight_only"),69        "text_encoder_2": TransformersTorchAoConfig(Float8WeightOnlyConfig()),70    }71)72pipe = FluxPipeline.from_pretrained(73    "black-forest-labs/FLUX.1-dev",74    quantization_config=pipeline_quant_config,75    torch_dtype=torch.bfloat16,76    device_map="balanced"77)78# safe_serialization set to `False` as we can't save torchao quantized model to safetensors format79pipe.save_pretrained("FLUX.1-dev-torchao-fp8", safe_serialization=False)80```