Team Ai
Modelpublic

XLabs-AI/flux-ip-adapter-v2

sourceHugging Faceapache-2.0updated 2y agoView on Hugging Face
325likes10kdownloads
Model Card

<img src="assets/banner-dark.png?raw=true" alt="Banner Picture 1" style="width:1024px;"/> <a href="https://discord.gg/FHY2guThfy"> <img src="https://github.com/XLabs-AI/x-flux/blob/main/assets/readme/light/join-our-discord-rev1.png?raw=true" style="width:1024px;"/> </a> <img src="assets/ipadapter0.jpg?raw=true" alt="example0" style="width:1024px;"/> <img src="assets/monaworkflow.jpg?raw=true" alt="Mona Anime Workflow 1" style="width:1024px;"/>

This repository provides a IP-Adapter checkpoint for FLUX.1-dev model by Black Forest Labs

See our github for comfy ui workflows.

Models

The IP adapter is trained on a resolution of 512x512 for 150k steps and 1024x1024 for 350k steps while maintaining the aspect ratio. We release v2 version - which can be used directly in ComfyUI!

Please, see our ComfyUI custom nodes installation guide

Examples

See examples of our models results below. Also, some generation results with input images are provided in "Files and versions"

Inference

To try our models, you have 2 options:

  1. 1.Use main.py from our official repo
  2. 2.Use our custom nodes for ComfyUI and test it with provided workflows (check out folder /workflows)

Instruction for ComfyUI

  1. 1.Go to ComfyUI/custom_nodes
  2. 2.Clone x-flux-comfyui, path should be ComfyUI/custom_nodes/x-flux-comfyui/, where is all the files in this repo
  3. 3.Go to ComfyUI/custom_nodes/x-flux-comfyui/ and run python setup.py
  4. 4.Update x-flux-comfy with git pull or reinstall it.
  5. 5.Download Clip-L model.safetensors from OpenAI VIT CLIP large, and put it to ComfyUI/models/clip_vision/*.
  6. 6.Download our IPAdapter from huggingface, and put it to ComfyUI/models/xlabs/ipadapters/*.
  7. 7.Use Flux Load IPAdapter and Apply Flux IPAdapter nodes, choose right CLIP model and enjoy your genereations.
  8. 8.You can find example workflow in folder workflows in this repo.

If you get bad results, try to set to play with ip strength

Limitations

The IP Adapter is currently in beta. We do not guarantee that you will get a good result right away, it may take more attempts to get a result. <img src="assets/ipadapter2.jpg?raw=true" alt="example2" style="width:1024px;"/> <img src="assets/ipadapter3.jpg?raw=true" alt="example3" style="width:1024px;"/> <img src="assets/ipadapter1.jpg?raw=true" alt="example1" style="width:1024px;"/> <img src="assets/ipadapter4.jpg?raw=true" alt="example4" style="width:1024px;"/> <img src="assets/ipadapter5.jpg?raw=true" alt="example5" style="width:1024px;"/> <img src="assets/ipadapter6.jpg?raw=true" alt="example6" style="width:1024px;"/>

License

Our weights fall under the [FLUX.1 [dev]](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md) Non-Commercial License<br/>