thedeoxen/refcontrol-FLUX.2-klein-9B-reference-pose-lora

RefControl FLUX.2 Klein 9B – Reference Pose LoRA

📝 Short description

A LoRA for FLUX.2 Klein 9B Base that fuses a reference image (identity) with a pose control map (skeleton / keypoints). It preserves identity and style from the reference while following the body pose and structure from the pose map.

Trigger phrase: apply pose from image 1 with reference from image 2

<Gallery />

📊 Examples

Each preview is a single combined image from ComfyUI: Pose → Reference → Result (left to right).

<table> <thead> <tr> <th>Pose → Reference → Result</th> </tr> </thead> <tbody> <tr> <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-pose-lora/resolve/main/images/a (1).png" width="960" /></td> </tr> <tr> <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-pose-lora/resolve/main/images/a (2).png" width="960" /></td> </tr> <tr> <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-pose-lora/resolve/main/images/a (3).png" width="960" /></td> </tr> <tr> <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-pose-lora/resolve/main/images/a (4).png" width="960" /></td> </tr> <tr> <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-pose-lora/resolve/main/images/a (5).png" width="960" /></td> </tr> </tbody> </table> ---

📖 Extended description

This LoRA was primarily trained on humans, but it also works with stylized characters and some objects. Its main purpose is to preserve identity — facial features, hairstyle, clothing, or object details — from the reference image, while adapting the subject to the pose and skeleton structure defined by the control map.

FLUX.2 Klein 9B Base already handles reference + pose transfer reasonably well with the right prompt alone. This LoRA builds on that capability — it improves consistency and better preserves character identity and pose fidelity than the base model without LoRA.

Part of the RefControl family: reference + control fusion for consistent, controllable generation on FLUX.2 Klein 9B Base.

⚙️ How to use

1. 1.Use the first image as the pose control map (skeleton / keypoints).
2. 2.Use the second image as the reference (character, person, or object).
3. 3.Add the trigger phrase `apply pose from image 1 with reference from image 2` in your prompt.
4. 4.Adjust LoRA weight (recommended 0.8–1.0) depending on how strongly you want to preserve identity.

ComfyUI requirenments

Pose extraction in the included workflow uses DWPose (via comfyui_controlnet_aux):

https://github.com/Fannovel16/comfyuicontrolnetaux

Base model

Trained on and recommended with `black-forest-labs/FLUX.2-klein-base-9B`. The undistilled Base variant is intended for LoRA training and custom pipelines (~50 inference steps, guidance_scale ~4.0).

The LoRA also works with the 4-step distilled `black-forest-labs/FLUX.2-klein-9B` for faster inference (~4 steps, guidance_scale ~1.0), but quality may be slightly lower — especially for identity and pose fidelity — compared to the Base model.

✅ Example prompt

apply pose from image 1 with reference from image 2

🎯 What it does

• —Preserves character or object identity across generations.
• —Adapts the subject to a new pose or action defined by the skeleton map.
• —Works best when the pose map has similar proportions and scale to the reference.

⚡ Tips

• —Best results when the pose map is not drastically different in body scale or framing from the reference.
• —Combine with text prompts to refine background, lighting, or mood.
• —DWPose skeleton maps on a black background work well as control input.

📌 Use cases

• —Character posing for illustrations, comics, or storyboards.
• —Consistent character design across different poses.
• —Re-posing stylized characters while keeping their identity.
• —Creating animation keyframes from static references.

📦 Files

• —Weights: `refcontrol_v2_poses.safetensors`
• —ComfyUI workflow: `refcontrol_pose_flux_klein_9b.json`
• —Repo: thedeoxen/refcontrol-FLUX.2-klein-9B-reference-pose-lora