styly-agents/instructdesign-kontext

InstructDesign-Kontext

Fine-tuned FLUX.1 Kontext [dev] model for transforming web interfaces with natural language instructions.

Model Description

InstructDesign-Kontext is a LoRA fine-tuned version of FLUX.1 Kontext [dev] specialized in:

• —Web interface transformations
• —UI/UX redesigns
• —Style transfers (dark mode, glassmorphism, etc.)
• —Device mockup generation
• —Design system applications

Training Details

• —Base Model: FLUX.1 Kontext [dev]
• —Training Method: LoRA fine-tuning
• —Dataset: 937 web interface transformation pairs
• —Training Steps: 10,000
• —Training Time: ~2 weeks (iterative)

File Structure

├── unet/
│   └── flux1-kontext-dev.safetensors (23GB)
├── loras/
│   └── flux_kontext_lora_v4_consolidated_000010000.safetensors (2.6GB)
├── clip/
│   ├── clip_l.safetensors (235MB)
│   └── t5xxl_fp8_e4m3fn_scaled.safetensors (4.9GB)
├── vae/
│   └── ae.safetensors (320MB)
└── workflow/
    └── workflow_api.json

Usage with ComfyUI

1. Download Models

Download all model files to your ComfyUI models directory:

# Main model
ComfyUI/models/diffusion_models/flux1-kontext-dev.safetensors

# LoRA
ComfyUI/models/loras/flux_kontext_lora_v4_consolidated_000010000.safetensors

# CLIP encoders
ComfyUI/models/clip/clip_l.safetensors
ComfyUI/models/clip/t5xxl_fp8_e4m3fn_scaled.safetensors

# VAE
ComfyUI/models/vae/ae.safetensors

2. Load Workflow

Import the workflow/workflow_api.json into ComfyUI.

3. Configure Parameters

Default optimal settings:

• —Steps: 20
• —CFG: 1.0
• —Guidance: 5.0
• —Sampler: er_sde
• —Scheduler: kl_optimal
• —LoRA Strength: 1.0

API Usage

See the GitHub repository for full API implementation.

# Example API request
import requests

response = requests.post("http://localhost:8000/api/generate", json={
    "positive_prompt": "Transform this interface to dark mode with purple accents",
    "input_image": "website.png",
    "seed": 42,
    "steps": 20,
    "cfg": 1.0,
    "guidance": 5.0
})

Transformation Capabilities

Style Transformations

• —Dark/light mode conversion
• —Glassmorphism effects
• —Neumorphic design
• —Material Design application
• —iOS/Android style transfer

UI/UX Improvements

• —Layout modernization
• —Accessibility enhancements
• —Color scheme updates
• —Typography improvements
• —Interactive element additions

Mockup Generation

• —Device frame placement
• —Environment contextualization
• —Billboard/poster mockups
• —Multi-device presentations

Example Prompts

"Transform this website to dark mode with purple gradient accents"
"Apply glassmorphism effect with blurred backgrounds"
"Convert to material design 3 with rounded corners"
"Place this interface on an iPhone 15 Pro in a coffee shop"
"Modernize this dated interface with current design trends"

Performance

• —Inference Time: ~45-70 seconds (NVIDIA GPU 16GB+)
• —VRAM Requirements: 16GB minimum, 24GB recommended
• —Quality: 85%+ instruction adherence rate

Docker Deployment

Complete Docker setup available in the GitHub repository with:

• —ComfyUI container
• —FastAPI wrapper
• —Queue management
• —Auto-scaling workers

Limitations

• —Requires significant VRAM (16GB+)
• —Best results with clear, specific instructions
• —Limited to single image transformations (no batch in single inference)

Citation

If you use this model, please cite:

@misc{instructdesign-kontext,
  title={InstructDesign-Kontext: Web Interface Transformation with FLUX},
  author={tercumantanumut},
  year={2024},
  publisher={Hugging Face}
}

License

This model is subject to the FLUX.1 [dev] Non-Commercial License.

Acknowledgments

• —FLUX.1 team for the base model
• —ComfyUI for the inference framework
• —10 weeks of iterative development