abuzreq/model-bending-knowledge-base

Model Bending Knowledge Base

This dataset records what happens when you bend the inside of a diffusion model. Bending means multiplying, rotating, adding noise to or otherwise changing the activations of a layer while the model generates.

Each record names:

• —the model and the exact part of it that was bent
• —the operation, the amount, and the denoising steps it covered
• —the full generation setup
• —the output, next to an unbent baseline made with the same setup

Artists can browse it to learn what a model does when bent. Agents, such as the comfyui-model-bending skill, query it to suggest starting recipes ("more abstract on SD1.5" → which bends tend to do that).

Bends are applied with ComfyUI-Model-Bending.

17454 records, 1715 cells and 10 findings (index built 2026-10-06T00:47:55Z). Sources: author_experiment 2940, paper 5656, sweep 8858. Model families: sd1 17454.

Facts are kept apart from interpretation

Every record folder records/<family>/<source>/<id>/ holds:

Other folders:

• —`baselines/` holds the unbent renders.
• —`findings/` holds claims about many records, such as a paper's results, with their authors and citation:
• —level: cell findings back the cells they cover.
• —level: general findings give study-wide context.
• —`vocab/effects.json` is the controlled list of effect tags.
• —`schema/` holds the JSON Schemas.

Prompts and input images are published only when their owner agreed (consent). Otherwise a salted key stands in, so records can still be counted per prompt.

AI-written interpretations are always labelled with the model that wrote them. Treat them as one reading of the image, not ground truth. Humans can add their own readings next to them.

Run a record in ComfyUI

Every record has a workflow.json, and the same workflow is embedded in its output.webp:

• —Drag the image onto ComfyUI to open a graph that renders it: the checkpoint and any separate loaders, the sampler, steps, cfg, seed, size, prompt and negative, and the bend in an Apply Bends from JSON node (ComfyUI-Model-Bending) with clamping off.
• —Prompts that were not shared appear as a placeholder; put in your own.
• —Expect a close match, not a pixel-exact one: renders can differ slightly across ComfyUI and torch versions and GPUs.

Stable ids

A record id is a hash of a fixed list of facts (id_scheme):

• —model: family, arch, checkpoint
• —setup: route, seed, sampler, scheduler, steps, cfg, size, denoise, prompt (or its key), negative, input key
• —bends: path, op, arguments, step window, blend

Adding records or new fields never changes a published id. Ids from before a migration resolve through migrations/id_scheme_*.json, published as index/id_aliases.json.

An absent negative means it was not recorded; "" means it was empty.

Cells and evidence

index/cells.jsonl groups single-bend records into cells: (model family, layer group, sub-module kind, module type, op, amount bucket, step window, route). Each cell carries:

• —record, seed and prompt counts
• —the checkpoints it was tested on
• —measurement summaries
• —effect tags, with who assigned them
• —an evidence grade:
• —anecdotal: one seed and one prompt
• —multi-seed or multi-prompt
• —replicated: at least two of each
• —study-backed: a cited finding covers the cell
• —what can go wrong (see below)

Broken renders

A render counts as broken (degenerate) only when it clearly failed:

• —CLIP ViT-B/32 reads it as noise or a blob field (clip_degenerate ≥ 0.90) and it lost part of its subject (prompt_retention < 0.85, CLIP's image–prompt match relative to the unbent baseline)
• —or it is static-like noise, or an all-black or all-white frame

These cut-offs were set from the dataset owner's labels on 59 renders near the boundary, so that no image they judged usable is marked broken. The cost is that some failures go unflagged. Losing the subject, flat textures and smooth blob fields are recorded (prompt_retention, pixel_flags) but do not count as broken on their own: artists often want them.

In index/cells.jsonl, broken renders are left out of a cell's measurement summaries, effect tags and examples. Each cell still reports what can go wrong:

• —degenerate_rate and broken_reasons: how often its bend broke, and why
• —signals: warning signs over all renders:
• —subject_fades: the share with prompt retention < 0.75
• —noise_or_blob_look: the share with a CLIP degeneracy score ≥ 0.6
• —failure_notes: what failing renders looked like, with their author
• —broken_examples: the broken records themselves

Sources

Contributing a run

The knowledge base takes full runs, not single pictures:

• —one model
• —all seven regions of its U-Net
• —at least three operations with three amounts each

One prompt and seed is enough.

Make a run with the bending skill's `kb_run.py` (init → render → check → submit). It renders on your own ComfyUI, embeds each picture's workflow, checks the format and the coverage, and opens a Pull Request here with your own Hugging Face login. Measurements and descriptions are optional; the maintainer adds whatever a run leaves out. Every interpretation must name its author, and AI-written ones must name their model. The steps are in the Navigator.

Requesting a model

Ask for a model in this dataset's Discussions: a discussion titled Model request: <name>. The pinned post explains what to include (link, licence, settings, 3–5 test prompts with negatives). Vote with 👍. The most wanted models get a full run.

Likes

People signed in to Hugging Face can like renders in the Navigator, once per account and render. Only the counts are published, in community/likes.jsonl ({record, likes}), by the likes server (a Netlify function); who liked what is kept privately. Cells in index/cells.jsonl sum their renders' likes as likes.

Citation

@misc{abuzuraiq2026unboxing,
  title  = {Unboxing Diffusion Models for the Arts: Interactive Model Bending and Practice-Based Explainability},
  author = {Abuzuraiq, Ahmed M. and Pasquier, Philippe},
  year   = {2026},
  eprint = {2607.22428},
  archivePrefix = {arXiv}
}