RenderFormer/renderformer-v2-demo

RenderFormer V2

A demo of **RenderFormer-V2: Neural Rendering with Heterogeneous Scene Primitives**.

RenderFormer replaces the path tracer: it takes a triangle soup with per-triangle material and emission tokens plus a camera, and directly predicts the HDR image — global illumination, soft shadows, glossy reflections and refraction included, with no ray tracing and no per-scene optimisation.

• —model: `RenderFormer/renderformer-v2` (transformer_512)
• —code: iamNCJ/RenderFormer-Studio

What this Space does

Upstream, a renderable RenderFormer-V2 scene is produced by a Blender/Cycles data pipeline. Blender is only used there to (a) render the path-traced ground truth and (b) report camera-to-world matrices — neither is needed to run the model, and the mvp matrix stored alongside is never read by the transformer. So this Space rebuilds the scene tensors entirely on CPU with trimesh plus the checkpoint's own BRDF→latent mappers (scene_prep.py), and the GPU only ever runs the transformer itself.

Two tabs:

• —Paper scenes — three scenes shipped with the repository, with camera orbit, tone-mapper choice, and live material replacement on the hero object.
• —Your own mesh — drop an .obj / .glb / .ply into the authors' studio set (floor, two coloured walls, one area light) and render it.

Asset attribution

• —Transparent torus and Cube pile scenes — authored by the RenderFormer authors, MIT-licensed, from the RenderFormer-Studio repository.
• —Dragon — “Dragon” by Delatronic, CC BY 3.0, via Benedikt Bitterli's Rendering Resources; scene setup by the RenderFormer authors.
• —Constant-width surface example mesh — CC0 1.0, Keenan Crane.