Plate to HDRI: a real 360° IBL from one photo, with an emitter gate that knows what glows

A single plate becomes a scene-linear 360° HDRI you can light a shot with. That's the whole job of the env-to-hdri stage in my VFX pipeline. This is the real version of the thing I relabeled a few days ago: the drag-in ComfyUI graph fakes a latlong and leaves a faint seam, this one builds the actual sphere and heals it. I wrote the whole method up publicly, failures and all: comfyui-examples/10-env-to-hdri.

The pipeline is four models in a trench coat. HunyuanWorld-1.0 does the true-360 extrapolation (one plate in, a 1920×960 equirectangular panorama out), running as a subprocess in its own venv (torch 2.5 / diffusers 0.34) so its FLUX-Fill dependency never collides with the rest of the stack. A ComfyUI pass then wrap-pads and upscales it 2× (4x-UltraSharp + TiledDiffusion, cropped back), and a second ComfyUI pass heals the wrap seam by rolling the pano 50%, inpainting the centre strip, and rolling it back. The result lands at 3840×1920. A BLIP caption of the plate steers the extrapolation when no prompt is given, so the invented half of the sphere stays on-theme instead of wandering off into a different scene.

The interesting part is the light, not the picture. A panorama is just pixels; an HDRI has to know what actually emits. Naive exposure lifts every bright region at once and the whole frame glows (white flowers, a pale wall, an overcast sky), so the scene ends up lit by a daisy. An emitter gate (DINO + SegFormer) classifies the scene first and only lets real sources (sky, sun, windows, practicals) cross 1.0. It treats interiors and exteriors differently, and it deliberately doesn't blow out an overcast sky into a fake sun.

The output is more than one file. The stage writes a scene-linear EXR, an LDR preview, and a six-face cube map with a PolyHaven-style cross, in both EXR and Radiance .hdr, so it drops into UE5 or GIMP as easily as Nuke or Houdini. An optional first step removes people: stills go through FLUX-Fill, moving clips through VOID clean-plate, so the light of the location stays but the crew doesn't.

The genuinely hard part was memory. FLUX-Fill wants ~34GB of RAM and won't share a GPU politely with a loaded ComfyUI, so the orchestration restarts ComfyUI lean right before the FLUX step and frees VRAM around it. A 128GB unified box eases that pressure, though the aarch64 model stack has to be built first. None of that is glamorous, and all of it is the gap between a pipeline that survives a real shot and a demo that only runs on the developer's machine.

The whole thing sits behind a modal in my VFX Tools front end (auto-caption the plate, adjust the prompt, submit) because an artist should get an IBL back, not a pipeline to operate.