Plate to HDRI: a real 360° IBL from one photo, with an emitter gate that knows what glows

A single plate becomes a scene-linear 360° HDRI you can light a shot with — that's the whole job of the env-to-hdri stage in my VFX pipeline. This is the real version of the thing I relabeled a few days ago: the drag-in ComfyUI graph fakes a latlong and leaves a faint seam, this one builds the actual sphere and heals it. I wrote the whole method up publicly, failures and all — comfyui-examples/10-env-to-hdri.

The pipeline is four models in a trench coat. HunyuanWorld-1.0 does the true-360 extrapolation — one plate in, a 1920×960 equirectangular panorama out — running as a subprocess in its own venv (torch 2.5 / diffusers 0.34) so its FLUX-Fill dependency never collides with the rest of the stack. A ComfyUI pass then wrap-pads and upscales it 2× (4x-UltraSharp + TiledDiffusion, cropped back), and a second ComfyUI pass heals the wrap seam by rolling the pano 50%, inpainting the centre strip, and rolling it back. The result lands at 3840×1920. A BLIP caption of the plate steers the extrapolation when no prompt is given, so the invented half of the sphere stays on-theme instead of wandering off into a different scene.

The interesting part is the light, not the picture. A panorama is just pixels; an HDRI has to know what actually emits. Naive exposure lifts every bright region at once and the whole frame glows — white flowers, a pale wall, an overcast sky — so the scene ends up lit by a daisy. An emitter gate (DINO + SegFormer) classifies the scene first and only lets real sources — sky, sun, windows, practicals — cross 1.0. It treats interiors and exteriors differently, and it deliberately doesn't blow out an overcast sky into a fake sun.

The output is more than one file. The stage writes a scene-linear EXR, an LDR preview, and a six-face cube map with a PolyHaven-style cross — in both EXR and Radiance .hdr, so it drops into UE5 or GIMP as easily as Nuke or Houdini. An optional first step removes people: stills go through FLUX-Fill, moving clips through VOID clean-plate, so the light of the location stays but the crew doesn't.

The genuinely hard part was memory. FLUX-Fill wants ~34GB of RAM and won't share a GPU politely with a loaded ComfyUI, so the orchestration restarts ComfyUI lean right before the FLUX step and frees VRAM around it. A 128GB unified box eases that pressure, though the aarch64 model stack has to be built first. None of that is glamorous, and all of it is the gap between a pipeline that survives a real shot and a demo that only runs on the developer's machine.

The whole thing sits behind a modal in my VFX Tools front end — auto-caption the plate, adjust the prompt, submit — because an artist should get an IBL back, not a pipeline to operate.