‹ All posts
Jul 19, 2026
#comfyui#3d#controlnet#face

GNM Head Studio — a parametric face rig for ComfyUI, and why the clay head isn't the point

Google released GNM (the Generative aNthropometric Model) — a parametric 3D head: you feed it identity, expression, pose and translation vectors and it spits out a mesh (17,821 vertices / 35,324 triangles). I wanted it as a ComfyUI node so I could drive ControlNet with exact head geometry, and eventually as a browser tool.

Three identity vectors up top; one head orbited −30° / 0° / +35° below — GNM rendered via nvdiffrast, TensorFlow-free

Most of the work wasn't the model. It was getting it to live next to ComfyUI without detonating either one. These are the notes.

The wall: "GNM needs TensorFlow"

Every reference to running GNM points you at TensorFlow. The official demos, the community ComfyUI pack (rethink-studios/ComfyUI-GNM), the identity/expression samplers — all TF. My first attempt died on exactly this, and I wrote it off as "GNM render needs TensorFlow, and TF doesn't fit in ComfyUI."

That belief was wrong, but the pain behind it is real.

Why TensorFlow and ComfyUI don't want to share a venv

ComfyUI is a PyTorch app. TensorFlow and PyTorch are two of the heaviest, most opinionated packages in the ecosystem, and dropping TF into a working ComfyUI venv breaks things in ways that are miserable to debug:

So the instinct — pip install tensorflow into the ComfyUI venv — is exactly the thing that wrecks the ComfyUI venv.

The actual fix: don't install TensorFlow at all

The key realization: GNM's shape model is backend-agnostic. The official github.com/google/GNM (package gnm.shape) is built on etils.enp and ships NumPy, JAX, PyTorch and TF subclasses. gnm_numpy and gnm_pytorch import zero TensorFlow. The model loads from gnm_head.npz (plain numpy arrays), not from the .h5 Keras files.

Which means the whole mesh pipeline — identity/expression blendshapes, the joint rig, linear-blend skinning — is just linear algebra. It runs in numpy/torch and pulls no protobuf, so it coexists with the main ComfyUI install untouched.

Install trick

# install the shape package WITHOUT its dependency list (which drags in TF)
pip install -e ./GNM-official/gnm/shape --no-deps
# then hand-install the base deps MINUS tensorflow
#   numpy>=2 is REQUIRED (np.linalg.cross), plus etils, scipy, etc.

--no-deps is the whole game: it stops pip from resolving GNM's requirements.txt, which is what would otherwise pull TF back in.

Three tiny source patches to import TF-free

A couple of modules still import tensorflow at the top even though the render path never uses it. Three small edits make the import clean:

  1. visualization/camera_conversions.py — wrap import tensorflow as tf in a try/except with a local _TFStub. Only the unused *_tf helpers need it.
  2. visualization/gnm_pyrender.py — PYOPENGL_PLATFORM = 'osmesa' → setdefault(..., 'egl') (GPU headless).
  3. visualization/render_gnm.py — epath.resource_path(_pkg).parent/... → epath.resource_path('gnm.shape')/'data'/'textures' (a MultiplexedPath has no .parent).

The renderer detour: pyrender → nvdiffrast

GNM bundles a pyrender path. It fights numpy 2.x: PyOpenGL's glGenTextures throws "No array-type handler" under numpy ≥ 2, and since both GNM and ComfyUI need numpy 2, downgrading isn't an option. pyrender is simply the wrong renderer here.

I switched the node to nvdiffrast (torch/CUDA, numpy-agnostic — the same rasterizer ComfyUI-3D-Pack uses). Headless RasterizeCudaContext, feed it GNM's verts/tris/UVs, do look-at + perspective → clip space, interpolate normals, clay shade, dr.antialias, flip NDC y. Six views render in ~0.7s.

pip install --no-build-isolation -e ./nvdiffrast-src

GNM depth and normal maps driving a depth ControlNet to a photoreal face

Isolation: keep the main ComfyUI pristine

Even with the TF-free path, I didn't want GNM's deps mingling with ComfyUI's. So the node runs GNM in a separate venv (venv-tf-free, torch 2.6.0+cu124) and the ComfyUI node bridges to it by subprocess. The main ComfyUI .venv never changes — protobuf 5.29.6 stays exactly where it was.

To make it fast, a small warm server (stdlib HTTP) keeps the model + nvdiffrast context resident and answers render requests over a socket: cold start ~3.7s, warm ~0.15s/render. With ComfyUI's Auto Queue on, dragging a slider updates near-realtime.

Machine-specific paths are all env vars (GNM_VENV_PY, GNM_MODELS_DIR, GNM_COMMUNITY_PACK, GNM_PORT) so nothing hardcoded ships in the repo.

Net result: the GNM node adds a whole 3D head model to ComfyUI and the base install is byte-for-byte unaffected.

The one place TF is genuinely required — and how to quarantine it

There is one part of GNM that needs TensorFlow: the semantic samplers. The "give me a HAPPY expression" / "give me an Asian female identity" generators are Keras VAE decoders shipped as .h5 files (identity_decoder_model.h5, expression_decoder_model.h5). Those genuinely need tf.keras to load.

My rule: never at ComfyUI runtime. I run the samplers offline, once, in a throwaway venv to bake their output into data, then throw the venv away. The ComfyUI node and the browser app never touch TF.

Even offline, the decoders had their own gotchas:

Recipe that actually loads them:

python3 -m venv /tmp/gnm-tf
/tmp/gnm-tf/bin/pip install "tensorflow-cpu==2.17.*" etils numpy scipy
# load WITHOUT TF_USE_LEGACY_KERAS -> native Keras 3
/tmp/gnm-tf/bin/python bake_samples.py   # writes vectors to .npy, then delete the venv

And the browser version: zero TF, zero server

The payoff of understanding that GNM's shape model is linear: identity, expression and shape are perfectly linear blendshapes (verified 0.00% error vs. the real model), so I precomputed them into a compact linear model the browser reconstructs as mesh = neutral + Σ delta. Pose/gaze run live as linear-blend skinning in JS. The whole thing runs client-side in WebGL — no Python, no TF, no backend. The samplers' output got baked into the identity/expression presets the same offline way.

The client-side GNM sculptor running in the browser on the visitor's own GPU

Takeaways

‹ Back to the devlog