From Hollywood VFX to Generative AI

Pipeline Capabilities

• VFX Infra Backbone: The frontend relies on the VFX Infra platform — alienrobot-server dispatches jobs over Tailscale to custom Python servers running the AI models on pinned GPUs, no ComfyUI in the loop
• Drag-&-Drop, Not ComfyUI: Artists upload a plate, pick an operation, and get results — no node graph to open, no workflow to debug
• Cleanplates: Netflix Void for background reconstruction behind masked subjects
• Keys & Mattes: Corridor Key, SAM3, and MatAnyone for chroma-free keying and temporally consistent video mattes
• Depth & 3D Body: Depth Anything depth maps, GVHMR SMPL mesh recovery, DWPose 133-point skeletons
• Plate → 360° HDRI: A multi-model pipeline — HunyuanWorld (FLUX) extrapolates a single plate to a true equirectangular 360° panorama (auto-captioned so it stays on-theme), a ComfyUI pass wrap-pads and seam-heals the upscale, and a DINO + SegFormer emitter gate classifies the scene so only real light sources exceed 1.0 — out to scene-linear EXR and cube-map faces for image-based lighting
• Tracker Module: Dedicated 3D person-tracking tab — locks GVHMR meshes and DWPose skeletons to the plate for matchmove, crowd replacement, and CG double work
• Studio Output: FBX / BVH / OBJ / EXR exports drop straight into Maya, Houdini, Blender, or Nuke

How It Works

• Look Before Done: get_image returns the rendered output to the model, and every tool response pushes it to judge the pixels — a green run is reported as valid, not correct
• The Ratchet: keep a change only if it beats the best-so-far, otherwise revert and try something else — held on disk, so it survives context compaction (adapted from Karpathy's AutoResearch)
• Discovery, Not Guessing: nodes, models, and ~550 workflow templates come from the live /object_info API and the open template catalog — never from training data
• Installs What's Missing: find_missing_nodes, install_node_pack, and install_model drive ComfyUI-Manager on your host, then restart it
• Token-Efficient by Necessity: compact node notation cuts object_info by 93% (measured across 987 nodes) — discovery gets re-paid on every pass of the loop
• Yours, Not a Service: points at hardware you already own — local, private, offline-capable, no account, ~1,200 lines of hackable MIT Python

Instant 3D Reconstruction

• Apple Research Model: Built on the SHARP monocular view synthesis architecture with DINOv2 backbone
• Real-Time Rendering: Custom point cloud shader with interactive rotation, zoom, and orbit controls
• Sub-Second Inference: Generate complete 3D scenes from any photograph in under 1 second on GPU
• Production-Ready Export: Outputs standard PLY files compatible with Gaussian Splatting renderers

What's Inside

• Natural-Language Meals: Type "big chicken salad with hard-boiled eggs and blue cheese" — or snap a plate, a nutrition label, or packaging — and the app parses it into structured calories and macros, asking clarifying questions if anything's ambiguous
• Plan Before You Eat: Preview the macro impact of a meal against today's calorie and protein budget before you log it — then log as eaten, or save as a repeatable template
• AI Coach: An in-app chat that reads today's snapshot plus a rolling 7-day window of your meals, glucose, weight, and labs. It reasons over your actual data with specific numbers, not generic advice
• Auto-Sync From Your Devices: Watch (sleep stages, heart rate, HRV, SpO2, blood pressure), continuous glucose monitor, and smart scale all stream in on their own — no manual entry
• Sleep + Glucose Overlay: Overnight glucose curve laid over REM / deep / light sleep stages so you can see which dips correlate with which phase
• Lab Parsing: Upload a lab report as a PDF or a photo — ApoB, LDL, A1c, Lp(a), PSA and more get extracted into structured rows for review
• Stock-Chart Metrics: 1 week, 1 month, 3 months, 1 year, 3 years, or all-time — a 3-year view stays readable thanks to adaptive bucketing
• Coach Accounts: Generate a scoped invite code so a clinician, wellness coach, or partner can read your data and chat alongside you. Revocable any time

What's Inside

• The Formatting Is the Product: Every chapter is hand-set by Claude Opus — one chapter, one run, no batch transforms — because where a line breaks and which word carries the weight is editorial judgment a regex can't make
• Breathable Reading: Strategic line breaks at phrase boundaries, generous whitespace, lead-in openers, drop-caps, smallcaps, and italics for interiority — a tall, narrow canvas instead of a wall of text
• Not One Word Added: The prose is identical to the source; only the presentation changes — no rewriting, no summarizing, no AI-written text
• Built for the Scroll Generation: Aimed at lapsed readers who scroll for hours but "can't focus on books" — meet them in the flow state they already live in
• A Curated Shelf: Frankenstein, Moby-Dick, Pride and Prejudice, Alice in Wonderland, Jekyll & Hyde, Romeo and Juliet, The Count of Monte Cristo — plus three translations of the Odyssey in progress
• Curator Pipeline: A node-tree content schema and a five-pass Claude Code curation workflow PUT finished chapters to the backend — typesetting a long novel this way costs real tokens, and that's the point

Platform Design

• Shared vfx-base Image: CUDA + PyTorch + Diffusers + common ML libs — every service FROMs a single, cached base layer for fast iteration
• Service Overlays: Each GPU worker ships its own Dockerfile in its own repo; this infra repo composes them in at runtime by tag
• Tailscale Jump Host: All SSH and Redis traffic tunnels through a Tailnet-only jump node. No ports are exposed to the public internet — ever
• Per-Service GPU Assignment: Each compose service reserves specific GPUs via deploy.resources.reservations.devices so tenants never contend for VRAM
• Shared Named Volumes: vfx-hf-cache, vfx-torch-hub, vfx-models — model weights downloaded once, reused by every service
• Supabase-Native: Workers write results (renders, mattes, splats, metadata) directly to Supabase storage + Postgres — no blob-handoff dance
• vfx-net Docker Network: Services talk internally over an isolated bridge — Redis, Postgres, and workers never touch host networking
• Cold-to-Hot in Minutes: docker compose up and a new GPU worker is online, monitored, and pulling jobs from the same queue as every other service

What's Inside

• Vibe-to-Pairing: Type the mood, audience, or topic and get a set of header/body font pairings — each with multiple options — drawn from real Google Fonts, not placeholder names
• Live Sample Layouts: Every pairing renders on actual headings, body copy, accents, and labels so you judge it in context, not on a single specimen line
• Refine in Plain English: Nudge a result — "make the body a serif, a touch more retro" — and it regenerates without starting over
• Use-Case Presets: Ready-made starting points for portfolios, news portals, weddings, fintech, SaaS dashboards, dev-tool docs, and more
• Type-System Playground: Carry a pairing into a full prototype — sizes, weights, tracking, leading — and tune the whole scale together
• Ship It Three Ways: Copy production CSS, export a spec to hand an LLM, download the fonts as a zip, or share the pairing by link

What's Inside

• One Stable Room Link: Create a room once and reuse the same URL all session — and again next class. Rooms live 30 days from the last push or vote
• Three Ways to Join: Paste the URL in chat, show a QR code, or read out a 4-letter code (A–Z minus I/L/O so it's never mistaken for 1 or 0)
• Push-to-Swap: Fill the form, push, and every student's screen updates within ~2 seconds — the previous poll auto-closes into history
• Live Results, Your Call: Watch counts come in live; toggle whether students see the tally or just cast their vote
• Revisit Past Polls: Every question pushed to the room is kept — click "make current" to re-run any of them
• One-Off Mode: Need a single quick poll with no room? Spin one up; it lives 24 hours and needs no stable URL

Why MultiPass

• 100% Client-Side: FFmpeg.wasm + ONNX runtime — no uploads, no cloud cost
• Three Passes, One Input: Get matte, depth, and normals from a single video
• Compositor-Ready: Outputs drop straight into Nuke, After Effects, or DaVinci Resolve
• Private by Design: Your footage never leaves your machine

Architecture

• React + Vite Frontend: Deployed to GitHub Pages, built for speed
• LLM Intent Layer: Parses the prompt, then picks retrieval, generation, or parametric design per tile
• Parametric Tile Engine: Procedural shape, gradient, and pattern synthesis for clean geometric sheets
• Client-Side PSD Build: Sheet is sliced on a canvas and assembled with ag-psd
• R2 Storage: S3-compatible presigned URLs for downloaded sheets

How It Works

• Tickets, Not Patches: Players describe what they want — new commands, rooms, mechanics, UI — and file a ticket from inside the app itself
• Agent-Driven Commits: LLM coding agents pull tickets, implement the change across frontend / backend / schema, and push — the app literally evolves itself
• Deterministic Local State: React / Vite shell with a deterministic world-state engine; DeepInfra-authored narration layered on top via the backend
• Full Stack, Already Wired: Frontend (GitHub Pages), backend service, database, and storage — every piece most apps need, ready for agents to extend
• Any-App Potential: Because the scaffolding is general, the same ticket loop could grow Hello World into a journaling app, a collab tool, a CMS — whatever the community asks for next

Built For Artists

• React Three Fiber: Real WebGL previews, not static thumbnails
• Metadata-Driven: Triangle counts, UVs, rig state, license, and more
• Fast Filtering: Slice the library by category, format, or attribute
• One-Click Download: Grab the asset, keep working

System Highlights

• Switchable Models: US/Western LLMs only — Llama 3.2, Llama 3.1, Mistral 7B, Phi-3 Mini, GPU-accelerated on Modal T4
• Vector Memory: ChromaDB + Sentence-Transformers with 23+ pre-seeded tactical patterns
• ARTAK Integration: Builds full MissionPlanning documents — assets, routes, waypoints — straight into MongoDB
• GIS Tooling: Interactive asset locator and coordinate picker with OSM + satellite tiles

Design Pillars

• Clean UI: Radix UI + Tailwind, no clutter, no dark patterns
• Interactive Maps: Leaflet-powered weather map with location search
• AI Narrative: Plain-language forecasts written by an LLM, not a robot template
• Offline-Ready: PWA manifest + service worker for a real app experience

Complete Generative AI Production Pipeline

• Real Production Pipeline: Automatic hierarchy, naming, versioning, and cloud sync. Projects → Sequences → Shots → Assets — built like a VFX or animation studio
• Storyboarding & Previs: Build full sequences and timelines with AI-generated imagery. Visualize your story before production begins
• Character & Asset Design: Manage character designs, variants, and creative iterations seamlessly. Maintain consistency across your entire production
• Generative AI Integration: Run top AI models in parallel for cinematic-quality concept, art, or shot generation. Compare results and iterate instantly
• Team Collaboration: Every prompt, ref, and generation stays organized and shareable. No more messy folders or untraceable prompts
• Transparent Spending: Built-in budget visibility for every project. View generation costs as they occur, monitor team-wide spending

Better Than Chat Exports

• Professional Themes: 14+ typography themes including technical, legal, academic, and resume styles
• Smart Icon System: Automatically converts emojis to crisp SVG icons (Lucide, Heroicons, etc.) for professional use
• Visual Control: Split editor with live preview and visible page guides to perfect your page breaks
• Private by Design: Totally free and private. Everything runs locally in your browser with no uploads

@APP[bg:gray-900]
  @SB[w:240 bg:gray-800]
    §PROJECTS
      +Project01
      -Project02
      -Project03
    §SEQUENCES
      +seq01
      -seq02
    §AI TOOLS
      -⚡Prompt
      -🎨Style
      -✓Consistency
  @MAIN
    @NAV[h:30 bg:gray-800]
      Project01 | seq01
      @TABS [Shots][Characters][References]
      @STATS "3 Seq | 30 Shots | 12 Done"
    @TIMELINE
      @CARD[w:40] "seq01_sh001"[blue] "COMPLETE"[green]
      @CARD[w:40] "seq01_sh002"[blue] "IN PROGRESS"[yellow]
    @PREVIEW
      @VIDEO[aspect:16/9]
      @CONTROLS ⏮️ ▶️ ⏭️ 🔊

@COMPONENT[Dialog]
  @HEADER "Confirm Action"
  @BODY
    "Are you sure you want to proceed with this action?"
  @FOOTER
    [Button secondary]Cancel[/Button]
    [Button primary]Confirm[/Button]
@COMPONENT

Web Audio Synthesis Engine

• Multi-layered Oscillators: 3 oscillators per voice (2 sawtooth + sub) for thick, rich sound
• Moog Model D Emulation: Authentic 4-pole ladder filter for classic analog warmth
• Effects Chain: Master filter → Delay → Reverb with real-time control
• Data Integration: Connect real-time data sources (weather, stocks, time) to control synth parameters

Supported Icon Libraries

• Lucide Icons: Modern, clean design for web applications
• Heroicons: Tailwind CSS optimized icon set
• Material Icons: Google's Material Design family
• And More: Font Awesome, Feather, Phosphor, Tabler, Boxicons, Remix

What's Inside

• Audio From Anything: Pull the track from a standalone audio file (MP3 / WAV / AAC / M4A) or extract it from a second video
• Fast Path: The video is stream-copied without re-encoding when the container is compatible — near-instant export
• Automatic Fallback: Incompatible sources (e.g. WebM / VP9) transparently fall back to an H.264 re-encode
• Private by Design: Runs entirely on-device via WebAssembly ffmpeg — files never leave the browser, even on a static host

What's Inside

• Render Modes: ASCII, Pop Art, Verse, Anime, and Dither — with tunable ASCII character size, density levels, and a pick of mono fonts (JetBrains Mono, VT323, Press Start 2P, DotGothic16…)
• Color Treatments: Monochrome, green and amber terminal palettes, or full color
• Smart Layers: Optional segmentation plus temporal-consistency controls to keep the effect stable frame-to-frame
• Effects Stack: Scanlines, vignette, chroma shift, film grain, glitch, CRT, invert, pixel sort, slit scan, feedback trails, kaleidoscope, halftone, posterize, and bloom
• In-Browser Export: 1080p or 4K output with audio preserved — nothing uploaded