post
Puppet scanning for game-engine integration
Puppet
Definition
Puppet scanning for game-engine integration is the practice of digitising a physical stop-motion puppet or set piece -- typically by 3D scanning or photogrammetry, after it has already been built and shot in the ordinary way -- so its geometry and hand-made surface can be rebuilt as a manipulable asset inside a real-time game engine, rather than existing only as frames of photographed motion. It is a further step past ordinary stop-motion fabrication (see [[puppet]]): the same object a crew built and posed for the camera is captured a second time, digitally, so a player -- not a camera operator -- can view and interact with it from angles the original shoot never had to solve.
Overview
The only documented case so far is Out of Words (WiredFly and Kong Orange, Denmark; published by Epic Games Publishing), a narrative co-op game whose cinematic sequences are conventional stop motion: puppets and sets built from paper, cellophane and other everyday materials (see [[puppet]] for the production's fabrication and durability details), shot frame by frame in front of a real camera and lens. Puppet fabricator Sofie Kjaer told VICE that once a puppet has served that purpose, "all physical puppets are scanned, with textures translated through scanning processes into the final game assets" -- the object built and animated for the cinematics becomes, via a second capture step, the model players see and move through in the interactive parts of the game.
Neither VICE's interview nor Xbox Wire's making-of feature on the same production names the specific scanning method (photogrammetry rig, structured-light scanner, or otherwise) or the software used to retopologise the resulting mesh and bake the puppet's own texture onto it. That gap matters: the claim here is narrower than "stop-motion games" in general -- many games use stop-motion-styled sprite animation without ever building a physical object -- and it is a different destination than the photogrammetry scanning already used inside film VFX (see [[after-effects-compositing]]) to build a CG stand-in for one effects shot. Here the digital twin is not a stand-in for a shot the camera could not get; it is the interactive object itself, while the pre-rendered cinematics the audience also sees stay conventional in-camera stop motion.
That combination -- practical fabrication kept intact through the whole cinematic pipeline, with a digitisation step added only where the object needs to become playable -- is a plausible answer to a problem CG-only productions do not have but pure stop-motion productions never needed to solve either: keeping a handmade object's actual surface, rather than a repainted digital approximation, visible in a context the camera never controls. Whether it becomes a repeatable studio practice or stays one project's bespoke solution is not answerable from a single production; a second sourced example, or a technical postmortem naming the scanning method, would settle it.
Workflow
- 1. Build the puppet or set piece to full production standard, exactly as for camera-based stop motion (see [[puppet]] for material and durability practice).
- 2. Shoot the character's cinematic performance in the ordinary way, hand-posed frame by frame in front of a real camera and lens.
- 3. Separately, scan the same physical object's geometry and actual surface -- by photogrammetry or an equivalent 3D-scanning method -- rather than modelling and repainting a CG approximation from reference photos.
- 4. Clean and retopologise the scan into a real-time-ready mesh, then bake the object's own captured texture onto it so the digital asset visually matches what the crew actually built.
- 5. Import the finished asset into the game engine as a player-facing interactive object, kept visually consistent with, but functionally separate from, the pre-rendered stop-motion cinematics.