Create a finished, studio-quality code-drawn animated video with a synced procedural soundtrack from reference images or videos and a rough topic or story idea. Trigger whenever the user provides visual references or asks for an animated explainer, motion graphic, animated story, reel/short, a video in a specific illustration style, or a code-drawn animation (like motion-studio-by-buck).
>-
# Motion Studio
Create studio-quality, hand-crafted 2D animations and explainer videos where every frame and the synced soundtrack are generated entirely in deterministic code (`piece.html`, `score.mjs`, and `render.mjs`), styled after user-provided visual references.
## When to Use
- The user wants an animated explainer video, motion graphic, short story, or social reel/short drawn in code rather than generated by a diffusion video model.
- The user uploads reference illustrations, paintings, botanical plates, or UI frames and asks for an animation in that visual style.
- The user mentions `motion-studio`, `motion-studio-by-buck`, or asks to generate code-drawn frames and procedural audio stitched into an MP4.
## Steps
### 1. Environment Check and Reference Study
1. Check what local tools are available if running in a terminal environment (`node`, `ffmpeg`, `ffprobe`, and `playwright` or `puppeteer`). If running in a browser or chat-only workspace without shell execution, generate the complete, self-contained project files (`piece.html`, `score.mjs`, `render.mjs`, and `build.sh`) so the user can preview the animation directly or render the MP4 locally.
2. Analyze every provided reference image or video to extract its visual DNA without copying literal subjects, poses, or signatures:
- **Color Palette**: Exact hex codes for paper/background tones, primary pigments, accents, and ink lines.
- **Surface and Texture**: Paper grain, watercolor edge bleed, granulated washes, stippling, canvas weave, or vintage plate borders.
- **Line and Shape Language**: Organic wobble, brush taper, geometric construction guides, or cut-paper layering.
- **Typography**: Serif vs. sans-serif hierarchy, italicized Latin/editorial labels, numeral styling, and tracking.
- **Motion Pacing**: Calm editorial reveals, playful spring physics, or continuous organic flow.
### 2. Write the Style Bible and Beat Sheet
1. Research the user's topic to identify one memorable, accurate visual metaphor and a concise factual takeaway.
2. Structure the timeline (default 15 seconds at 30 fps, 1080x1920 for 9:16 or 1920x1080 for 16:9) around four narrative phases: **The Hook** (0-3s), **The Build** (3-9s), **The Breath** (a quiet pause around 9-12s), and **The Payoff** (12-15s).
3. Draft a synchronized **Beat Sheet** mapping exact time ranges (`Time`), visual choreography (`Picture`), and audio events (`Sound`).
### 3. Build the Code-Drawn Animation (`piece.html`)
Create a self-contained HTML5 document (`piece.html`) using HTML5 Canvas and/or SVG that renders any timestamp `t` deterministically:
1. **Deterministic Time and PRNG**:
- Never rely on unseeded `Math.random()` or wall-clock delta time. Use a seeded PRNG (such as `mulberry32`) so every frame is 100% repeatable.
- Expose `window.seekTo(t)` (where `t` is time in seconds) and check `new URLSearchParams(location.search).get('t')`. When `t` is not provided in the URL, play interactively in the browser with `requestAnimationFrame` and Web Audio preview so the user can watch it live.
2. **Hand-Crafted Procedural Aesthetics**:
- **Paper and Texture Layer**: Pre-render a procedural paper texture canvas with subtle noise grain, warm undertone, and soft vignette so the background never looks like flat digital white or cold grey.
- **Watercolor and Ink Techniques**: Simulate watercolor washes by stacking 3 to 5 low-opacity deformed polygons with darkened pigment edges. Simulate hand-drawn ink lines by subdividing paths and offsetting vertices with coherent 2D sine/simplex noise.
- **Choreography and Easing**: Use custom cubic-bezier and damped spring functions. Stagger secondary motion (like petals opening, ripples expanding, or nerve signals traveling) and keep all essential text inside safe margins (at least 10% inset from edges).
### 4. Build the Frame Renderer (`render.mjs`)
Create a Node.js script (`render.mjs`) using Playwright or Puppeteer to render frames from `piece.html`:
1. Support a `--preview` flag that renders 6 to 8 key still frames across the timeline (e.g., `t = 0.5, 2.0, 4.5, 7.5, 10.5, 12.0, 14.0`) as PNGs for rapid visual inspection.
2. In full render mode, step frame-by-frame (`frame = 0` to `duration * fps - 1`), call `window.seekTo(frame / fps)`, capture the viewport screenshot, and pipe the PNG buffers into `ffmpeg` (`-f image2pipe -framerate 30 -i -`) to produce the video stream.
### 5. Visual Self-QA Loop
When shell execution is available, run `node render.mjs --preview` and inspect the rendered PNG stills before generating audio:
- Verify that focal subjects stay centered and do not collide with or obscure typography when limbs/elements flare outward.
- Check that background tones feel warm and textured rather than flat grey.
- Look for array-index wrap artifacts, stray construction outlines, or clipped labels near the viewport edges, and fix them in `piece.html`.
### 6. Synthesize the Synced Soundtrack (`score.mjs`)
Create a dependency-free Node.js script (`score.mjs`) that writes a 48000 Hz, 16-bit stereo WAV file (`score.wav`) locked to the exact timestamps in the Beat Sheet:
1. Build modular synthesis helpers in pure JavaScript:
- **Plucks and Marimbas**: Sine/triangle oscillators with fast attack, exponential decay, and harmonic overtones.
- **Warm Pads and Chords**: Detuned sine/sawtooth voices passed through a gentle one-pole low-pass filter with slow attack and release envelopes.
- **Organic Foley**: Pitch-swept sine droplets for water drips, short filtered noise bursts for ticks/suckers/brushstrokes, and resonant bell harmonics for focal reveals.
2. Shape the master amplitude curve to mirror the narrative arc: gentle intro, building rhythmic momentum, a noticeable drop to near-silence during **The Breath**, and a warm, full resolution chord on **The Payoff**.
3. Check integrated loudness with `ffmpeg -i score.wav -af ebur128 -f null -` and normalize so the final mix sits around `-16 LUFS` without clipping.
### 7. Stitch Final MP4 and Present Summary
1. Merge the rendered frames and `score.wav` using `ffmpeg`:
`ffmpeg -y -i video_only.mp4 -i score.wav -c:v libx264 -pix_fmt yuv420p -crf 18 -c:a aac -b:a 192k -shortest output.mp4`
2. Present the final response in this clear structure:
- **Concept & Reference Breakdown**: Explain the visual metaphor and how each reference's palette, texture, or pacing was adapted without copying literal poses.
- **Beats Table**: A Markdown table with `Time`, `Picture`, and `Sound` columns matching the animation.
- **One Fact to Know**: A brief accuracy note explaining the science or history behind the visual metaphor.
- **Easy Tweaks**: A numbered list of 4 concrete adjustments the user can request next (timing, typography, secondary details, or soundtrack mix).
- **Project Files Summary**: List the generated files (`piece.html`, `score.mjs`, `render.mjs`) and output video details.
## Gotchas
- **Flat Digital Look**: Standard Canvas primitives (`ctx.arc`, `ctx.fillRect`) look robotic on their own. Always layer procedural paper grain, subtle vertex jitter, and pigment edge darkening to achieve an illustrated feel.
- **Non-Deterministic Motion**: Using `Math.random()` inside the draw loop causes flickering between frames. Always seed a deterministic generator or compute positions as pure functions of `t` and element index.
- **Flat Audio Dynamics**: Avoid playing a constant background loop at uniform volume. Ensure the soundtrack drops in volume during the pause beat so the final payoff feels earned.
- **Text Collisions**: Animated arms, stems, or particles often swing into title areas at peak extension. Clamp motion bounds away from the top and bottom text safe zones.