Generating a Vertical Video Ad With AI: Keyframe Anchoring
Generating a vertical video ad with AI usually fails in one of two ways: the product drifts into something the store doesn't actually sell, or the camera refuses to stay put and the whole scene hallucinates around it. Text-to-video hands a single model both jobs at once — inventing what is on screen and moving it — and it is reliably bad at the first. This case study walks through the technique that fixes both, keyframe anchoring: let an image model own every frame's identity, then give a first-last-frame video model nothing to do but interpolate the motion between two frames you already approved. The deliverable is a real 15-second, 9:16 paid-social spot for Roomix, a home redesign iOS app, produced end to end on ModelRunner using only Google models.
The finished 15-second vertical video ad — 1080×1920, generated entirely from Google models on ModelRunner.
The Project: A Video Ad for a Home Redesign App
Roomix runs its core redesign feature on ModelRunner, and its next marketing milestone was paid social — a vertical video ad. Seven earlier ad variants (A–G) already existed, but every one of them was a composite: static AI stills animated in a browser and stitched with ffmpeg. They look clean, but nothing in them actually moves — the motion is CSS. Variant H is the first ad where the motion itself is generated, produced under a hard brief: ModelRunner only, Google models only.
The Problem: Product Fidelity and a Locked Camera
The creative concept was an AR-style shopping demo. A frosted-glass e-store panel slides up over a real living room; a finger taps three real, purchasable products; the app "materializes" them into the space; a swipe carousel flips the sofa through several styles; a drag-and-drop finale finishes the room. The raw material was one ordinary photo — a family living room with a brown corner sofa in evening light — plus a catalogue of couches and products shot flat on seamless backgrounds.
That premise is hostile to a video model in two specific ways:
- Product identity must survive. The emerald Chesterfield in the ad has to be the Chesterfield from the catalogue, not a green sofa the model invented. An ad that shows furniture the store doesn't sell is worse than no ad.
- The room must not move. The whole gag is that the UI and furniture animate while the room stays nailed down. Any camera drift and the composite reads as a hallucination rather than a product demo.
Ask a text-to-video model for "a sofa swaps for another sofa" and you get a dissolve between two sofas it made up. The earlier variants dodged this by refusing generated motion altogether — animating stills in a browser timeline captured with Puppeteer. The bet here was that the job could instead be decomposed so each model does only what it is good at.
Why ModelRunner: The Whole Google Stack on One Endpoint
1. It's already in the stack. Roomix calls ModelRunner for its core redesign feature, and its App Store screenshots and 15-language localization run both went through it. Same API key, same MCP tools, same request/poll pattern — no new service to onboard.
2. The whole Google stack is on one endpoint. The constraint was "Google models only," and ModelRunner carries the entire chain the ad needs — google/nano-banana-2/edit for stills, google/veo-3.1/* for motion, google/lyria2 for music. One catalogue call (list_models search=google) surfaced them all. No separate Vertex AI project, no three sets of credentials.
3. Veo 3.1 exposes the endpoint that makes this tractable. Of the Veo variants, first-last-frame-to-video is the one that matters: it interpolates between a pinned start frame and a pinned end frame. That single capability is what turns "hope the model draws the right sofa" into "the model may only draw the path between two frames I already control."
4. MCP made it native tool calls. Claude Code drove the run through run_model, wait_for_request / get_request, and create_upload_url. Graded keyframes were streamed to storage out-of-band via presigned PUT (curl -T), so the image bytes never passed through the model as tokens.
Keyframe Anchoring: The Image Model Owns Identity, the Video Model Owns Motion
This is the load-bearing decision, and it generalizes past ads to any place you need a video model to stay on-brand. Generate every keyframe first with an image-editing model, chaining each Nano Banana edit off the previous keyframe rather than the original photo. Then let Veo do nothing but interpolate between two pinned endpoints.
Phase 1 — Keyframes (serial, 8 frames) google/nano-banana-2/edit
source photo → K00 plate (16:9 → 9:16 reframe)
K00 + product refs → K01 (storefront UI)
K01 → K02 (three cards selected) ← each edit takes the PREVIOUS keyframe as images[0]
K00 + product refs → K04 (chesterfield materialized)
K04 → K05 (bouclé) → K06 (japandi) → K07b (cognac + dock) → K08 (finished room)
Phase 2 — Colour normalise + upload (local + presigned PUT)
per-channel gain → exact 1080×1920 → create_upload_url → curl -T → file_url
Phase 3 — Motion (parallel ×7) google/veo-3.1/first-last-frame-to-video
(Kn, Kn+1) + prompt → 8s clip @1080p 9:16, with native SFX
Phase 4 — Music (×1) google/lyria2
Phase 5 — Assembly (local ffmpeg)
trim to the action window → retime to the beat → concat → CTA type → mix → loudnorm
This splits the work along the seam where the models' abilities diverge: Veo handles motion well and object identity badly; Nano Banana is the reverse. Because K04 (Chesterfield) and K05 (bouclé) are the identical room differing only in the sofa, Veo's job on that clip is not "invent a bouclé sofa" but "move this rigid object out while that one moves in." That is why the swipe reads as one solid object sliding past another instead of a crossfade.
The chaining is not optional. Renders align with each other, not with the source — so each edit generated independently from the original photo would drift a few pixels, and the "locked room" illusion collapses. Chaining each frame off the last keeps them registered.
The Chain Leaks Colour — Fix It in Post, Not the Prompt
Chaining buys alignment and costs colour. Measured on a fixed patch of floorboard across the chain:
| Keyframe | R | G | B | R−B |
|---|---|---|---|---|
| K00 plate | 129 | 102 | 81 | +48 |
| K04 chesterfield | 136 | 106 | 89 | +47 |
| K06 japandi | 137 | 101 | 86 | +51 |
| K07b cognac | 143 | 101 | 86 | +57 |
| K08 final | 143 | 94 | 78 | +65 |
The green channel bleeds out edit by edit while red climbs, so the tail frames go magenta. Each step is only ~2 units and invisible inside a clip, but the accumulated shift makes the hero beauty frame the ugliest in the film. And Nano Banana ignores white-balance instructions in the prompt — explicitly telling it "preserve the exact white balance, do not push toward pink or magenta" changed nothing (green still landed at 93). The fix is mechanical, not generative: measure the same physical surface in each keyframe, compute a per-channel gain back to the reference frame (K04's 136,106,89), and apply it before upload. Repeated edits also deposit chroma mottling, cleaned with hqdn3d at luma_spatial=0 and chroma strength scaled to how many passes each frame accumulated.
A Fixed-Duration Endpoint Is an Accounting Fact ($3.20/clip)
first-last-frame-to-video always returns exactly 8 seconds, billed at $0.40/output-second = $3.20/clip, with no duration parameter. A 1.3-second sofa swipe therefore costs the same $3.20 as an 8-second one. So the discipline is: prompt for slow, deliberate motion, then retime to the beat locally. Rather than guess where the action sits, each clip was profiled with a frame-delta energy scan:
== C7 (mean abs frame delta per 0.5s)
t= 0.0s 0.06
t= 1.5s 1.51 ####
t= 2.5s 4.73 ############## ← the drag-and-drop
t= 5.0s 1.61 ####
t= 6.5s 0.12 ← dead
Two clips pause mid-shot — the glass panel arrives, waits, then the cards populate — so their two live windows were spliced together and the dead beat dropped rather than retimed through. A build script and manifest record every trim window, retime factor, and request ID, so recutting the timing costs nothing further on Veo. Nine clips bought fifteen seconds, and that arithmetic was knowable before the first render.
Three Traps That Cost Re-Renders
- Veo scales a "fingertip" to whatever it can touch. With on-screen UI in frame it renders a correctly UI-scaled finger — the tap and drag beats worked first time. In a bare wide shot with nothing to touch, it scales the hand to the room and sweeps a giant photorealistic arm across the frame. Two swipe clips were re-rendered (+$6.40) with no hand at all — the sofas ride an "invisible carousel rail," which is closer to the intended physical motion anyway. Negative prompt:
hand, finger, arm, skin, giant hand. - Never bake CTA type into a Veo endpoint. Asked to morph letters across 8 seconds, a video model garbles them. The final keyframe was regenerated clean and the type set in post with
drawtext. (Two ffmpeg quirks: useSFNS.ttfand fake bold withborderw=2:bordercolor=white, because the fontconfig patternfont='Helvetica Neue:style=Bold'collides with:, ffmpeg's own option separator.) - Veo drifts slightly from the frame it was pinned to. Cutting from the last clip back to the source keyframe popped — the coffee table jumped a few pixels. The end hold freezes the clip's own last frame instead. Verify the pin; don't assume it.
Audio: Native SFX + a Lyria Bed, Mixed to −14 LUFS
Veo emits native SFX per clip — glass taps, the shatter, fabric whooshes — which survive the retime via a pitch-preserving atempo chain and sit under the Lyria bed. Both land quiet: the raw mix measured −25.8 LUFS with a huge crest factor (the shatter peaks while the bed sits far below). loudnorm in linear=true mode can't fix that — it clamps gain against the true-peak ceiling and undershoots, landing at −15.8. Inserting an acompressor to lift the bed then normalising hits the −14 LUFS social target cleanly. The Lyria track itself is instrumental lo-fi house at 118 bpm.
Results: A 15-Second Google-Only Video Ad for $30.11
- A 15.000-second vertical spot — 1080×1920, 24 fps, 360 frames, −14.4 LUFS / −1.1 dBTP.
- Every generated pixel and every note is Google. Nano Banana 2 for the eight keyframes, Veo 3.1 for all motion and sound effects, Lyria 2 for the music. ffmpeg only cut, retimed, and mixed.
- Product fidelity held. The Chesterfield, the bouclé, the Japandi platform, the cognac mid-century sofa, the brass arc lamp, the walnut bookshelf, the terracotta Berber rug, the green velvet armchair and the round walnut table are all the supplied catalogue items, recognisable frame to frame.
- The room never moves. Bookshelf, coffee table, rug, lamp and window are pixel-stable across all four sofa states — which is the entire reason the swipe carousel reads as physical.
- Total cost ~$30.11 — keyframes $1.21 (12 charged Nano Banana runs at ~$0.101 each), Veo $28.80 (9 clips × $3.20; 7 kept, 2 re-rendered for the giant-hand bug), Lyria $0.10.
- Fully reproducible from the request IDs in the appendix below.
The Bigger Picture: Decomposing a Task Along the Model-Ability Seam
This is the latest step in one trajectory. ModelRunner first powered Roomix's core renders, then populated the app's test asset library with AI-generated images, then produced its App Store marketing screenshots, then localized those assets into 15 languages, and now generates its ad creative — motion and sound included. Every step reused the same API key and the same MCP tools; only the prompt, the model, and the destination changed.
The lesson specific to this run is about decomposing a task along the seam where model abilities diverge. The naïve framing — "generate a video of a sofa being swapped" — hands a video model a job it is bad at (object identity) bundled with one it is good at (motion), and you get a plausible dissolve of furniture nobody sells. Splitting it so an image-editing model owns what is in the frame and the video model owns only the path between two frames I already approved turns a hallucination risk into an interpolation problem. That generalises well past ads: any time you need a video model to stay on-brand, give it fewer degrees of freedom, not a better prompt.
FAQ
How do you keep a product looking consistent in an AI-generated video?
Use keyframe anchoring. Generate every frame first with an image-editing model — each frame chained off the previous one so they stay aligned — then hand a first-last-frame video model only the job of interpolating between two frames you already approved. It never invents the product, because it is never asked to draw the product.
What is first-last-frame-to-video?
It is a Veo 3.1 endpoint that generates motion by interpolating between a pinned start frame and a pinned end frame, unlike plain image-to-video, which has no end-frame control. On this endpoint it returns a fixed 8-second clip with no duration parameter.
How much does it cost to generate a short AI video ad?
This 15-second spot came to about $30.11: nine Veo clips at $3.20 each ($28.80), eight keyframes at roughly $0.10 each ($1.21), and one Lyria music track ($0.10). Because the video endpoint is fixed at 8 seconds per clip, the cost is knowable before the first render.
Why does chaining image edits shift the colour, and how do you fix it?
Each edit leaks a little of the green channel, so the tail frames drift magenta. Prompt-level white-balance instructions are ignored; the fix is mechanical — measure the same surface in every frame and apply a per-channel gain back to a reference frame before assembly.
Roomix is built with SwiftUI and SwiftData, targeting iOS 17.0+. The ad described here was authored and executed by Claude Code using the ModelRunner MCP server with google/nano-banana-2/edit, google/veo-3.1/first-last-frame-to-video, and google/lyria2.
Appendix: Request IDs and output URLs
Keyframes — google/nano-banana-2/edit (~$0.101/run)
Each row's edit takes the previous keyframe as images[0], plus the catalogue product shots as additional references.
| Keyframe | Request ID | Output URL |
|---|---|---|
K00_plate (16:9 → 9:16 reframe) | MaujBgUYZi1A13qsSEXQZ | https://media.modelrunner.ai/QOY4YfBPy9izm9gXhgtQN.png |
K01_ui (storefront panel) | 39x24wbdfra13E7Brv3JF | https://media.modelrunner.ai/nPGNSZOgx3Kztveb6tniF.png |
K02_selected (rings, cart 0→3, holograms) | 8ssdRIWDBBiTmdTpF7m6f | https://media.modelrunner.ai/ADKnnnNMgfgjn2PXTdsDU.png |
K04_chesterfield | 0JoNxVa2YwbAgaxn8ulKa | https://media.modelrunner.ai/uzXYLnDJB4Pde9PgG3Ixk.png |
K05_boucle | fcaMpCgPO8VjIkx9CcOIU | https://media.modelrunner.ai/KzPLtnlv6BFz329VVNd0S.png |
K06_japandi | tJbFerq1AtyDLTRIYQF1O | https://media.modelrunner.ai/EcbS3kj6eoTAUdG0AO7ho.png |
K07_cognac | fibYSt9J61lx5PuIhraso | https://media.modelrunner.ai/mQpo4F0FdPuYh4GtIJMML.png |
K07b_cognac_dock | FFX8n1q0ZGgfVbL2plNAO | https://media.modelrunner.ai/mPqeqwHDSpmS2VlS9cdN8.png |
K08_final (clean, no type) | PK9JNcLC4QOFoiimXlu38 | https://media.modelrunner.ai/7bp9wclh1sqcg1T5M9wab.png |
Discarded keyframes: IFq3h2ZuNOwRfShsqik6g (UI panel too opaque — sofa invisible behind the glass), kmFr1b2rIweOXGtJ9FFrn (grew a second bookshelf instead of replacing the first), m6hDPGT1b2YycdRQF8p5t (CTA baked in — pulled so Veo couldn't morph the letters). One run, cVPJiVRhL2jN88ovAPyaf, hit an upstream Google 502 and was not billed; a verbatim retry succeeded.
Motion — google/veo-3.1/first-last-frame-to-video (8 s fixed, $3.20/clip, 1080p, 9:16)
| Clip | Request ID | From → To | Beat | Windows kept | Speed | Cut |
|---|---|---|---|---|---|---|
| C1 | LEjPZMtsPiTPpA56jwIW2 | K00 → K01 | glass rises, cards populate | 0.00–3.40 + 4.30–5.30 | 1.69× | 2.67 s |
| C2 | 9IwtggZssIAZiwnAQHhRi | K01 → K02 | three taps, holograms, cart 0→3 | 0.00–8.00 | 3.02× | 2.67 s |
| C3 | w3XIl1UefHPFZbKIrZfpN | K02 → K04 | press Visualize, shatter, materialize | 0.60–5.30 | 1.62× | 2.92 s |
| C4 | 0iraXwuyB1UDswaqQLlQf | K04 → K05 | swipe → bouclé | 1.20–6.40 | 4.00× | 1.30 s |
| C5 | yuAnLG2rxGvNob6cyVI3D | K05 → K06 | swipe → japandi | 0.00–2.80 | 2.15× | 1.34 s |
| C6 | OQZDE0J6wZABcnbOiZlEY | K06 → K07b | swipe → cognac, dock slides in | 0.30–3.40 + 6.40–8.00 | 3.36× | 1.42 s |
| C7 | lxVi37RfZxqAD2dnHIWce | K07b → K08 | drag-drop rug/armchair/table, UI clears | 1.20–7.30 | 2.49× | 2.47 s |
| C8 | — | freeze of C7's last frame | hold on the finished room | — | 1.00× | 0.40 s |
Discarded clips (the giant-hand bug): S0ahwVOfurQ8BARrtVqTR (C4 v1) and Ea88QtPEQmuM3gOPBTRGb (C6 v1). Both were prompted for a "fingertip swipe" in a wide shot; both rendered an oversized arm, and C6 v1 additionally blew the side dock up into a white column. Re-prompted with no hand and an explicit negative prompt.
Music — google/lyria2
qYXx0iO399lDm0EmX6eHW · seed 715 · https://media.modelrunner.ai/tuAMtXlcPxZAIuQn7gsTM.wav — ~32.8 s of instrumental lo-fi house at 118 bpm; the first 15 s are used, mixed at 0.58 gain against the SFX at 0.85 (so the bed sits ~3.3 dB under), with a 0.6 s fade-out.
Veo inputs (graded keyframes, uploaded once via presigned PUT)
Run parameters
Veo: aspect_ratio: "9:16" · resolution: "1080p" · fixed 8 s output, no duration param on this endpoint.
Nano Banana: image_size: "9_16_2k" · multi-image input via images: [previous_keyframe, ...product_refs] · prompt-level white-balance instructions are ignored — correct colour in post.
Post (ffmpeg): per-channel gain to K04's floor balance (136,106,89) → hqdn3d chroma denoise scaled to edit depth → crop/scale to exact 1080×1920 → setpts retime with an atempo chain on audio → acompressor → two-pass loudnorm → −14 LUFS.
Keep reading
- Generating App Store Screenshots With AI: A Home Redesign App Case Study8 min read
Generate App Store marketing screenshots with AI for a home redesign iOS app, device mockups and ASO-aware headlines included, in one Claude Code session.
- Generating Demo Assets for an iOS App With AI: A Home Redesign App Case Study7 min read
Generate AI demo assets for an iOS app and load them into the Simulator's Photos library with xcrun simctl addmedia, via a Claude Code workflow.
