Skip to main content
NewSeedance 2.5 is live→
ModelRunner
bytedance avatar

LatentSync 1.0 API

bytedance/latentsync

Re-sync a talking-head video's mouth to a new audio track with audio-conditioned diffusion — the catalog's lowest-priced lip sync at $0.014 per output second.

edit
0.014 per second of output video
GitHub
License
Weights
Paper
OpenAPI

Model Input

Input

URL of the source video — an MP4 with one clearly visible, front-facing speaker in every frame; the run fails on any frame where no face is detected. The result keeps this video's frame size, and its identity, lighting and background; only the mouth region is regenerated. Video running past the end of the audio is discarded, so trim it to roughly the audio's length.

URL of the speech track the speaker should appear to say (MP3, AAC, WAV or M4A). The result runs for the shorter of this track and the video, trimmed down to a whole multiple of 0.64 seconds — supply audio slightly shorter than the video and expect the last fraction of a second to be cut.

Additional Settings

Customize your input with more control.

Min: 0 - Max: 10

Strength of the audio conditioning during diffusion. The default of 1 leaves classifier-free guidance off; values above 1 switch it on. The model's own demo exposes 1–3.5, though the field accepts up to 10.

Random seed. 0 (the default) draws a fresh random seed on every run, so repeated calls with identical inputs differ; any positive integer is used as given for a repeatable run.

You need to be logged in to run this model and view results.
Log in

Model Output

Output

Loading
Generated in 43.244 seconds
Logs (1 lines)

Model Example Requests

Examples

Example output 1Example output 2

LatentSync 1.0 API

LatentSync 1.0 is a video-to-video AI model by bytedance. On ModelRunner it runs through a REST API or via MCP from any AI assistant, at $0.014 per second of video.

POST https://queue.modelrunner.run/bytedance/latentsync

cURL

# Submit a request to the queue. Input fields go at the top level of the
# body. The optional reserved "metadata" object holds your own flat string
# tags — stored on the request, never sent to the model; filter later with
# GET https://queue.modelrunner.run/requests?metadata=<url-encoded JSON>.
curl -X POST https://queue.modelrunner.run/bytedance/latentsync \
  -H "Authorization: Key $MRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "seed": 0,
    "audio_url": "https://media.modelrunner.ai/r2Nq5Eh8szuYnxfAVRVDG.wav",
    "video_url": "https://media.modelrunner.ai/Waq0PrjekwA7pCyykfy0Z.mp4",
    "guidance_scale": 1,
    "metadata": {
      "project": "my-project"
    }
  }'
# → { "request_id": "...", "status_url": "...", "response_url": "..." }

# Poll status_url until "COMPLETED", then fetch the result
curl "https://queue.modelrunner.run/bytedance/latentsync/requests/$REQUEST_ID/status" \
  -H "Authorization: Key $MRUN_API_KEY"
curl "https://queue.modelrunner.run/bytedance/latentsync/requests/$REQUEST_ID" \
  -H "Authorization: Key $MRUN_API_KEY"

JavaScript

import { modelrunner } from "@modelrunner/client";

const result = await modelrunner.subscribe("bytedance/latentsync", {
  input: {
    "seed": 0,
    "audio_url": "https://media.modelrunner.ai/r2Nq5Eh8szuYnxfAVRVDG.wav",
    "video_url": "https://media.modelrunner.ai/Waq0PrjekwA7pCyykfy0Z.mp4",
    "guidance_scale": 1
  },
});
console.log(result);

Python

import os
import requests

headers = {"Authorization": f"Key {os.environ['MRUN_API_KEY']}"}

submitted = requests.post(
    "https://queue.modelrunner.run/bytedance/latentsync",
    headers=headers,
    json={
      "seed": 0,
      "audio_url": "https://media.modelrunner.ai/r2Nq5Eh8szuYnxfAVRVDG.wav",
      "video_url": "https://media.modelrunner.ai/Waq0PrjekwA7pCyykfy0Z.mp4",
      "guidance_scale": 1
    },
).json()

# Poll submitted["status_url"] until "COMPLETED", then:
result = requests.get(submitted["response_url"], headers=headers).json()

Input parameters

Input parameters of LatentSync 1.0
NameTypeRequiredDescription
video_urlstring (uri)yesURL of the source video — an MP4 with one clearly visible, front-facing speaker in every frame; the run fails on any frame where no face is detected. The result keeps this video's frame size, and its identity, lighting and background; only the mouth region is regenerated. Video running past the end of the audio is discarded, so trim it to roughly the audio's length.
audio_urlstring (uri)yesURL of the speech track the speaker should appear to say (MP3, AAC, WAV or M4A). The result runs for the shorter of this track and the video, trimmed down to a whole multiple of 0.64 seconds — supply audio slightly shorter than the video and expect the last fraction of a second to be cut.
guidance_scalenumbernoStrength of the audio conditioning during diffusion. The default of 1 leaves classifier-free guidance off; values above 1 switch it on. The model's own demo exposes 1–3.5, though the field accepts up to 10. Default: 1.
seedintegernoRandom seed. 0 (the default) draws a fresh random seed on every run, so repeated calls with identical inputs differ; any positive integer is used as given for a repeatable run. Default: 0.

Machine-readable: OpenAPI schema · llms.txt

Use LatentSync 1.0 from Claude & Cursor (MCP)

Point Claude Code, Claude Desktop, Cursor, or any MCP client at the ModelRunner MCP server and LatentSync 1.0 becomes a tool your assistant can call directly — it authorizes via OAuth (no API key in config) and runs this model with the run_model tool using the endpoint bytedance/latentsync.

MCP client config (Claude Desktop, Cursor)

{
  "mcpServers": {
    "modelrunner": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "https://mcp.modelrunner.run/mcp"]
    }
  }
}

Claude Code

claude mcp add --transport http modelrunner https://mcp.modelrunner.run/mcp

Then ask your assistant, for example: “Run bytedance/latentsync on ModelRunner to generate video”. MCP setup guide.

Model Details

Model Details

LatentSync 1.0 takes a talking-head video and a separate speech track and re-animates the speaker's mouth to match the new audio. Identity, lighting and background carry over from the source; only the mouth is regenerated, and the result keeps the input's frame size. It is audio-conditioned latent diffusion: Whisper audio embeddings are cross-attended into the diffusion model with no intermediate landmark or motion representation, and TREPA, LPIPS and SyncNet losses keep mouth motion steady frame to frame. At $0.014 per second of delivered video — about $0.28 for a 20-second clip — it is the lowest-priced lip sync in the catalog. Feed it an MP4 with one clearly visible, front-facing speaker plus an MP3, AAC, WAV or M4A speech track; there is no text prompt.

## Best for - Dubbing talking-head footage into another language so the lips match the translated voiceover - Re-voicing an interview, tutorial, or ad with a cleaner or re-recorded audio take - Fixing a take where the recorded speech and the mouth have drifted out of sync - Chaining after a text-to-speech model: generate the voice, then lip-sync the footage to it - High-volume or long-form lip sync where cost per second matters

## Choose another model when - You need control over how a length mismatch is handled — this model always trims to the shorter of the two; `sync/lipsync/v2` exposes `sync_mode` with loop, bounce, silence and remap - You are starting from a still photo rather than footage — use `bytedance/omnihuman/v1.5` or `wan-video/wan/v2.7/image-to-video/audio-driven` to animate a portrait - You still need to produce the speech — this endpoint takes an existing audio file and no text prompt, so run a text-to-speech model first and feed its output here

## Tips - Supply audio slightly shorter than the video: the result runs for the shorter of the two, trimmed to a whole multiple of 0.64 seconds, so a trailing fragment of speech is dropped - Trim the source video to roughly the audio's length; extra video is discarded and only slows the run

## Limitations - Every frame needs a detectable face: a cutaway or a frame with no face fails the whole run (nothing is charged) - Frame rate is not adjustable — the result is delivered at 25 fps, so other rates are resampled - Mouth detail is modest; this generation was not trained at high resolution - The soundtrack comes back re-encoded to 16 kHz mono AAC

To run via the ModelRunner JavaScript client: ```js import { modelrunner } from "@modelrunner/client";

const result = await modelrunner.subscribe("bytedance/latentsync", { input: { video_url: "https://media.modelrunner.ai/NLaN6i1oQvbd8Mn4oEDXs.mp4", audio_url: "https://media.modelrunner.ai/lZfxwe6ZN6ZikXQhVtiq7.wav", }, }); ```