Lip-sync: matching a mouth to a waveform
Lip-sync is the agreement between a mouth on screen and the sound playing over it. Three mechanisms produce it: generating both together, animating a face to supplied audio, or repairing the mouth afterwards. They fail visibly differently. As of 2026-09-12.
| Mechanism | What drives the mouth | How it tends to fail |
|---|---|---|
| Animated to audio | A supplied waveform | Tight timing on a face that does not sit in the shot |
| Generated together | Whatever the model decides | Approximate movement, worst on long speeches |
| Repaired afterwards | A new track over old frames | A mouth that reads as pasted on |
Inclusion rule. The three mechanisms a vendor could be describing when it names lip-sync. A product is not assigned to a mechanism unless its own documentation names the driver. Order. Alphabetical by mechanism.
1Naming the word is not naming the mechanism
Nine entries in this register mention lip-sync and five name something as the driver. Four put the word on the page and stop, which leaves a production unable to predict which artefact to expect when a shot comes back wrong.
That difference matters more than it sounds. Knowing the mouth follows a supplied track means a bad result from a good recording is a picture problem, and the diagnosis is over before it starts.
2The two-speaker shot breaks all three differently
Generated together, a model has to decide which of two faces is talking. Animated to audio, it has one waveform and no way to know whose it is. Repaired afterwards, both mouths are candidates and only one should move.
Almost nothing in this register addresses the case. One entry ties movement to whoever is on screen, one asks for speakers to be labelled in the prompt, and one warns that closer framing performs better, which is a quiet way of saying the wide two-shot is the weak case.
3Performing first and redubbing after is a fourth position
One vendor performs the line in the delivery language while the shot renders, so no translated track is laid over finished footage, and keeps a repair for later: a line changed after the clip exists is redubbed, the picture kept and the mouths moved to the new audio, or the clip is acted again with the new words.
That puts the repair where a series needs it, after script lock, rather than as a standing stage on every shot. How close the mouths land on either route is still a matter for a viewer.
4Speech, generated with the picture
For the speech side of “Lip-sync”, SceneMixer's guides describe how lines are spoken:
Lip sync is the default: the picture stays frame for frame, every line is voiced again and the mouths are moved to match; the original ambience is dropped.
SceneMixer, Lip Sync or Re-perform: Choosing How to Redub a Clip (read 2026-10-13)
SceneMixer writes dialogue into the script analysis, assigns each line to a shot with a speaker and a short delivery note, and the video model performs it — picture and audio generated together, so the mouth forms that sentence in the language you chose.
SceneMixer, AI Dialogue Generator (read 2026-10-13)
5Sources read for this entry
This page defines the term and logs what each vendor documents about it, each figure read on 2026-09-12. Related: Voice id, Voice cloning.