Introducing Multi-Speaker Lip-Sync: Revolutionizing Dubbed Content with AI
An AI-powered video and audio localisation tool, ASSISTANT, has announced the launch of its new Multi-Speaker Lip-Sync feature. With AI-powered lip-sync, 750,000 users can translate their content into 130+ languages to sound as fluent as a native speaker.
Overcoming the Challenge of Lip-Sync Discrepancy
For a long time, there has been a lack of synchronization between lip movements and voices in dubbed content. Experts believe this is one of the reasons why dubbing is relatively unpopular in English-speaking countries. In fact, lip movements make localized content more realistic and therefore more appealing to audiences.
The Importance of Lip Reading and Language Learning
According to a study by Yukari Hirata, a professor known for her work in linguistics, watching lip movements (rather than gestures) helps to perceive difficult phonemic contrasts in the second language. Lip reading is also one of the ways we learn to speak in general.
Rask AI’s Multi-Speaker Lip-Sync Feature
Today, with Rask’s new feature, it’s possible to take localized content to a new level, making dubbed videos more natural. The AI automatically restructures the lower face based on references. It takes into account how the speaker looks and what they are saying to make the end result more realistic.
How it Works
According to Maria Chmir, founder and CEO of Rask AI, the new feature will help content creators expand their audience. The AI visually adjusts lip movements to make a character appear to speak the language as fluently as a native speaker.
Technology Backed by Generative Adversarial Network (GAN) Learning
The technology is based on generative adversarial network (GAN) learning, which consists of a generator and a discriminator. Both the generator and the discriminator compete with each other to stay one step ahead of the other. The generator clearly generates content (lip movements), while the discriminator is responsible for quality control.
Beta Release Available to All Rask Subscription Customers
(Editor’s note: This article is sponsored by )
