This new AI dubbing tool can translate audio while preserving the speaker's voice
If you have ever watched a film or show that has been dubbed, you know how unnatural it feels. Oftentimes, the speaker's voice won't change amongst different characters, or the patterns and fluctuations are entirely off. ElevenLab's new AI feature is meant to remedy that problem.
On Tuesday, ElevenLabs unveiled AI Dubbing, a new feature that can convert spoken content to another language while keeping the voice, speech patterns, emotions, and intonations of the original speaker.
Also: Gen AI a job threat? On the contrary, human workers have much to gain
The AI Dubbing feature is meant to allow audiences globally to enjoy their favorite content in their native language by converting it in just minutes.
The AI Dubbing feature combines ElevenLab's multilingual speech synthesis, voice cloning, text, and audio processing to preserve the nature of the original content when creating a translation, according to the company.
In the release, the CEO and co-founder of ElevenLabs, Mati Staniszewski, shared his negative experience with dubbed content growing up and how the new technology could be a solution.
"Growing up in Poland, we watched English language movies that had been dubbed by a single narrator. It means every actor has the same voice. It takes so much magic out of the experience," said Staniszewski.
"The release of AI Dubbing is our biggest step yet towards eliminating these linguistic barriers of content. It will help audiences enjoy any content they want, regardless of the language they speak."
Also: AI and data: Honing hyper-personalization to build the bank of the future
The AI Dubbing feature will support over 20 languages currently supported by the Eleven Multilingual v2 model.
The full list includes English, Japanese, Chinese, German, Hindi, French, Korean, Portuguese, Italian, Spanish, Indonesian, Dutch, Turkish, Filipino, Polish, Swedish, Bulgarian, Romanian, Arabic, Czech, Greek, Finnish, Croatian, Malay, Slovak, Danish, Tamil, and Ukrainian.
AI Dubbing is available for all users starting today, and the pricing is dependent on how many characters you want to be dubbed.