While chatting with Geektime, CEO and co-founder Ofir Krakowski explains that development requires the combination of different technologies like advanced research of learning models, Deep Learning, signal processing, unique actor voice modeling, Neural style transfer, and NLP. The service’s initial launch will include deep dubs in English, Spanish, French, and German. In order to keep the actor’s unique voice, the platform learns and trains on different clips of the actor’s original voice, which it then converts to the target language.
The software developed by Israeli company Deepdub allows content creators and production companies to localize content from one language to another by injecting the different phases with Deep Learning. Or if you want, “Deep Fake” the human voice. Now, an Israeli startup wants to toy with AI algorithms to create a dubbing platform for foreign languages but in the actor’s original voice - eliminating the need for local voice-over actors. While we might recognize the distinctive voices of famous Hollywood actors, to people living in dubbed over countries, these famous voices mean nothing. This resulted in streaming services (like Netflix) or Hollywood film distributors to adapt their content and provide voice-over versions - even for famous Hollywood stars - in the native language.
However, for many countries around the world, such as Russia, China, France, Italy, and Spain for example, experience voice-over, dubbed content on a daily basis, heck it’s even part of the culture. Dubbed content is an alien concept to the American or British television viewer, that is unless you watch Chinese martial arts films.