Yandex has developed a voice-over function for English-language videos on YouTube and other popular video services.
English-language videos in Russian voice acting can be watched in Yandex.Browser from a computer and Android devices and in the Yandex application for Android and iOS. When a video is enabled, the user is prompted to automatically translate it. The original soundtrack is muted, and then a new one is superimposed on it. For voice-overs, both male and female voices are used.
According to the company, the translation is carried out in several stages. At the first stage, Yandex recognizes speech, converts it into text, and uses neural networks to break it down into sentences. After that, the gender of the speaker is determined, the sentence is translated into Russian, and the voice is synthesized. Then Yandex combines translation with video. The translation process takes a few minutes.
According to the head of Yandex.Browser Roman Ivanov, the company started to implement the idea “to completely erase language boundaries on the Internet” ten years ago. Currently, Yandex services can translate text, images, video, and in the future, developers plan to add new language pairs and voices to video translation.
