Skype Translator: Breaking Down Language and Hearing Barriers

In 1966, Star Trek introduced us to the notion of the Universal Translator. Such a device allowed Captain Kirk and his crew to communicate with alien species, such as the Gorn, who did not speak their language, or even converse with species who did not speak at all (e.g., the Companion from the episode Metamorphosis). In 1979, Douglas Adams introduced us to the “Babelfish” in the Hitchhiker’s Guide to the Galaxy which, when inserted into the ear, allowed the main character to do essentially the same thing: communicate with alien species who spoke different languages.

Although flawless communication using speech and translation technology is beyond the current state of the art, major improvements in these technologies over the past decade have brought us many steps closer. Skype Translator puts together the current state of the art in these technologies, and provides a speech translation service in a Voice over Internet (VoIP) service, namely Skype. With Skype Translator, a Skype user who speaks, say, English, can call a colleague or friend who speaks, say, Spanish, and be able to hold a bilingual conversation mediated by the translator.

In the Skype Translator project, we set ourselves the ambitious goal of enabling successful open-domain conversations between Skype users in different parts of the world, speaking different languages. As one might imagine, putting together error-prone technologies such as speech recognition and machine translation raises some unique challenges. But it also offers great promise.

The promise of the technologies is most evident with children and young adults who accept and adapt to the error-prone technology readily. They understand that the technology is not perfect, yet work around and within these limitations without hesitation. The ability to communicate with children their own age, irrespective of language, gives them access to worlds that fascinate and intrigue them. The stunning simplicity of the questions they ask, e.g., “Do you have phones?” or “Do you like wearing uniforms in school?”, shows how big the divide can be (or is perceived to be), but it also shows how strongly they wish to connect. Because they also readily adapt the modality of the conversation, e.g., using the keyboard when speech recognition or translation may not be working for them, means they also readily accept the use of the technology to break down other barriers as well. Transcriptions of a Skype call, a crucial cog in the process of speech translation, are essential for those who do not hear, as are the text translations of those transcripts. Freely mixing modalities and readily accepting them offers access to those who might otherwise be barred access. Adjusting the design of Skype Translator to accommodate those with deafness or hard of hearing added features that benefited all users. The technologies behind Skype Translator not only break down the language barrier, they also break down the hearing barrier.