Microsoft has previously demonstrated holograms, but the translation aspect of the technology is a first for the company. It appears to be a development project for the time being, but with Microsoft’s vast resources, it could soon become a commercial reality. You don’t have to speak another language to appear fluent in it to anyone, anywhere.
At Microsoft’s Inspire partner conference, Microsoft executive Julia White demonstrated how it is now possible to not only create an incredibly life-like hologram of a person but also have the hologram speak another language in the person’s own voice.
This demonstration was made possible by combining two existing technologies mixed reality and neural text-to-speech and it foreshadows a future in which technology greatly reduces existing barriers to human communication.
The Hololens is Microsoft’s take on augmented reality, or “mixed reality.” These holograms can be used to display information, blend with the real world, or even simulate a virtual world by utilizing multiple sensors, advanced optics, and holographic processing that blends seamlessly with its environment.
Microsoft has created a hologram that will transform someone into a digital speaker of another language. The software giant unveiled the technology during a keynote at the Microsoft Inspire partner conference this morning in Las Vegas.
This demonstration was made possible by the use of two existing technologies: mixed reality and neural text-to-speech. White’s seemingly identical, life-sized hologram was initially created at the company’s Mixed Reality Studio. The voice of White was then sampled using Azure’s AI technology, which converted it to Japanese using Neural-Text-to-Speech. This is how her vocal tone and delivery style were preserved in the Japanese version of her presentation.
TEAM EFFORT
Azure executive Julia White, a HoloLens 2 headset, and White’s hologram were all part of Microsoft’s demonstration of the technology. White’s hologram began as a small green outline of a hologram that she could hold in her hand, but as soon as she said two simple words, “render keynote,” the small hologram grew into a fully rendered, human-sized replica of White and began delivering the keynote speech in Japanese, in a voice that matched White’s.
The first step in how Microsoft pulled off this remarkable demonstration was to create White’s seemingly identical, life-sized hologram. To accomplish this, she went to one of the company’s Mixed Reality capture studios and was recorded delivering her presentation in English. That would be enough to make anyone wearing one of Microsoft’s HoloLens devices feel as if White were standing right in front of them, delivering her speech.
Microsoft then used its neural text-to-speech technology to create White’s “personalized voice signature” to make it appear as if he was speaking Japanese. It could then have the hologram say anything, in any language, and the audio would sound like White’s voice.
THE FUTURE IS HERE
Of course, an international audience could benefit from White’s presentation even if Microsoft didn’t turn her into a hologram or use AI to replicate her voice, they could watch her on their television or computer screen and listen along as a human translator repeated White’s words in their native language.
However, this technology eliminates many of the barriers that existed between the speaker and the audience. Consider a world leader delivering a speech and every person on the planet feeling as if the leader was in the room with them, speaking the local language. Or a world-class professor giving a lecture that anyone could attend and understand without having to leave their homes or learn their teacher’s language.
Then there’s the exciting prospect of what might happen next. Mixed reality contact lenses that eliminate the need for a headset entirely? AI that can translate speech in real-time into the native speaker’s voice?
As White stated at the end of her demonstration, “all of these technologies exist today.” The future has arrived.”
While this latest advancement in Microsoft’s hologram technology is unlikely to be commercially available anytime soon, the possibilities for its application are intriguing, as it could have far-reaching effects on communications, travel, and international business. Furthermore, it appears at least according to a website dedicated to promoting the device that the HoloLens 2 is primarily aimed at businesses, as many of the apps and solutions developed for the headset are intended for use in workplaces rather than at home.
While this technology is not yet available to the general public, it shows that language and even distance are no longer barriers. In the not-too-distant future, speakers like White will be able to use this technology to deliver presentations all over the world while simultaneously being in multiple locations and speaking multiple languages.
