In an effort to provide assistance where it can, Microsoft in 2016 launched Project Tokyo, a partnership among researchers in the U.S., U.K., China, Japan, and India to explore technologies that might help those with impairments interact with the world around them.
Four years later, it’s borne fruit with a device — a modified version of Microsoft’s HoloLens augmented reality headset — imbued with algorithms that provide info about people within the wearer’s surroundings.
According to a blog post published by Microsoft, the research group began by following athletes and spectators with varying levels of vision on a trip from the U.K. to the 2016 Paralympic Games in Rio de Janeiro, Brazil, observing how they interacted with other people as they navigated airports, attended sporting venues, and went sightseeing, among other activities. Machine learning experts on the Project Tokyo team then developed the aforementioned algorithms, which run on graphical processing units housed in a PC connected to a HoloLens from which the front lenses have been removed.