Caltech is using mixed reality to improve the lives of blind people. New research combines augmented reality with computer vision algorithms that will allow developers to build software that will enable objects to “talk.”
The headset, CARA (Cognitive Augmented Reality Assistant), is designed to translate the visual world into English audio. The researchers believe that the device can be used in banks, stores, museums, and other locations to make those spaces more accessible for the blind.
“Imagine you are in a world where all the objects around you have voices and can speak to you,” said Markus Meister, one of the authors of the paper and faculty member at the Tianqiao and Chrissy Chen Institute for Neuroscience at Caltech. “Wherever you point your gaze, the different objects you focus on are activated and speak their names to you. Could you imagine getting around in such a world, performing some of the many tasks that we normally use our visual system for? That is what we have done here—given voices to objects.”