The white cane is still one of the most common tools that blind and visually impaired people use to stay mobile. But think instead how valuable an electronic device would be if it could communicate on the surrounding environment to its user.
Catalin gradually lost his sight during childhood. Nowadays he teaches in a school for visually impaired people and he’s among the volunteers at the Politehnica University of Bucharest testing a new kind of navigation system for blind people. His experience will help researchers customize the device for users.
“The system includes a 3D camera that continuously scans the environment, very fast, 20 times a second,” says Alin Moldoveanu, Technical Coordinator of the Sound of Vision Project and Professor at the Politehnica University of Bucharest. “Then we process that information with computer vision algorithms and we break down every individual object. Each individual object is presented to the user through 3D computer-generated sounds and he hears the object through vibrations on a belt on his abdomen.”
The user is guided in real-time by the 3D sounds together with vibrations or haptic feedback. The intensity of the feedback varies depending on the user’s distance from the object. Researchers on this European project say it’s like creating a new sense for visually impaired people, who are also able to read signs with the app.
“The user when he wants to read a text just presses the button and the system look for texts and indicates him where the texts are,” says Professor Moldoveanu. “He can look toward that direction and hear the text: pharmacy or supermarket or metro station”
The system identifies the most important objects like stairs, doors or people moving around. Every change in the surrounding environment is immediately detected. Users can also customize some functions according to their needs.
“Blind people can only ‘see’ what the white cane touches, it means what surrounds them one meter in front and up to one meter in height,” says Runar Unnporsson, Sound of Vision Project Coordinator. “What the Sound of Vision prototype does is to allow them to see further and higher up.”
Research on outdoor use of the device is being led by the Technical University of Iasi.
“Regarding outdoor use, we rely on two stereo cameras, two cameras that see the environment at the same time, and based on these two images we estimate distances from the objects in the environment,” says Simona Caraiman, Professor of Computer Engineering at the Technical University of Iasi. “We need to understand that this is a life-critical application, so the system cannot fail when signaling dangerous situations like a hole in the ground, for example.”
The three-year project started in 2015 with the participation of nine partners from five European countries, two from Iceland, three from Poland, one partner from Hungary and Italy, and three from Romania.
Researchers are currently fine-tuning the indoor and outdoor systems to combine them together. The next step will be to miniaturize the device for commercialization within the next two years.
