Huawei is using artificial intelligence (AI) to help blind people “see” the faces of people with whom they’re speaking. The Chinese company debuted Facing Emotions, an app for the Huawei Mate 20 Pro that can “translate” seven different universal emotions — anger, fear, disgust, happiness, sadness, surprise, and contempt — into seven corresponding sounds.
Huawei says it partnered with the Polish Blind Association to aid in the Android app’s development, and that it recruited a group of blind testers and interviewed blind community representatives about their needs. Those insights helped to shape the app’s user experience, even down to its colors and sounds.
“Facing Emotions was created to push the possibilities of AI technology. By asking ‘how can AI help the blind and visually impaired to experience or see more of the world?’” Huawei wrote in a press release. “[It makes] it possible for the blind and visually impaired to have a richer experience and understanding when communicating with others.”
Facing Emotions uses the Mate 20 Pro’s back cameras to scan the faces of conversation partners, identifying facial features like eyes, nose, brows, and mouth, and their positions in relation to each other. An offline, on-device machine learning algorithm interprets the detected emotions as sounds, which the app plays on the handset’s loudspeaker.
Blind composer Tomasz Bilecki’s sound clips — which Huawei describes as “short, concise, simple, and non-invasive” — were tested by the aforementioned team of blind consultants. And designer Janek Kochanski developed a 3D printed phone holder that allows visually impaired people with canes to use the app hands-free, which is now available for free on Shapeways and partners in key countries.
The debut Facing Emotion app’s debut follows on the heels of Huawei’s StorySign, which taps AI tools such as image recognition and optical character recognition (OCR) to translate popular children’s books into sign language.
“The Facing Emotions campaign embodies Huawei’s mission to bring new technology to everyone, especially those that need it most,” Wang Tao, president of Huawei’s Consumer Business Group said in a statement.
