Amazon Launches New Alexa Skill That Identifies Groceries for People Who Are Blind and Visually Impaired

Amazon launched a new first-party Alexa Show skill called Show and Tell in order to help the visually impaired identify groceries and other household items like boxes, cans, or other packaged items that are hard to identify by touch.

Amazon worked with the Vista Center for the Blind in Santa Cruz, California, as well as its own blind and low-vision employees to develop Show and Tell. Users can ask Alexa on their Echo Show “what am I holding?” or “what’s in my hand?” to activate the Show and Tell feature. Alexa first helps guide the user in placing the object in front of the camera on the smart display, then using a combination of computer vision and machine learning, Alexa is able to identify the item and tell the user. Right now, Show and Tell is restricted to second-generation Echo Show devices in the U.S., though there are plans to extend it to other countries soon.

“The whole idea for Show and Tell came about from feedback from blind and low vision customers,” Amazon’s Alexa for Everyone team head Sarah Caplener said in the blog post announcing the new feature. “We heard that product identification can be a challenge and something customers wanted Alexa’s help with. Whether a customer is sorting through a bag of groceries, or trying to determine what item was left out on the counter, we want to make those moments simpler by helping identify these items and giving customers the information they need in that moment.”

Amazon has been eager to show off how accessible Alexa is to people with visual and other impairments of late. The company debuted an ad this summer in the UK specifically highlighting how the voice assistant can aid someone with low-vision. The ad was developed in partnership with the Royal National Institute of Blind People, an arrangement that includes Amazon staff training people at the RNIB on how Alexa can help those with visual impairments.

“We’re excited to see features such as Show and Tell added to the Echo platform and appreciate Amazon’s commitment to enhancing the Echo with features that will significantly benefit people who are blind or visually impaired,” Kevin Lynch, president and CEO of National Industries for the Blind, the largest employment resource for people who are blind in the U.S.

However, Alexa’s voice technology can aid more than the blind and low-vision community. The voice assistant also aids those with limited hearing by offering real-time captioning of its responses and Tap to Alexa, a non-audio way to communicate with the voice assistant. Voice also offers access to technology to people who may not be able to type or use a touchscreen.

Much like the recent ad, the new feature is a good way to build public approval for a product at the center of some turmoil over who has access to recordings made by Alexa. Amazon would much prefer people discuss how Alexa is improving the lives of people with disabilities instead of what the voice assistant may have recorded and distributed to contractors to review.