Be My Eyes, a Danish startup, applied the AI model to a new feature called Virtual Volunteer, for blind or partially sighted people.
What are some of the features of the object-recognition tool?
The object-recognition tool can answer questions about any image that it’s sent.
For instance, if a user is hungry. They could simply photograph an ingredient and request related recipes.
If they’d rather eat out, they can upload an image of a map and get directions to a restaurant. On arrival, they can snap a picture of the menu and hear the options. If they then want to work off the added calories in a gym, they can use their smartphone camera to find a treadmill.
What has been the feedback from the blindness community?
Several of the beta testers have used the phrase ‘life-changing’ when describing the product. This has a chance to be transformative in empowering the community with unprecedented resources to better navigate physical environments, address everyday needs, and gain more independence.
How does GPT-4 differ from previous iterations?
Virtual Volunteer takes advantage of an upgrade to OpenAI’s software. Unlike previous iterations of the company’s vaunted models, GPT-4 is multimodal, which means it can analyze both images and text as inputs.
The image recognition offered by GPT-4 is superior, and the analytical and conversational layers powered by OpenAI increase value and utility exponentially.
How will this change the experience for the Be My Eyes users?
Be My Eyes previously supported users exclusively with human volunteers. According to OpenAI, the new feature can generate the same level of context and understanding. But if the user doesn’t get a good response or simply prefers a human connection, they can still call a volunteer.
Ultimately, the platform will provide users with both support and opportunities. Be My Eyes will also soon help businesses to better serve their customers by prioritizing accessibility.
How can people test the Virtual Volunteer?
If you or someone you know is visually impaired and wants to test the Virtual Volunteer, you can register for the wait list here.
