/>
X
Innovation

How Google Lookout's AI can describe images for the visually impaired

With Image Q+A, users can ask questions - via voice or text - about any uploaded image. Here's how to access it.
Written by Sabrina Ortiz, Editor
boyfacegooglescreenshot-2024-02-07-153113.png
Screenshot by David Grober/ZDNET

Even before the generative AI boom, the Google Lookout app leveraged AI to assist the visually impaired and blind community explore their surroundings using their phone cameras. Launched in March 2019, the app recently added a handy AI-powered feature -- Image Q+A. 

Also: What is Google Bard? Here's everything you need to know

The Image Q+A feature allows users to ask questions about an image by uploading the photo and using their voice or entering text. The user will then receive a detailed description of the image addressing their concerns. 

For example, you can ask questions regarding what color a subject in the image is, specific details regarding a subject, such as their facial expression, and even ask the app to read text within the image, such as what a sign says. 

Although the feature was released in the fall, Google this week shared more insight about the feature via a post on X, formerly Twitter, also showcasing how users have benefited from the technology.

Underlying the technology is Google's AI model, which was trained to understand and give specific descriptions of videos according to Google. 

You can access the feature on the app, which is free to download. However, the feature is only available in English in the US, UK, and Canada. 

Also: I just tried Google's ImageFX AI image generator, and I'm shocked at how good it is

The app also includes several other innovative features, including a Text mode, which allows users to skim the text and hear it read aloud; a Food Label mode, which can identify packaged foods by their label; a Currency mode, which can identify dollars, euros, and Indian rupees quickly; and more. 

Editorial standards