Google sharpens up Now on Tap with precise text, image search

Google is filling some key gaps in its Now on Tap search feature, such as identifying images and returning contextual information about them.
Written by Liam Tung, Contributing Writer

Now pressing the home button will bring up information about the image on screen and links to relevant apps.

Image: Google

Google has rolled out some useful new features in Now on Tap, which started shipping with Android Marshmallow last year.

Now on Tap helps users retrieve information relevant to their current screen by holding down the home button. The feature is an evolution of the Google Now assistant for Android but is exclusive to Android Marshmallow, which still only runs a small percentage of Android phones and tablets.

While Now on Tap was meant to help users find relevant information without requiring them to leave their current screen, it often wasn't able to find the right information on a screen with lots of text.

With this update, Marshmallow users can pinpoint the exact word or phrase on a page for Google to look up by highlighting the text. For a word lookup, Now on Tap will present a definition as well as links to relevant apps.

"If you use Now on Tap in an app, email, chat, or news article with a lot of text, sometimes the results aren't as precise as you'd like. Starting today, you can give Google a nudge by selecting exactly what you want help with, and you'll get the right information, right away," Google product manager Aneto Okonkwo said in a blogpost.

For now, word definitions are limited to English but Google is rolling it out for other languages in coming weeks, according to Okonkwo.

Another notable shortcoming in Now on Tap was the inability for it to identify images and return contextual information about them. That's also been addressed in this update.

Pressing the home button for an image on screen will bring up cards with information about the image and links to relevant apps. It works with Google Photos as well as other image-heavy apps, such as Pinterest.

The feature also works in the camera app for real-time search. Okonkwo noted that the camera search works for "certain images or objects" including, but not limited to, well-known structures such as the Bay Bridge, and could be useful for calling up information by pointing the camera to a movie poster or magazine.

Read more

Editorial standards