Ask Google Photos to help make sense of your gallery

Google is inserting more of its Gemini AI into many of its product and the next target in its sights is Photos. At its I/O developer conference today, the company's CEO Sundar Pichai announced a feature called Ask Photos, which is designed to help you find specific images in your gallery by talking to Gemini.

Ask Photos will show up as a new tab at the bottom of your Google Photos app. It'll start rolling out to One subscribers first, starting in US English over the upcoming months. When you tap over to that panel, you'll see the Gemini star icon and a welcome message above a bar that prompts you to "search or ask about Photos."

According to Google, you can ask things like "show me the best photo from each national park I've visited," which not only draws upon GPS information but also requires the AI to exercise some judgement in determining what is "best." The company's VP for Photos Shimrit Ben-Yair told Engadget that you'll be able to provide feedback to the AI and let it know which pictures you preferred instead. "Learning is key," Ben-Yair said.

You can also ask Photos to find your top photos from a recent vacation and generate a caption to describe them so you can more quickly share them to social media. Again, if you didn't like what Gemini suggested, you can also make tweaks later on.

For now, you'll have to type your query to Ask Photos — voice input isn't yet supported. And as the feature rolls out, those who opt in to use it will see their existing search feature get "upgraded" to Ask. However, Google said that "key search functionality, like quick access to your face groups or the map view, won't be lost."

The company explained that there are three parts to the Ask Photos process: "Understanding your question," "crafting a response" and "ensuring safety and remembering corrections." Though safety is only mentioned in the final stage, it should be baked in the entire time. The company acknowledged that "the information in your photos can be deeply personal, and we take the responsibility of protecting it very seriously."

To that end, queries are not stored anywhere, though they are processed in the cloud (not on device). People will not review conversations or personal data in Ask Photos, except "in rare cases to address abuse or harm." Google also said it doesn't train "any generative AI product outside of Google Photos on this personal data, including other Gemini models and products."

Your media continues to be protected by the same security and privacy measures that cover your use of Google Photos. That's a good thing, since one of the potentially more helpful ways to use Ask Photos might be to get information like passport or license expiry dates from pictures you might have snapped years ago. It uses Gemini's multimodal capabilities to read text in images to find answers, too.

Of course, AI isn't new in Google Photos. You've always been able to search the app for things like "credit card" or a specific friend, using the company's facial and object recognition algorithms. But Gemini AI brings generative processing so Photos can do a lot more than just deliver pictures with certain people or items in them.

Other applications include getting Photos to tell you what themes you might have used for the last few birthday parties you threw for your partner or child. Gemini AI is at work here to study your pictures and figure out what themes you already adopted.

There are a lot of promising use cases for Ask Photos, which is an experimental feature at the moment and that is "starting to roll out soon." Like other Photos tools, it might begin as a premium feature for One subscribers and Pixel owners before trickling down to all who use the free app. There's no official word yet on when or whether that might happen, though.

Catch up on all the news from Google I/O 2024 right here!