Google’s Gemini on Android, its AI replacement for Google Assistant, will soon be taking advantage of its ability to deeply integrate with Android’s mobile operating system and Google’s apps. At the Google I/O 2024 developer conference on Tuesday, the company announced that users will be able to pull up the Gemini overlay on top of the app they’re using in more ways. It’s also updating Android’s built-in AI model, Gemini Nano. 

Soon, Android users will be able to drag and drop AI-generated images directly into their Gmail, Google Messages and other apps. Meanwhile, YouTube users will be able to tap “Ask this video” to find specific information from within that YouTube video, Google says. 

Those who pay for the upgraded Gemini Advanced will also have the ability to use an “Ask this PDF” option that lets you get answers from the document without having to read through all the pages. Gemini Advanced subscribers pay $19.99 per month for access to AI and receive 2TB of storage along with other Google One benefits.

Already, Gemini on Android could do other things like generate captions on photos, ask questions about articles you’re reading, and perform other generative AI tasks, similar to other AI chatbots. However, OpenAI upstaged Google’s event to announce a GenAI model, GPT-4o (with the o standing for “omni”), that works with text, speech, and video, including what the phone’s camera is seeing. So despite Gemini’s built-in advantages, it will have some competition on mobile devices.

Google says the latest Gemini on Android features will roll out to hundreds of millions of supported devices over the next few months. Over time, Gemini will evolve to offer other suggestions related to what’s on your screen as well. 

Meanwhile, the on-device foundation model on Android, Gemini Nano, will be upgraded to include multimodality. That means it will be able to process text input as well as other means of processing information, including sights, sounds, and spoken language. 

We’re launching an AI newsletter! Sign up here to start receiving it in your inboxes on June 5.