In a significant stride towards enhancing user experience on Android, Google has unveiled its early-stage developer capabilities designed to connect applications with intelligent agents and personalized assistants, notably Google Gemini. This initiative aims to redefine the app ecosystem while prioritizing privacy and security as foundational elements.
While we are in the early, beta stages of this journey, we’re designing these features with privacy and security at their core as our first step in exploring this paradigm shift as an app ecosystem.
AppFunctions
At the heart of this development is AppFunctions, a feature introduced with Android 16 and now receiving comprehensive attention. This functionality allows applications to expose specific capabilities for access by agent apps, enabling seamless execution of tasks directly on the device.
AppFunctions is an Android 16 platform feature and an accompanying Jetpack library that allows apps to expose specific functions for callers, such as agent apps, to access and execute on device.
Developers can delineate their app’s functionalities as tools for AI assistants like Gemini, paralleling the Model Context Protocol (MCP) commonly utilized for server-side tools, but with the advantage of local execution. Here are some illustrative use cases:
- Task management and productivity
- User request: “Remind me to pick up my package at work today at 5 PM“.
- AppFunction action: The caller identifies the relevant task management app and invokes a function to create a task, automatically populating the title, time, and location fields based on the user’s prompt.
- Media and entertainment
- User request: “Create a new playlist with the top jazz albums from this year“.
- AppFunction action: The caller executes a playlist creation function within a music app, passing context like “top jazz albums for 2026” as the query to generate and launch the content immediately.
- Cross-app workflows
- User request: “Find the noodle recipe from Lisa’s email and add the ingredients to my shopping list“.
- AppFunction action: This request utilizes functions from multiple apps, retrieving content from an email app and extracting relevant ingredients to populate a shopping list app.
- Calendar and scheduling
- User request: “Add Mom’s birthday party to my calendar for next Monday at 6 PM“.
- AppFunction action: The approved agentic app invokes the calendar app’s “create event” function, parsing context to create the entry without manual input.
A practical demonstration of AppFunctions can be seen with the Samsung Gallery app on the Galaxy S26, where users can simply ask Gemini to retrieve specific photos. The assistant intelligently identifies the request, triggers the appropriate function, and presents the results within the Gemini app, enhancing user engagement through a multimodal experience.
Instead of manually scrolling through photo albums, you can now simply ask Gemini to “Show me pictures of my cat from Samsung Gallery.” Gemini takes the user query, intelligently identifies and triggers the right function, and presents the returned photos from Samsung Gallery directly in the Gemini app, so users never need to leave.
Moreover, Google has integrated AppFunctions into its Calendar, Notes, and Tasks features within Google apps and OEM defaults, showcasing the practical applications of this technology.
UI Automation
In addition to AppFunctions, Google is also advancing a UI automation framework designed for AI agents and assistants. This framework enables the intelligent execution of generic tasks across installed applications, offering developers a streamlined approach to enhance their apps’ capabilities without extensive coding.
This is the platform doing the heavy lifting, so developers can get agentic reach with zero code. It’s a low-effort way to extend their reach without a major engineering lift right now.
Looking ahead, Google plans to expand these capabilities with Android 17, aiming to reach a broader audience of users, developers, and device manufacturers. The company is currently collaborating with a select group of app developers to refine user experiences as this ecosystem evolves.
We are currently building experiences with a small set of app developers, focusing on high-quality user experiences as the ecosystem evolves. We plan to share more details later this year on how you can use AppFunctions and UI automation to enable agentic integrations for your app. Stay tuned for updates.
