app interaction

Oppo's Multi-X team has introduced X-OmniClaw, an open-source AI agent for Android that operates on the device without cloud processing. It uses the camera, screen, and voice functionalities to perform tasks across applications. Unlike cloud-based platforms, X-OmniClaw processes information locally, with the cloud serving as a supplementary resource. The architecture integrates three perception channels into a unified pipeline, allowing it to interpret scenes and user requests effectively. It transforms local data into semantic entries for long-term memory, processes gallery photos into descriptions, and filters out sensitive information. X-OmniClaw captures user behavior into reusable skills, enabling direct navigation to app pages through deeplinks. Demonstrations show its ability to retrieve product prices, assist with homework, and create highlight albums from photos. The project is built on the open-source HermesApp codebase and is accessible on GitHub. It draws inspiration from existing models, including Google's local model and ByteDance's UI-TARS, while enhancing functionality through on-device execution and structural XML data integration.