development tasks Archives

AppWizard

September 18, 2026

Android Bench 2.0 focuses on long-horizon tasks, agent evaluations

Google has released version 2.0 of Android Bench, which focuses on managing complex development tasks rather than minor adjustments. The new version evaluates tasks that may take engineers days or weeks to complete, such as adding features and building applications. A continuous scoring method has replaced the previous pass or fail system, assessing completion rates based on functionality, visual fidelity, and adherence to instructions. Various AI models have been tested, with GPT-6 Astra achieving a 28% pass rate, significantly lower than the previous scores around 90%. No model has achieved a 100% pass rate in porting cross-platform applications, with the best reaching 80%. AI performs better in writing new code than in refactoring existing code, facing challenges with architectural complexity and runtime validation. Android Bench utilized agents from model providers for evaluations and plans to incorporate various model combinations in future updates.

AppWizard

September 18, 2026

Google just put the latest AI models through a brutal coding test

Google has launched Android Bench 2.0, a benchmark for evaluating large language models (LLMs) and AI agents on complex Android development tasks. This version focuses on long-horizon tasks (LHTs) that are more intricate than those assessed by the original benchmark. Key tasks include upgrading dependencies, adding major new features, and building Android apps from scratch. The new grading system uses continuous scoring instead of a binary pass-or-fail method, providing a more detailed performance evaluation. Currently, GPT-6 Astra leads the leaderboard with a 28% pass rate, followed by Gemini 3.8 Flash at 8%. Other evaluated models include Claude Fable 5.1, GPT-5.6 Sol, and Claude Opus 5. Google plans to expand the leaderboard with more models and results.