Microsoft has recently introduced an innovative on-device small language model named “Mu,” designed to enhance the functionality of AI agents within the Windows 11 Settings app. This model operates on Neural Processing Units (NPUs), allowing it to leverage the AI-optimized hardware found in Copilot+ PCs. These AI agents, which were first announced earlier this year, are capable of automating and executing various tasks on users’ PCs, provided that the necessary permissions have been granted.
Insights into Mu’s Development
For those keen on understanding the intricacies of this technology, Microsoft has shared valuable insights into the development of Mu. The model was crafted with a keen awareness of the limitations posed by NPUs, particularly concerning memory constraints. This strategic approach ensures that Mu operates efficiently, utilizing exclusively NPU-optimized operations to maximize performance.
Vivek Pradeep, Microsoft Corporate Vice President and Distinguished Engineer for Windows Applied Sciences, elaborated on the model’s design in a recent blog post. He emphasized the importance of creating an AI-powered agent that comprehends natural language and can seamlessly modify relevant settings within the app. “Our goal was to create an AI-powered agent within Settings that understands natural language and changes relevant undoable settings seamlessly,” Pradeep stated. He highlighted the integration of this agent into the existing search box, which necessitated ultra-low latency to accommodate a wide range of settings.
In pursuit of these goals, Microsoft developers explored various models but ultimately required a smaller solution to meet their latency targets. The Mu model was meticulously fine-tuned to achieve response times of under 500 milliseconds, aligning perfectly with the company’s vision for a responsive and reliable agent capable of scaling across hundreds of settings.
Currently, the AI agents powered by the Mu model are undergoing testing among Windows Insiders, providing an opportunity for early adopters to experience this cutting-edge technology. Additionally, those participating in the Windows Insider program can also explore a significant update to the Recall app, further enhancing their interaction with Windows 11.