Microsoft is once again integrating artificial intelligence into Windows 11, introducing a suite of features that includes the intriguing successor to its earlier Recall feature. This new addition, known as Copilot Vision, acts as a remote assistant that processes information in the background, akin to a backseat driver for your digital activities.
AI Innovations in Windows 11
According to Microsoft, “AI is changing the way we use our PCs,” a sentiment that garners mixed reactions from users. The company emphasizes that “Windows 11 is the home for AI,” claiming to offer the most comprehensive AI experiences available today on Copilot+ PCs. This includes exclusive features like Recall (in preview), Click to Do (in preview), and enhanced Windows search capabilities, all designed to transform user interaction with their devices.
Copilot Vision builds upon the controversial Recall feature, which was initially limited to Copilot+ systems equipped with a neural co-processor. Unlike Recall, which faced criticism for security vulnerabilities and was subsequently delayed, Copilot Vision captures continuous screenshots for analysis by an optical character recognition system and a large language model. However, unlike its predecessor, this data is processed on Microsoft servers rather than locally.
A Microsoft spokesperson assured users that their data would not be stored long-term, except for transcripts of conversations with the Copilot assistant, and clarified that this information would not be used for model training or advertising purposes.
The vision for Copilot Vision is to evolve into “a true companion,” providing users with a deeper understanding of their objectives and offering step-by-step guidance to achieve them. While the screen monitoring feature is only activated during a Copilot session, it aims to be more proactive than previous iterations, reminiscent of the animated assistants from Microsoft Office in the late 90s.
Currently, Copilot Vision is available exclusively in the United States, with plans to expand to select “non-European countries” in the near future. This move appears to acknowledge the European Union’s AI Act. However, users outside this region will still benefit from other AI enhancements included in the latest Windows 11 update, which remains the only mainstream version of the operating system under official support.
Agentic AI and Enhanced Features
The update introduces the first “agentic” AI within Windows 11, capable of adjusting system settings on behalf of users. Powered by a local language model named Mu, this feature is currently available on Copilot+ systems using Qualcomm Snapdragon hardware, with Intel and AMD support anticipated soon. The agent accepts natural language commands, allowing users to instruct it to perform tasks like connecting to Bluetooth devices or adjusting screen resolution.
Vivek Pradeep, a Microsoft engineer, explained that the goal was to create an AI agent that understands natural language and seamlessly modifies relevant settings. He acknowledged the challenges posed by the extensive range of Windows settings, particularly when overlapping functionalities arise. For instance, a simple command like “increase brightness” could refer to multiple monitors, necessitating a refined approach to prioritize commonly used settings.
However, Microsoft has not addressed how it plans to tackle the “hallucination” problem associated with language models, where the AI may generate responses that are not grounded in reality. This issue is particularly concerning for agentic AI, which takes direct actions based on user prompts.
Additionally, Microsoft has rolled out new actions for its “Click to Do” feature, now available in preview outside the European Economic Area. This includes tools to enhance reading skills, utilize generative AI capabilities in Microsoft Word, and activate an “Immersive Reader” mode with adjustable visuals and text-to-speech functions.
AI Enhancements Across Applications
The company’s AI initiatives extend beyond Windows 11. The Photos app now features an AI-powered “Relight” function, simulating the effect of multiple virtual light sources in photographs. The classic Paint application has also received an update, introducing a “sticker generator” that transforms text prompts into cartoonish images, along with an “object select” tool for easier element selection in complex images. Even the Snipping Tool has been upgraded with a “perfect screenshot” feature that automatically resizes the capture area based on screen content.
In a notable shift, the latest update also marks the end of the traditional Blue Screen of Death (BSoD), which has been a staple of Windows since its inception. The BSoD has been replaced with a Black Screen of Death, which retains the classic acronym while promising a more streamlined and readable error message format. Microsoft has also introduced a “quick machine recovery” system to automate fixes during widespread outages, enhancing user experience without requiring complex manual intervention.
As these features begin to roll out through a controlled feature rollout system, users eager to access the latest updates are encouraged to enable the “Get the latest updates as soon as they’re available” option in Windows Update. Those who prefer to avoid further AI integration in their operating system may find limited alternatives outside of Microsoft’s ecosystem.
Introducing the Surface Laptop 5G
In conjunction with its AI advancements, Microsoft has unveiled the Surface Laptop 5G, which supports Copilot+. This new device boasts a 13.8″ display, larger than the refreshed 13″ Surface Laptop and 12″ Surface Pro. It features a choice of Intel Core Ultra (Series 2) processors, each equipped with a 40 tera-operations per second (TOPS) neural co-processor for local AI models. An optional built-in 5G cellular modem is also available, ensuring continuous connectivity to Microsoft 365 Copilot and other cloud tools, with availability expected in 2025.
For additional details, interested parties can visit the Microsoft website.
