Microsoft has embarked on an exciting journey with its latest feature for Copilot on Windows 11, currently in testing. This new capability allows users to share their entire desktop with Copilot, transforming the way individuals interact with Microsoft’s AI assistant. By enabling Copilot to view and engage in conversation about what’s displayed on the screen, users can expect a more integrated and dynamic experience.
Enhanced Interaction with Copilot Vision
Earlier this year, Microsoft unveiled Copilot Vision for Windows 11, initially allowing users to share individual applications or windows. The latest testing phase expands this functionality, permitting the sharing of the entire desktop. This means that Copilot can now observe multiple applications simultaneously, enhancing its ability to assist users.
In a recent blog post, Microsoft highlighted the benefits of this feature: “It can help analyze content, provide insights, and answer your questions, coaching you through it aloud. Get tips on making improvements to your creative project, help with improving your resume, or guidance while navigating a new game.”
(Image credit: Microsoft)
Copilot Vision transforms the Copilot app into a genuinely collaborative tool, allowing it to work alongside users in real-time. By utilizing natural language, users can converse with Copilot as it observes their activities, facilitating discussions on various topics while seamlessly switching between applications.
While Copilot currently does not have the capability to directly manipulate the desktop, it can guide users by highlighting areas of the screen that require attention when assistance is requested. If users encounter challenges with specific applications, Copilot Vision is poised to provide step-by-step guidance, making tasks more manageable.
In addition to this innovative feature, Microsoft has announced several other enhancements for Copilot this year, including Copilot Memory. This feature enables Copilot to build a personalized profile, learning users’ habits, preferences, and dislikes across devices.
Looking ahead, Microsoft has teased the introduction of a customizable persona for Copilot, allowing users to create a character for interaction. The prospect of bringing back a nostalgic figure like Clippy adds a playful twist to the user experience.
Last year, Microsoft significantly updated the Copilot app, introducing a redesigned user interface and a native Windows 11 application, moving away from a web-based model. Since then, the company has been gradually rolling out new features, including Copilot Pages and deep research capabilities. For users with Copilot+ PCs, the Copilot key will provide quick access to features like Copilot Vision with a simple press.
Currently, the desktop sharing feature is being tested across all Windows 11 Insider Preview channels, including Release Preview, Beta, Dev, and Canary Channels. Users can acquire the update through the Microsoft Store, although it will likely be released in waves, meaning not everyone will see it immediately.
It is important to note that Copilot Vision is currently available only in the United States, and the ability to share your desktop is similarly restricted by region. As such, users in markets without access to Copilot Vision will not be able to participate in this testing phase at this time.
