Microsoft is venturing into uncharted territory by requesting Windows users to grant its Copilot feature unprecedented access to their files and activities. This move comes after a previous attempt to integrate AI deeply into the PC ecosystem was met with swift scrutiny, leading to a retraction of the feature. The stakes are high, and the company’s internal documents reveal a more cautious outlook than its public presentations suggest.
What Copilot can now reach on a Windows PC
At the recent Windows and Surface event, Microsoft unveiled its new Copilot, which is designed to harness what the company refers to as hybrid intelligence on Copilot+ PCs. With user consent, Copilot can analyze relevant content on the device, including files and recent activities, and perform various tasks such as organizing files, diagnosing device issues, troubleshooting, coding, and executing workflows. Some operations are handled by local AI models, while more complex tasks are processed in the cloud. Pavan Davuluri, Microsoft’s executive vice president for Windows and devices, stated that Copilot “understands your PC and takes action with your permission.”
Additionally, Autopilot, described as a persistent and proactive agent, shares similar capabilities. During the event, Copilot chief Jacob Andreou demonstrated its utility by showcasing how it could assist with tax season tasks, such as gathering necessary information from an accountant. The taskbar search function is also evolving into a command line, capable of executing thousands of system actions, from enabling dark mode to sending text messages.
Microsoft’s own security team already wrote the warning
While Microsoft has implemented several safeguards, including opt-in activation and restricted folder access, the risks associated with such powerful features have not gone unnoticed. The company’s security executives, Dana Huang and Logan Iyer, have articulated concerns regarding the potential for agents to “read files, invoke services, modify environments, and chain operations together,” which introduces “new risk to control and trust.” They also highlighted the evolving capabilities of large language models (LLMs) that could potentially escape their containment.
Some of the features intended to enhance auditability of agent activities are still in development. Microsoft Entra is expected to soon differentiate between agent and user actions, and the management of process containers through Intune is also forthcoming. However, a footnote in the announcement suggests that “governance at scale may require additional services,” indicating a gap between policy and practical implementation for IT departments.
The Recall precedent Microsoft cannot shake
This is not Microsoft’s first foray into advanced features that raised security concerns. The Recall feature, which captured screen content, was quickly criticized when security researcher Alex Hagenah discovered that its data was stored in an unencrypted local database. Following this revelation, Microsoft had to revise Recall, making it off by default, linking it to Windows Hello, and adding encryption before moving it to the Windows Insider program for further testing.
Furthermore, Davuluri’s assertion that Windows is “evolving into an agentic OS” sparked significant backlash, prompting him to disable comments on his post after receiving over 400 negative responses. His acknowledgment of the need for improvement reflects the ongoing challenges Microsoft faces in navigating user sentiment regarding these changes.
Why opt-in puts the risk on Microsoft
The opt-in nature of Copilot is designed to empower users, but it also places the onus of accountability squarely on Microsoft. Since no features are activated by default, the extent of Copilot’s capabilities hinges on users actively granting access. Each user who opts out is effectively expressing their reservations about Microsoft’s track record rather than the feature itself. Privacy concerns stemming from previous initiatives like Recall continue to loom large over this launch.
The hybrid features are set to roll out over the coming months, contingent on device specifications, market conditions, and chip availability. Taskbar search actions began reaching Windows Insiders on October 7, with an opt-in link for the new Copilot and taskbar search anticipated in select markets later this year. Notably, Microsoft reports that over 40 percent of new business laptops are now being designed as Copilot+ PCs, with the Surface Laptop Ultra, priced at ,599, scheduled for release on October 16.
While Microsoft has established a sandbox environment to mitigate risks, the true test will come when a Copilot agent inadvertently mismanages a file. In such an event, the blame is unlikely to fall on the AI model; rather, it will be directed at Windows itself.
Copilot Gets Control of Windows and Your Files as Microsoft’s Recall Bill Comes Due
Microsoft is venturing into uncharted territory by requesting Windows users to grant its Copilot feature unprecedented access to their files and activities. This move comes after a previous attempt to integrate AI deeply into the PC ecosystem was met with swift scrutiny, leading to a retraction of the feature. The stakes are high, and the company’s internal documents reveal a more cautious outlook than its public presentations suggest.
What Copilot can now reach on a Windows PC
At the recent Windows and Surface event, Microsoft unveiled its new Copilot, which is designed to harness what the company refers to as hybrid intelligence on Copilot+ PCs. With user consent, Copilot can analyze relevant content on the device, including files and recent activities, and perform various tasks such as organizing files, diagnosing device issues, troubleshooting, coding, and executing workflows. Some operations are handled by local AI models, while more complex tasks are processed in the cloud. Pavan Davuluri, Microsoft’s executive vice president for Windows and devices, stated that Copilot “understands your PC and takes action with your permission.”
Additionally, Autopilot, described as a persistent and proactive agent, shares similar capabilities. During the event, Copilot chief Jacob Andreou demonstrated its utility by showcasing how it could assist with tax season tasks, such as gathering necessary information from an accountant. The taskbar search function is also evolving into a command line, capable of executing thousands of system actions, from enabling dark mode to sending text messages.
Microsoft’s own security team already wrote the warning
While Microsoft has implemented several safeguards, including opt-in activation and restricted folder access, the risks associated with such powerful features have not gone unnoticed. The company’s security executives, Dana Huang and Logan Iyer, have articulated concerns regarding the potential for agents to “read files, invoke services, modify environments, and chain operations together,” which introduces “new risk to control and trust.” They also highlighted the evolving capabilities of large language models (LLMs) that could potentially escape their containment.
Some of the features intended to enhance auditability of agent activities are still in development. Microsoft Entra is expected to soon differentiate between agent and user actions, and the management of process containers through Intune is also forthcoming. However, a footnote in the announcement suggests that “governance at scale may require additional services,” indicating a gap between policy and practical implementation for IT departments.
The Recall precedent Microsoft cannot shake
This is not Microsoft’s first foray into advanced features that raised security concerns. The Recall feature, which captured screen content, was quickly criticized when security researcher Alex Hagenah discovered that its data was stored in an unencrypted local database. Following this revelation, Microsoft had to revise Recall, making it off by default, linking it to Windows Hello, and adding encryption before moving it to the Windows Insider program for further testing.
Furthermore, Davuluri’s assertion that Windows is “evolving into an agentic OS” sparked significant backlash, prompting him to disable comments on his post after receiving over 400 negative responses. His acknowledgment of the need for improvement reflects the ongoing challenges Microsoft faces in navigating user sentiment regarding these changes.
Why opt-in puts the risk on Microsoft
The opt-in nature of Copilot is designed to empower users, but it also places the onus of accountability squarely on Microsoft. Since no features are activated by default, the extent of Copilot’s capabilities hinges on users actively granting access. Each user who opts out is effectively expressing their reservations about Microsoft’s track record rather than the feature itself. Privacy concerns stemming from previous initiatives like Recall continue to loom large over this launch.
The hybrid features are set to roll out over the coming months, contingent on device specifications, market conditions, and chip availability. Taskbar search actions began reaching Windows Insiders on October 7, with an opt-in link for the new Copilot and taskbar search anticipated in select markets later this year. Notably, Microsoft reports that over 40 percent of new business laptops are now being designed as Copilot+ PCs, with the Surface Laptop Ultra, priced at ,599, scheduled for release on October 16.
While Microsoft has established a sandbox environment to mitigate risks, the true test will come when a Copilot agent inadvertently mismanages a file. In such an event, the blame is unlikely to fall on the AI model; rather, it will be directed at Windows itself.