Microsoft is venturing into uncharted territory by inviting Windows users to grant its Copilot feature unprecedented access to their files and activities. This request comes at a time when the company is acutely aware of the scrutiny surrounding its AI initiatives. The last time Microsoft allowed an AI feature extensive access to user data, it faced significant backlash, prompting a swift retraction.
What Copilot can now reach on a Windows PC
During the recent Windows and Surface event, Microsoft unveiled a new iteration of Copilot, which it describes as having hybrid intelligence on Copilot+ PCs. With user consent, Copilot can now comprehend relevant content on the device, including files and recent activities, and perform various tasks across Windows. These tasks range from organizing files and assessing device diagnostics to troubleshooting issues, writing code, and executing workflows. While some operations are handled by local AI models, more complex tasks are processed in the cloud. Pavan Davuluri, Microsoft’s executive vice president for Windows and devices, emphasized that Copilot “understands your PC and takes action with your permission.”
Additionally, Autopilot, characterized by Microsoft as persistent, proactive, and personal, shares this local access. In a demonstration, Copilot chief Jacob Andreou illustrated its utility during tax season, showcasing how it could assist in gathering necessary documents from an accountant. The taskbar search is also evolving, transforming into a command line capable of executing numerous system actions, from enabling dark mode to sending text messages.
Microsoft’s own security team already wrote the warning
Microsoft has implemented several safeguards to mitigate risks associated with this new functionality. Activation is opt-in, folder access is limited, and Copilot’s agents operate under separate accounts from the user. The introduction of Microsoft Execution Containers, which became generally available with Windows 11, allows organizations to define the extent of an agent’s access to files and networks. Microsoft asserts that these policies are beyond the agent’s control, ensuring that it cannot autonomously escalate its permissions.
However, the internal warnings from Microsoft’s own security team highlight the potential risks. Dana Huang and Logan Iyer, corporate vice presidents for Windows security and platform, cautioned that agents capable of “reading files, invoking services, modifying environments, and chaining operations” introduce “new risk to control and trust,” particularly as large language models (LLMs) evolve. This acknowledgment underscores the importance of containment measures in the deployment of Copilot.
Some features intended to enhance auditability of agent activities remain under development. Microsoft Entra is expected to soon differentiate between agent and user activities, while Intune management for process containers is also forthcoming. A note in the announcement indicates that “governance at scale may require additional services,” suggesting a gap between policy implementation and practical execution for IT departments.
The Recall precedent Microsoft cannot shake
This is not Microsoft’s first experience with a controversial feature. The company previously faced criticism with the Recall feature, which captured screen content but was found to store data in an unencrypted local database. Following security concerns raised by researcher Alex Hagenah, Microsoft disabled Recall by default, added encryption, and shifted its rollout to the Windows Insider program, delaying the preview further.
The backlash continued when Davuluri described Windows as “evolving into an agentic OS,” which sparked over 400 largely negative responses, leading him to disable comments. His acknowledgment of the need for improvement reflects the ongoing challenges Microsoft faces in this domain.
Why opt-in puts the risk on Microsoft
The opt-in design choice is both a strategic move and a source of accountability for Microsoft. Since no features activate automatically, the onus is on users to grant access, which may reflect their confidence in Microsoft’s track record more than the functionality itself. Concerns linger from the Recall incident, as highlighted by Cryptopolitan, emphasizing that merely adopting a container system does not inherently enhance safety.
The hybrid features necessitate a Copilot+ PC and are set to roll out in the coming months, with variations in timing based on device, market, and chip specifications. Taskbar search actions began reaching Windows Insiders on October 7, and an opt-in link connecting the new Copilot with taskbar search is anticipated in select markets later this year. Microsoft reports that over 40 percent of newly manufactured business laptops are Copilot+ PCs, with the Surface Laptop Ultra, priced from ,599, launching on October 16.
While Microsoft has established a sandbox to contain agent activities, the true test will be how it manages the inevitable challenges that arise. Should a Copilot agent mismanage a file, the repercussions will likely extend beyond the model itself, landing squarely on the Windows platform.
Copilot Gets Control of Windows and Your Files as Microsoft’s Recall Bill Comes Due
Microsoft is venturing into uncharted territory by inviting Windows users to grant its Copilot feature unprecedented access to their files and activities. This request comes at a time when the company is acutely aware of the scrutiny surrounding its AI initiatives. The last time Microsoft allowed an AI feature extensive access to user data, it faced significant backlash, prompting a swift retraction.
What Copilot can now reach on a Windows PC
During the recent Windows and Surface event, Microsoft unveiled a new iteration of Copilot, which it describes as having hybrid intelligence on Copilot+ PCs. With user consent, Copilot can now comprehend relevant content on the device, including files and recent activities, and perform various tasks across Windows. These tasks range from organizing files and assessing device diagnostics to troubleshooting issues, writing code, and executing workflows. While some operations are handled by local AI models, more complex tasks are processed in the cloud. Pavan Davuluri, Microsoft’s executive vice president for Windows and devices, emphasized that Copilot “understands your PC and takes action with your permission.”
Additionally, Autopilot, characterized by Microsoft as persistent, proactive, and personal, shares this local access. In a demonstration, Copilot chief Jacob Andreou illustrated its utility during tax season, showcasing how it could assist in gathering necessary documents from an accountant. The taskbar search is also evolving, transforming into a command line capable of executing numerous system actions, from enabling dark mode to sending text messages.
Microsoft’s own security team already wrote the warning
Microsoft has implemented several safeguards to mitigate risks associated with this new functionality. Activation is opt-in, folder access is limited, and Copilot’s agents operate under separate accounts from the user. The introduction of Microsoft Execution Containers, which became generally available with Windows 11, allows organizations to define the extent of an agent’s access to files and networks. Microsoft asserts that these policies are beyond the agent’s control, ensuring that it cannot autonomously escalate its permissions.
However, the internal warnings from Microsoft’s own security team highlight the potential risks. Dana Huang and Logan Iyer, corporate vice presidents for Windows security and platform, cautioned that agents capable of “reading files, invoking services, modifying environments, and chaining operations” introduce “new risk to control and trust,” particularly as large language models (LLMs) evolve. This acknowledgment underscores the importance of containment measures in the deployment of Copilot.
Some features intended to enhance auditability of agent activities remain under development. Microsoft Entra is expected to soon differentiate between agent and user activities, while Intune management for process containers is also forthcoming. A note in the announcement indicates that “governance at scale may require additional services,” suggesting a gap between policy implementation and practical execution for IT departments.
The Recall precedent Microsoft cannot shake
This is not Microsoft’s first experience with a controversial feature. The company previously faced criticism with the Recall feature, which captured screen content but was found to store data in an unencrypted local database. Following security concerns raised by researcher Alex Hagenah, Microsoft disabled Recall by default, added encryption, and shifted its rollout to the Windows Insider program, delaying the preview further.
The backlash continued when Davuluri described Windows as “evolving into an agentic OS,” which sparked over 400 largely negative responses, leading him to disable comments. His acknowledgment of the need for improvement reflects the ongoing challenges Microsoft faces in this domain.
Why opt-in puts the risk on Microsoft
The opt-in design choice is both a strategic move and a source of accountability for Microsoft. Since no features activate automatically, the onus is on users to grant access, which may reflect their confidence in Microsoft’s track record more than the functionality itself. Concerns linger from the Recall incident, as highlighted by Cryptopolitan, emphasizing that merely adopting a container system does not inherently enhance safety.
The hybrid features necessitate a Copilot+ PC and are set to roll out in the coming months, with variations in timing based on device, market, and chip specifications. Taskbar search actions began reaching Windows Insiders on October 7, and an opt-in link connecting the new Copilot with taskbar search is anticipated in select markets later this year. Microsoft reports that over 40 percent of newly manufactured business laptops are Copilot+ PCs, with the Surface Laptop Ultra, priced from ,599, launching on October 16.
While Microsoft has established a sandbox to contain agent activities, the true test will be how it manages the inevitable challenges that arise. Should a Copilot agent mismanage a file, the repercussions will likely extend beyond the model itself, landing squarely on the Windows platform.