Don’t trust Windows 11’s Recall feature? A new report will only harden your resolve to never use the AI-powered search

Windows 11’s Recall feature is once again under the microscope following a recent analysis that highlights both its advancements and shortcomings in handling sensitive information. This AI-driven tool, designed to enhance user experience through natural language search capabilities, relies on regularly captured screenshots of user activity on Copilot+ enabled PCs. Despite its impressive functionality, Recall has faced ongoing scrutiny regarding privacy and security since its initial introduction.

According to a report by Avram Piltch from The Register, while Recall has improved since its preview launch, it still falters in critical areas. For instance, during testing on a Lenovo Yoga Slim 7x, the feature occasionally failed to exclude sensitive financial information, such as credit card numbers, from its screenshots. In one test scenario, Recall captured a screenshot of a mock checkout page devoid of identifying labels, allowing sensitive data to slip through its filter. When the labels were present, however, Recall effectively blocked the information, raising concerns about its consistency across various online shopping platforms.

Moreover, Piltch noted that Recall also captured a text file containing fictitious usernames and passwords. The feature successfully avoided screenshots when the word “password” was included, but it faltered when that term was absent, illustrating a gap in its ability to recognize sensitive content without explicit markers. This inconsistency raises questions about the reliability of Recall’s filtering capabilities, especially given the diverse formats of online forms.

In another instance, while reviewing his online bank account, Piltch observed that Recall took screenshots revealing his account balance and deposit history, which could pose a risk if accessed by malicious actors. Fortunately, the feature did manage to obscure the account number and routing information. However, during PayPal usage, Recall captured the username on the login page, a lapse that could compromise user security.

Analysis: Better – but still not good enough

The analysis reveals that Recall’s shortcomings primarily stem from its inability to recognize sensitive information when it lacks clear labeling or is only partially visible. This raises the question of how stringent we should be in our evaluation of the feature. If I were a user of Recall, I would certainly feel disappointed by its failure to adequately protect critical data like credit card numbers and passport images. Given the nature of sensitive information, one would expect a more robust filtering mechanism capable of identifying and blocking such data, even in less-than-ideal conditions.

Conversely, the issue of capturing a file filled with passwords may not be as concerning, as the absence of a clear pattern makes it less predictable. Nonetheless, the frequency of lapses in filtering raises valid concerns about the overall effectiveness of Recall. Currently, the feature remains in preview mode, and Microsoft acknowledges that it may overlook sensitive information, encouraging users to provide feedback for improvement.

While Recall is undoubtedly evolving, the current state of its sensitive data filter leaves much to be desired. Piltch’s earlier tests indicated that the feature performed significantly worse upon its initial release, yet the persisting issues suggest that it is not yet ready for widespread adoption. The potential for future enhancements remains, but doubts linger regarding Recall’s ability to fully safeguard sensitive information against inadvertent exposure.

It’s important to note that users are not required to enable Recall, as it is turned off by default on Copilot+ PCs. Additionally, while accessing the screenshots would necessitate physical or remote access to the user’s device, the possibility of an in-person attack remains a legitimate concern, particularly if an individual is familiar with the user’s Windows Hello PIN. This reality underscores the need for a more reliable filtering system to ensure user security in all scenarios.

Winsage
Don't trust Windows 11's Recall feature? A new report will only harden your resolve to never use the AI-powered search