EU plan to force messaging apps to scan for CSAM risks millions of false positives, experts warn | TechCrunch

Experts Raise Alarms Over EU’s CSAM-Scanning Proposal

In a significant challenge to proposed European Union legislation, a cadre of security and privacy experts has voiced their concerns regarding the legal mandate that would compel messaging platforms to scrutinize users’ private communications for child sexual abuse material (CSAM). The open letter, published on Thursday, cautions that this could lead to a deluge of false positives, potentially numbering in the millions daily.

The debate over the EU’s initiative has intensified since its introduction two years prior, garnering criticism from independent experts, members of the European Parliament, and even the EU’s Data Protection Supervisor. These parties have warned of the overreach and potential ineffectiveness of the proposed measures, which would obligate platforms not only to detect known CSAM but also to employ vague and untested scanning technologies to uncover unrecognized CSAM and identify instances of grooming in real-time.

This legislative push, according to detractors, seeks the unfeasible and threatens to disrupt internet security and privacy. They argue that the implementation of sweeping surveillance and experimental technologies, such as client-side scanning, could do more harm than good. In spite of these concerns, the EU appears to be moving forward with the plan.

The expert signatories of the latest letter, which include luminaries such as Harvard’s Bruce Schneier and Johns Hopkins University’s Dr. Matthew D. Green, as well as tech researchers from IBM, Intel, and Microsoft, assert that recent amendments from the European Council do not rectify the foundational issues with the proposal.

Previously, in July, a separate letter signed by 465 academics had cautioned that the detection technologies integral to the legislation are “deeply flawed and vulnerable to attacks,” potentially eroding the protections provided by end-to-end encrypted (E2EE) communications.

Scrutinizing Council Amendments and MEP Counterproposals

Efforts by MEPs in the European Parliament last fall to present a revised approach, which suggested more targeted scanning and protection of E2EE, have yet to resonate with the European Council, which plays a pivotal role in the EU legislative process. The Council’s position will significantly shape the final form of the law.

The Belgian Council presidency’s March amendment, leading discussions for EU Member States, is critiqued in the open letter for its failure to address the deep-seated issues of the Commission’s approach. The experts contend that even with proposed tweaks—such as risk categorization, protection of cybersecurity and encryption, and vetting of detection technologies—the proposal remains a precursor to a security and privacy catastrophe.

Moreover, the experts dissect the Council’s strategy to mitigate false positives, arguing that the approach will still lead to an overwhelming volume of unwarranted alarms.

The Prospect of Massive False Positives with Messaging Platforms

The experts caution that the proposed approach could result in an inordinate number of false positives. They illustrate this by hypothesizing the performance of a hypothetical CSAM and grooming detector with a 0.1% false positive rate, applied to a platform like WhatsApp—which processes 140 billion messages daily. Even with conservative estimates, this scenario could yield 1.4 million false positives every day, a figure that becomes unmanageable when considering other messaging services.

Another Council suggestion to limit detection orders to “high-risk” messaging apps is viewed as inconsequential by the signatories. They argue that this categorization would still adversely affect a vast array of services, particularly as the use of E2EE is on the rise.

Encryption at the Heart of the Debate

The letter reiterates experts’ longstanding assertions regarding the incompatibility of detection measures with the principles of E2EE. They argue that any attempt to introduce detection capabilities inherently compromises the confidentiality that E2EE is designed to provide.

Concurrently, European police chiefs have called for platforms to accommodate lawful access to identify illegal activities while preserving encryption—a statement seen as a bid to influence legislative action on the CSAM-scanning regulation.

Should the EU persist with its current trajectory, experts warn of dire repercussions, suggesting that it would set a dangerous precedent for internet filtering and significantly impact digital privacy rights. They also foresee broader global implications for democratic societies.

While an EU source did not provide detailed insights into the Member States’ ongoing discussions, they confirmed that a working party meeting scheduled for May 8 will revisit the contentious proposal to combat child sexual abuse.

AppWizard
EU plan to force messaging apps to scan for CSAM risks millions of false positives, experts warn | TechCrunch