NPU technology

In 2024, Microsoft introduced the "Copilot+ PC" branding for AI-capable laptops, while Apple launched Apple Intelligence. These developments have led to mixed outcomes, with features like real-time translations and on-device speech-to-text being beneficial, but others, such as Windows Recall, still proving their value. By 2025, mainstream developers are expected to integrate on-device AI into Windows applications, influencing consumer purchasing decisions. The term "TOPS" (Trillions of Operations Per Second) is becoming important for evaluating the AI performance of Windows laptops, with a minimum of 40 TOPS required for Microsoft's "Copilot PC+" designation. Qualcomm's Copilot+ PCs reported around 45 TOPS, significantly higher than Intel's 11 TOPS. By the end of 2024, premium Windows laptops are expected to see a three- to four-fold increase in NPU performance compared to 2023 models. Analysts speculate further performance improvements may occur towards the end of 2025. Despite the potential for a two- to three-fold enhancement in on-device AI performance, experts caution against overemphasizing TOPS figures, which may not accurately reflect real-world performance. The lack of a unified API for leveraging NPU capabilities in Windows complicates matters for users of Copilot+ laptops without Qualcomm chips. Although AMD and Intel have released competitive chips, Qualcomm currently holds an advantage with exclusive support for certain applications. Microsoft is promoting its low-level machine learning API (DirectML) and the Windows Copilot Runtime, which may enhance the Copilot+ PC ecosystem. While cloud-based AI solutions remain an option, the cost of these services is expected to rise, making on-device AI more appealing. The introduction of ChatGPT Pro highlights the financial implications of cloud access compared to on-device NPU usage, which incurs no additional costs. The pace of on-device AI adoption in Windows' software ecosystem is anticipated to accelerate in 2025.