Click To Do lets you feed text to AI models
Click To Do harnesses the power of optical character recognition (OCR) technology to transform your screen into an interactive canvas. By scanning your current display, it allows you to select text as if it were tangible, effectively turning a screenshot into a dynamic tool for engagement.
For instance, selecting an email address prompts the Click To Do menu to offer the option to “Send email,” facilitating immediate correspondence. Similarly, highlighting a website URL provides the choice to “Open website,” launching it effortlessly in your preferred web browser—thankfully, not limited to Microsoft Edge.
When you select more than ten words, the experience becomes even more intriguing. Users are presented with a suite of actions powered by the Phi Silica language model, which operates on the NPU of your Copilot+ PC. This enables functionalities such as summarizing text, generating bulleted lists, or even rewriting content in various tones.
This innovative approach marks one of Microsoft’s initial ventures into integrating NPU-driven text actions on Copilot+ PCs. However, due to the reliance on screenshots, there is a limitation on the volume of text that can be processed at once.
This design choice aims to enhance user experience, particularly when considering that local language models running on Windows PCs often lack the robustness of cloud-based large language models like ChatGPT. Users may find themselves questioning the utility of local applications when more powerful alternatives are readily available online, highlighting a common challenge faced by many Copilot+ PC AI features.
Additionally, the platform includes an “Ask Copilot” feature, which allows users to send selected text directly to Microsoft’s Copilot AI chatbot. There’s also a “Draft with Copilot in Word” option, enabling users to initiate a Word document with the assistance of Microsoft’s AI.
While it is possible to relay information to a cloud-based chatbot, this functionality is exclusive to users of Microsoft’s Copilot for home use or Microsoft 365 Copilot for business applications, adding a layer of complexity to the overall experience.
