Introducing Copilot Labs and Copilot Vision | Microsoft Copilot Blog

AI technology continues to evolve at an unprecedented pace, marking one of the fastest periods of change in history. As we engage with users and gather insights from their experiences, it becomes increasingly clear just how transformative this revolution can be. However, it also highlights the necessity for careful and deliberate progress.

Our goal is to fully harness the capabilities of AI Copilots, delivering rich and meaningful experiences while prioritizing safety and responsibility. To achieve this balance, we are excited to introduce Copilot Labs, a platform designed to trial our most advanced tools with a select group of users before a broader rollout. This initiative allows us to gather feedback, learn, and refine our offerings, ultimately enhancing both their effectiveness and safety.

Introducing Think Deeper

The first feature available through Copilot Labs is Think Deeper, which empowers Copilot to tackle more complex problems. Utilizing cutting-edge reasoning models, Think Deeper can assist with a range of tasks, from solving intricate math problems to evaluating the costs associated with home projects. This feature may take a bit longer to respond, but it aims to provide detailed, step-by-step solutions to challenging inquiries. Starting today, Think Deeper will be rolled out to a limited number of Copilot Pro users in Australia, Canada, New Zealand, the United Kingdom, and the United States.

Next on the Horizon: Copilot Vision

Another significant limitation of the Copilot experience has been its inability to comprehend the context of users’ actions. While language serves as a powerful communication tool, much of the context surrounding any task extends beyond verbal expression. To address this, we are piloting Copilot Vision within Copilot Labs, allowing users to grant Copilot the ability to see what they see.

When integrated into Microsoft Edge, Copilot Vision can interpret the content of the page you are viewing. It can answer questions, suggest next steps, and assist with various tasks—all through natural language interaction. This innovative feature elevates the practical utility of Copilot to new heights.

In developing Copilot Vision, we have prioritized the interests of users and creators. This feature is entirely opt-in, activated only when users find it beneficial. You maintain complete control over when, how, and if you wish to utilize it. Importantly, none of the content—be it audio, images, text, or conversations with Copilot—will be stored or used for training purposes. Once the feature is closed, all data is permanently discarded.

To ensure a responsible rollout, we are currently restricting the service from engaging with paywalls and sensitive content. Initially, Copilot Vision will only operate on a pre-approved list of popular websites, respecting their machine-readable controls on AI. As we move forward, we aim to expand this list while maintaining a strong commitment to safety and responsibility. Additionally, Copilot Vision is designed to drive traffic to websites without directly interacting with the web; it will provide answers rather than take actions on behalf of users.

For those interested in more details, an in-depth Q&A is available, outlining the safeguards integrated into Copilot Vision and the broader Copilot Labs initiative. Our focus remains on balancing utility with responsibility, and we are eager to hear your thoughts on this experimental feature. Initially, access will be limited to a select group of Pro users in the United States, and we hope you find it beneficial as we continue to refine our offerings.

Winsage