We all know that Windows Copilot aims to be the biggest revolution within a Microsoft operating system since we switched from commands to interface. And for those who are not familiar with Windows Copilot, here’s a quick guide.
Microsoft came out with its announcements of Windows Copilot 365 and Copilot for Windows in 2023 and presented Copilot as a general virtual assistant to help you with your entire digital life.
So far, we haven’t seen Copilot reach its full potential, but it seems like we’re one step closer: Microsoft is preparing to add a new powerful feature that will allow users to take a screenshot, send it to Copilot, and ask it to explain what’s in it.
What do we know about GPT-4 and Copilot
From what we know, the “add a screenshot” button is being rolled out to the general public, which means you may already be able to try it. If not, you should be able to do so very soon.
This button should appear in the Copilot panel, asking you to select a part of the screen, confirming that you have captured what you want to comment with Copilot, and then upload it to the Copilot panel or the right side of Bing.
Once the screenshot is loaded, you can talk about it and ask about anything it contains or is related to through the Bing chat or Copilot in Windows 11.
As you can see in the screenshot, the new button is located next to the existing image upload button. You can try the new feature on the Microsoft Copilot website or in Bing Chat.
The media Windows Latest has its own demonstration of this new feature, and in Tech Radar too. First, they made a screen selection that showed a complete description of the YouTube channel of the video creation platform HeyGen and asked it to say two things: who owns the channel and what it is about.
Copilo returned this: “This is the YouTube channel of HeyGen, a state-of-the-art video creation platform that turns text into professional spokesperson videos in a matter of minutes. They offer premium avatars that speak in multiple languages and professional video templates for various use cases such as marketing, e-learning, and corporate communication. You can find more information about HeyGen on their website”.
This is quite accurate, and it has reminded me of a particular feature that I really like about Bing Chat and Copilot: they provide, in an easy and very visible way, sources and websites that you can visit to verify the information.
Once the screen is selected, you can make marks on it and draw. You can also add specific visual instructions to help Copilot understand your query, and you can move the selection window to another part of the screen.
According to Windows Latest, Bing’s chat has recently received an upgrade called ChatGPT-4 that gives it a new level of functionality, and it is likely that this will also come to Copilot.
Microsoft makes it clear that Copilot is the future
In a significant development, Microsoft plans to add a physical Copilot button to the hardware of newly manufactured products starting in 2024.
Microsoft does this in its ongoing effort to make computing, especially AI-powered computing, simpler and more seamless for users. Yusuf Mehdi, Executive Vice President and Chief Marketing Officer of Microsoft, detailed and confirmed this in a recent blog post on the Windows Experience.
According to Microsoft itself, the introduction of the Copilot key will be the most notable update to the Windows keyboard in almost thirty years. They compare this future introduction to the addition of the Windows Start key, which makes it clear how important this direction is for Microsoft.
