3 AI features coming to Copilot+ PCs that I wish were on my MacBook
Microsoft kicked off Build with a Surface event today, which launched a new computing category called Copilot+ PC. This new line of artificial intelligence PCs will give users an elevated AI experience in their everyday workflow, with new functionalities to help users quickly generate content, get live captions and translations, and find anything they've seen on their PC with their voice.
Also: Every Copilot+ PC Microsoft just announced to take on Apple's M3 MacBooks
These features will be available to Copilot+ PC users with the press of a button... literally. Microsoft's Copilot+ PCs, much like every new Windows system released over the past few months, will each have a dedicated Copilot button on the keyboard.
1. GPT-4o coming to Copilot on Copilot+ PCs
Microsoft announced that GPT-4o, OpenAI's latest AI model, will soon come to Copilot. GPT-4o's text, visual, and audio capabilities will be available through Copilot on the latest PCs. Microsoft showcased an example of a Minecraft player asking Copilot how to make a sword, with the bot able to see what was on the user's screen as his character moved around.
Also: I went hands-on with Lenovo's new Snapdragon X Elite laptops, and they're scary thin
"I see that crafting table in front of you, which is great for crafting a sword. You'll need some materials. Can you open your inventory? Just press E on your keyboard," Copilot explained to the user. The bot then explained the resources needed to craft a sword, how to escape a zombie in the game, and how to create a shelter quickly. Of course, the use cases go beyond gaming, but it's an example of how the multimodal AI model works with the improved hardware of Copilot+ PCs.
2. 'Recall' makes searching much more intuitive
Recall is a feature that records your screen and uses AI to index any information so you can easily search for it later. Instead of entering a keyword into a search bar to find an email, file, or website, users can just ask Recall to "find that purple heart necklace I saw online last week" with their voice and have Recall intelligently pull up the website from their browsing history. It's creepy but magically useful.
Also: Windows' new Recall feature lets you ask AI for that file or email you can't find
This feature saves the captured content on-device, so users' screen recordings will not be sent to the cloud at any point. Users will also be able to adjust their settings to opt out of capturing certain things and edit, view, and delete recordings. Microsoft says it will not use the captured data to train AI models.
3. A new Copilot experience
Besides using the dedicated Copilot key, the new AI assistant experience has changed with the addition of drag-and-drop. Now, users can drag and drop items into the Copilot window and have the AI tell them more about them. For example, if you see a historical photo online, you can drag and drop it onto the Copilot window and tell the bot to give you more context of what was happening in that photo and who's in it.
Also: Want to avoid Google Search AI overviews? Here are your options
The Copilot AI companion will also get a more flexible app experience, with resizeable windows that let users snap it to the side or maximize to view in full screen.
Other AI features in Copilot+ PCs
- Cocreator in Paint: Allows users to generate and refine images in near real-time with five different machine learning (ML) models running locally in tandem in the NPU. The Cocreator feature also has a creativity slider for different artistic outputs.
- Live captions with live translation: This feature provides consistent English captions in real-time for any audio that passes through your Copilot+ PCs across all applications and video platforms, including Microsoft Teams, Zoom, Chrome, Netflix, etc. It supports over 40 languages and works on-device, even while offline.
- Photos app gets restyled: The Photos app is getting a new Restyle image option to easily transform images and videos with AI, including removing the background on videos, designing a new background or foreground, generating new images, or using preset filters, all on-device.
- Windows Studio Effects: The New Copilot+ PCs will include an AI-powered enhancement feature for visual and audio quality during video calls and recordings. Some new functionalities include automatic framing, portrait blur, voice focus, and creative filters.