Microsoft (MSFT) is making major updates to its Copilot artificial intelligence platform for Windows, expanding the software’s capabilities with a number of new features and introducing the ability to speak directly to the AI assistant.
The announcements, which the company made Tuesday during a press event at its Microsoft Experience Center in New York, also include a pop-up Click to Do option that provides you with tools and actions to complete tasks related to what you see on your screen. , an updated version of Windows Search, the ability to upgrade your old photos up to 8K resolution, and an AI-powered editing tool to erase or add objects to your images.
The company also said it was working on a feature called Copilot Vision, which can view your Edge browser screen and understand questions you ask about what you’re looking at.
All of these options come as Microsoft continues its AI efforts on both the consumer and enterprise sides. The company launched its first wave of Copilot+ PCs in May, including its own line of Surface laptops and devices from partner companies including Dell, HP and Lenovo.
Microsoft shares lost more than 1% on the day, in line with Tuesday’s general market trend.
In addition to its announcements on Tuesday, Microsoft also announced that it was preparing to roll out its previous Delayed reminder feature for Windows 11. Recall is designed to capture screenshots of what you do on your computer, providing you with a kind of continually updated visual library of the apps you’ve used and websites you’ve visited.
Security researchers, however, have opposed the app, saying hackers could exploit it to steal user information. Microsoft has since updated the offering, saying it has improved its overall security and made it an opt-in service.
The new Click to Do feature gives you a set of contextual options to complete tasks that match what you’re looking at on your screen. Press the Windows key, then click the mouse and you’ll get a pop-up toolbox that lets you do things like remove or blur the background of a photo or perform a visual search using Bing . Microsoft says the feature will also work with text, helping you rewrite or summarize what you’ve written.
The improved Windows Search feature now lets you search for files and photos using descriptions of what they might include rather than their actual names. So if you’re looking for a file about an article you wrote about new Microsoft features, you can search using something like “new Microsoft updates” and the search should pull up the correct file.
Microsoft’s improved photo editing and scaling will help you remove unwanted objects from your photos, while Super Resolution in Photos will improve the look of your old images, so you can recover the old photos you have taken on your first generation iPhone and make them. it looks like you photographed them on the latest iPhone 16.
Then there’s Copilot Vision, which Microsoft says will only be available to a select set of Pro subscribers through Copilot Labs for now. In a pre-recorded demo, Microsoft demonstrated how a user could talk to Copilot about shows they might want to watch while browsing the Rotten Tomatoes web page. The AI is able to look at the screen and make suggestions about what it sees, telling the user why they might enjoy certain TV shows or movies.
Microsoft says that nothing is saved in your conversations with Copilot Vision after the session ends and that the company will not train its AI using your chats.
All of these updates come ahead of the all-important holiday shopping season. The addition of new AI features as PC sales begin to take off again should help consumers gravitate toward Microsoft Copilot+ PCs. But that of Apple (AAPL) Apple Intelligence is also expected to land on its Mac line of computers in the coming months, setting up an AI showdown for the company’s laptop and desktop businesses.
Email Daniel Howley at dhowley@yahoofinance.com. Follow him on Twitter at @DanielHowley.
Read the latest financial and business news from Yahoo Finance.