
Microsoft Aims for Voice-Controlled Windows with New AI Copilot
Microsoft is really betting big on AI, and the next version of Windows will be packed with it. The goal? To get you to ditch your keyboard and mouse and start bossing your PC around using your voice, just like Captain Kirk commanding the Enterprise. It's an interesting vision, but I'm not entirely convinced everyone's ready to start shouting commands at their computers.
Starting soon, Microsoft is rolling out more "experimental" features and apps centered around its Copilot AI. They've even put a dedicated Copilot key on new PCs. Once enabled, you'll be able to simply say "Hey, Copilot" to start interacting with your computer. Remember Cortana? Well, this is kind of like that, but on steroids. I can’t help but wonder, is this something users really want?
Copilot Vision, which was already available in Edge, is now expanding to the entire Windows ecosystem. This means AI image recognition is coming, allowing Copilot to understand what's on your screen. So, instead of giving super-detailed instructions, you can just say something like, "Show me how to improve Spotify audio quality," and Copilot will highlight the relevant settings. This has the potential to save you from having to look up how-to articles.
Microsoft gave a demo of Copilot Vision, and while it responded quickly to math problems and online shopping queries, it wasn't perfect. When asked to identify image resolution settings on a Shopify account, it pointed to the wrong area. It's a common issue with live demos, and it highlights the challenges of getting AI to understand context perfectly. I wonder how reliable this will be in everyday use.
This AI vision system can also describe images on your screen. Microsoft suggests using this to create a resume from your portfolio. Another example showed Copilot generating musical ideas from a hummed tune. I don't know about you, but the tune I heard wasn't exactly inspiring. AI shopping is also getting a boost, with Copilot analyzing your browser tabs to find products. Google has similar AI shopping features, with virtual try-on options.
Expect to see new Copilot commercials soon, designed to "teach" you how fun it is to use your voice with Copilot. Beyond that, there are even more features aimed at getting you hooked on AI. Windows Insiders will get beta features that integrate Copilot more deeply into the taskbar, potentially replacing the regular search bar. Don't worry, you can still use it for file and settings searches.
Microsoft claims people already talk to their PCs for dictation and accessibility purposes. However, there's a big difference between those uses and having a full-blown conversation with your computer, especially if you have coworkers nearby. For now, typing to Copilot Vision is limited to beta testers.
Beta testers will also get to try out the new Copilot Actions app, which Microsoft describes as an "AI agent" capable of performing actions across different apps and files. In the AI world, an "agent" is a collection of AI models working together. On Windows, this could mean Copilot takes over your PC to run programs and complete tasks. Another company’s AI, Claude, showed similar capabilities last year.
Microsoft has been pushing hardware-specific AI capabilities for a while, with "AI PCs" in 2023 and "Copilot+ PCs" in 2024. Now, they claim that "every Windows 11 PC" is an AI PC once connected to Microsoft's cloud-based AI. It’s not unfair to say that the AI PC “hasn’t really come alive yet.” Microsoft tried to introduce Recall, a feature that would screenshot everything you do on your PC and allow you to search through it. Security researchers discovered this could expose sensitive data, so Microsoft delayed its release for a year. Even after security improvements, Recall still isn't perfect, and some companies have blocked it over data security concerns.
The Copilot Actions app and other features are "opt-in," and the Actions app is disabled by default. Microsoft promises you can take control "at any time" and that the program will ask for permission for specific actions. Since these features are cloud-based, your data will be processed on a remote server. Microsoft assures users that it isn't storing or misusing your prompts or screen content. However, after the Recall debacle, trusting Microsoft with your data is a bit harder. If Microsoft wants people to ignore privacy concerns for the sake of AI, they need to offer a more compelling reason than an AI that hums tunes on request.
2 Images of Windows Copilot:


Source: Gizmodo