What exactly is an “AI PC”? What makes it better than a non-AI PC? Is this a product that consumers are asking for?
Exactly… It’s just marketing hype garbage
Like the article states: it contains an NPU. It’s also not targeted at consumers.
I suppose a better way to phrase it is- why is an NPU necessary? What does it enable these machines to do that a Surface sans NPU can’t?
And yes, these are business-oriented. But my question remains the same - is built-in AI a feature that businesses, as consumers of this product, are asking for? And presumably this is just the beginning, and future personal devices from Microsoft will have NPUs too. I haven’t heard any clamoring for that, but I could very well also just not be noticing the people that are
MS is making a strong push in AI business, e.g. I was told their salespeople are highly incentivized to sell copilot products, and not much else. At a big company that is strategic partner & key account (etc.) for MS, we are looking at MS Dynamics implementation for CRM, and they barely answered our emails, while on a copilot related project they are throwing people at us.
They are betting they can make a lot of money from it if they capture and monetize the corporate AI market early (whatever that might actually be), the demand / customer need is definitely not high currently.
I suppose a better way to phrase it is- why is an NPU necessary? What does it enable these machines to do that a Surface sans NPU can’t?
It can basically handle neural network/AI tasks more efficiently than a regular CPU/GPU can.
And yes, these are business-oriented. But my question remains the same - is built-in AI a feature that businesses, as consumers of this product, are asking for?
Yes, deserved or not, AI is currently on everyone’s mind in the business world. Working as a software dev, every client these days asks if we “do AI”, so we pretty much have to reluctantly learn and use it. And many of those clients are very protective of their data and don’t just want to put them on some web service, like OpenAI. So there’s certainly demand for locally running AI tasks.
It can run leela chess zero faster, for example (if it’s implemented)
It means it’s only one generation behind Apple in ML performance instead of two or three.
Serious answer: it means it has intel’s latest generation of laptop chips with better ML acceleration, and — better sit down for this cuz it’ll blow your mind — a Copilot key on the keyboard, which nobody outside of Microsoft’s branding department ever asked for.
I’ll be interested to see the benchmarks. Intel should be tripping over themselves to catch up.
The “AI PC” specification requires a minimum of 40TOPs of AI compute which is over double the 18TOPs in the current M3s. Direct comparison doesn’t really work though.
What really matters is how it’s made available for development. The Neural engine is basically a black box. It can’t be incorporated into any low level projects because it’s only made available through a high-level swift api. Intel by comparison seems to be targeting pytorch acceleration with their libraries.
a Copilot key on the keyboard, which nobody outside of Microsoft’s branding department ever asked for.
I wouldn’t have had room for a Super or Hyper modifier key in Linux without Microsoft getting the Windows key added to keyboards, so…
What exactly is the use-case for a laptop or a tablet with ML acceleration? I can see the need in embedded devices; a self-driving car is a good example. Large-scale AI services are going to run in a datacenter. Who exactly is the target consumer for a laptop or tablet with an NPU?
A person who wants to filter out noise from calls but doesn’t have an Nvidia GPU
A person who wants to translate things, but doesn’t want to send work documents to the internet, etc.
A person who wants to translate things, but doesn’t want to send work documents to the internet, etc.
I’m sure Microsoft will respect their privacy…
Who said anything about using Microsoft?
The NPU is inside the machine whether you use windows or Linux
It’s the Microsoft surface. And those NPU optimised networks from Microsoft aren’t available for Linux (yet)
I have no interest in the networks themselves, I want the NPU to use with my programs
With what drivers
You’re basically completely wrong about how AI is going to scale. We’re not going to be stuck on, say tinkertoy models on our phones and gigantic mega-models exclusively in the cloud. That’s insane. We have good language models that will run on an ordinary laptop already. You can scale models to more or less any size and there is amazing research coming out constantly regarding how to do AI more efficiently. People are running tons of ML code on their PC’s already and the demand will only go up as companies – like Microsoft – bundle more and more features that rely on AI code into their software, more SDK’s and libraries come out that support it, etc.
Also, the feasibility of deploying AI code to more users goes up the more users have them in their devices.
Also, the main trick of NPU’s is efficient matrix math, especially the use-case of applying a single operation to entire matrices at once, which AIUI is foundational to tensor math. Plain old CPU’s are trash at this because they have to iterate over each individual entity in the matrix and apply the operation separately. NPU’s, as I guess they’re coming to be called, are designed to do those operations massively in parallel. There are likely tons of applications for this beyond just ML code that we haven’t even imagined yet.
It’s a bit like asking in 1995 what the use case for a graphics card is when you can go to an arcade and gameboys exist. At that exact moment in time, based on the exact cards that were available in literally 1995, it might have been hard to imagine that by 2024 we’d all have dedicated graphics chips of some kind in our computers – in fact, we’d be hard-pressed to imagine devices without them – and that some of the biggest computing companies in the world would be graphics card manufacturers. Yet here we are.
You have to pay attention to the research as it develops, and you have to realize that they don’t just show up to markets to satisfy pre-existing demands, they create markets and create new demand where none existed before. That’s how the tech industry works.
Look at Apple for some on device examples of this.They’ve been using their ML silicon for photo editing, accessibility stuff, image classification, language, etc.
There’s a push on mobile phones, too, which are even more battery-constrained.
I don’t know if I’d call it “AI” so much as just “parallel computation”.
For phones, maybe better local speech recognition.
Can we please go back to different names for different products…?
Why have surface pro and surface laptop when you can just have surface pro for the tablets and something else for the laptop?
Makes it so much more difficult when trying to troubleshoot shit.
Oh, wait this is the same company that named completely separate software “Outlook” and “Outlook (new)”…
Same with Teams
