What is an NPU? TOPS, AI PCs and when it actually helps

Almost every new laptop and phone now advertises an NPU and a number measured in TOPS. Marketing makes it sound like the most important part of the chip. In reality, an NPU is a specialist: very good at a few jobs, and irrelevant for many others. This guide explains what it is, how to read the numbers and when it matters.

What an NPU is

An NPU (neural processing unit) is a block of silicon designed for the maths behind neural networks. That maths is mostly huge numbers of multiplications and additions on matrices of numbers. A CPU can do this, but it is built to handle many different kinds of tasks one after another. A GPU can do it quickly, but it uses a lot of power. An NPU does one narrow thing with as little energy as possible.

You will rarely find an NPU as a separate chip in consumer devices. It usually sits inside the main processor next to the CPU and GPU cores. Different companies give it different names: Apple calls it the Neural Engine, Qualcomm calls it the Hexagon NPU, Intel and AMD simply say NPU, and Google's Tensor phone chips include their own TPU.

NPU vs CPU vs GPU

The key word is efficiency. Running a blur filter on your webcam all day on the GPU would drain the battery. On the NPU, the same job uses a fraction of the power, so it can stay on all the time.

What TOPS really means

TOPS stands for trillions of operations per second. It is a theoretical peak: how many simple calculations the NPU could do if every unit were busy all the time. Three things make the raw number less useful than it looks:

1. Precision. TOPS is usually quoted at INT8 (8-bit integers). Some vendors quote INT4 or "sparse" figures, which can double the number without any real change in speed. Always check which precision is used.
2. Real workloads rarely hit the peak. Memory speed, software and drivers often matter more than the headline figure.
3. Platform TOPS vs NPU TOPS. Some brands add CPU, GPU and NPU together into one large "platform TOPS" number. For Copilot+ features, only the NPU figure counts.

That is why claims like Intel's 700 TOPS target for 2028 need context about precision before you can compare them.

What NPUs are used for today

On Windows, Microsoft's Copilot+ PC label requires an NPU rated at 40 TOPS or more, along with at least 16 GB of RAM and 256 GB of storage. Those machines unlock features that run on the NPU, such as:

• Windows Studio Effects: background blur, eye contact correction and automatic framing for video calls.
• Live Captions with on-device translation.
• Recall and Click to Do, which analyse what is on your screen locally.
• Image tools in Paint and Photos that generate or edit pictures on the device.

On phones, NPUs have been doing quiet work for years. Apple added its Neural Engine with the A11 chip in 2017. Today NPUs handle portrait mode, Face ID, photo search, voice typing, call screening and on-device assistant features without sending your data to the cloud.

Example NPU ratings

The AMD chip shows why the NPU is only part of the story. As we covered in our Ryzen AI Halo report, that machine can run models with more than 300 billion parameters. The reason is its 192 GB of unified memory, not its 55 TOPS NPU.

Do NPUs help with local AI chatbots?

Partly. Small models of a few billion parameters can run on an NPU, and some apps already use it for on-device assistants. But for larger language models, the bottleneck is usually memory: how much you have and how fast it is. Generating each word means reading most of the model from memory, so a fast GPU with plenty of memory, or a chip with a large unified memory pool, beats a high TOPS number. Many popular local AI tools still run mainly on the GPU or CPU, with NPU support arriving gradually.

If running models such as Llama or Qwen on your own machine is the goal, read our laptop for local AI guide and our explanation of how much RAM you need before worrying about TOPS.

Should you buy a laptop for its NPU?

• Yes, it is a sensible tick-box if you buy a Windows laptop to keep for four or more years. More apps will use the NPU over time, and Copilot+ features need it.
• No, do not pay extra just for a higher TOPS figure. The difference between 45 and 50 TOPS is unlikely to matter. Screen, battery, memory and build quality will affect you far more.
• For gaming and creative work, the GPU still does the heavy lifting.