Also known as: NPU

A neural processing unit (NPU) is an on-device accelerator built to run neural-network inference efficiently — typically integrated into a phone, laptop, or edge device’s main chip so AI features work without the cloud.¹

An NPU is one block inside a system-on-a-chip, alongside the CPU and GPU: a wide array of multiply-accumulate lanes that runs an already-trained neural network right on the device. Keeping inference local — no round-trip to the cloud — is what makes on-device AI fast and low-power, and its headline figure is TOPS, trillions of operations per second.

Overview

An NPU is a class of AI accelerator optimized for inference at the edge: running an already-trained model with low latency and minimal power, rather than the large-scale training that data-center TPUs and GPUs handle. NPUs are usually one block inside a larger system-on-a-chip, sitting alongside the CPU and GPU and handling tasks such as camera processing, voice recognition, and background-blur. Their headline spec is TOPS (trillions of operations per second) at a given power budget.

Where it fits

The NPU is what makes edge AI practical: instead of streaming data to a server, a device runs the model locally. For a distributed scanner like GopherTrunk, an SBC-class NPU could in principle do on-the-edge classification of decoded audio or signals near the antenna, keeping bandwidth and latency low — though GopherTrunk’s core DSP is conventional hardware acceleration territory, not neural-network work.

Sources

1. AI accelerator — Wikipedia, on NPUs and related on-device machine-learning hardware. ↩

See also

• AI accelerator
• Tensor processing unit (TPU)
• System on a chip (SoC)
• Edge AI
• Graphics processing unit (GPU)
• Hardware acceleration