Also known as: on-device AI, edge inference
Edge AI is the practice of running machine-learning models directly on local devices — single-board computers, cameras, sensors — instead of sending data to the cloud for inference.1
Overview
The appeal is concrete: inference next to the data is lower-latency, keeps private data on the device, and keeps working without a network. A doorbell that recognises a face locally answers in milliseconds and never uploads the video; a cloud version has to survive a round trip and trust a remote server with the footage. Bandwidth costs drop too, because only the small result travels rather than a continuous raw stream.
The cost is that small devices have limited compute, so edge AI leans on efficient, quantised models and on dedicated AI accelerators. Rather than run a full-precision network on a general CPU, edge deployments shrink the model — lower numeric precision, pruning, distillation — and hand the heavy math to purpose-built silicon like Google Coral’s Edge TPU or the GPU on an NVIDIA Jetson. It is a specific case of the broader move toward edge computing.
Edge vs cloud inference
| Cloud inference | Edge AI | |
|---|---|---|
| Latency | Network round trip | Immediate, local |
| Privacy | Data leaves the device | Data stays put |
| Offline | Fails without a link | Keeps working |
| Compute available | Effectively unlimited | Small, power-bound |
| Model size | Large, full precision | Quantised, compact |
| Ongoing cost | Bandwidth + server time | One-time hardware |
Where it fits
Edge AI shows up wherever round-tripping to a server is too slow, too costly, or impossible: factory cameras, doorbells, robots, and home automation hubs. In a GopherTrunk-style deployment, edge AI could classify or flag activity in decoded data on the same node that does the radio work — spotting a signal of interest, sorting traffic, or triggering a recording — and send up only the results rather than every sample, keeping a remote capture node useful even on a thin link.
Sources
-
Edge computing — Wikipedia, on processing data near its source, including on-device inference. ↩