Also known as: Compute Unified Device Architecture
CUDA is NVIDIA’s parallel computing platform and programming model that lets ordinary, general-purpose code run on the GPU instead of only the CPU.1
Overview
A CUDA program splits work into a kernel — a small function executed in parallel by thousands of lightweight threads, each handling one element of the data. Threads are organised into blocks, and blocks into a grid, so the same code scales from a small GPU to a large one without being rewritten. The platform exposes the GPU through extensions to C and C++ (and bindings for Python, Fortran, and others), plus tuned libraries such as cuBLAS for linear algebra and cuDNN for neural networks.
Because it is proprietary to NVIDIA hardware, CUDA competes with the cross-vendor OpenCL and with newer portable frameworks, but its mature tooling, profilers, and library ecosystem made it the de facto standard for GPU computing.2 The programming model is the practical face of GPGPU: it hides the graphics pipeline entirely and presents the GPU as a general, massively parallel co-processor.
How it works
CUDA divides the machine into a host (the CPU and its memory) and a device (the GPU and its memory). Work moves across that boundary in a fixed rhythm, and the thread hierarchy maps onto the hardware’s execution units:
| Concept | Meaning | Maps to |
|---|---|---|
| Kernel | Function launched to run in parallel | GPU program |
| Thread | One instance of the kernel | Single lane |
| Block | Group of threads sharing fast memory | Streaming multiprocessor |
| Grid | All blocks of one launch | Whole GPU |
| Host ↔ device copy | Moving data over PCI Express | The main overhead |
The host-to-device copy is the cost that decides whether offloading pays off: if the compute per byte is low, the transfer dominates and the CPU would have been faster.
Where it fits
CUDA is the bridge that turned the GPU from a graphics device into a general accelerator, and it underpins most modern AI accelerator workloads on NVIDIA hardware. For a signal-processing pipeline like GopherTrunk, a CUDA kernel can run massively parallel work — large FFTs across many channels, batched FIR filtering, or a polyphase channelizer splitting one wide capture into hundreds of channels — far faster than a CPU. The catch is the host-device transfer: for a handful of narrowband channels the cost of shipping samples to the GPU often outweighs the gain, so CUDA earns its keep only when the channel count and sample rate are high.