Field Guide · algorithm

Also known as: LMS, least mean squares, LMS algorithm, Widrow-Hoff LMS

The least-mean-squares (LMS) algorithm is the workhorse adaptive-filter update rule: it adjusts the filter taps by a small step down the instantaneous gradient of the squared error, using the simple recursion w ← w + μ·e·x*.1 Introduced by Bernard Widrow and Ted Hoff in 1960, it trades the slower convergence of an exact least-squares solution for an update that costs only a handful of multiply-accumulates per sample, which is why it appears in equalizers, echo cancellers, and noise cancellers everywhere.

start w₀ minimum each step: w += μ·e·x*
LMS descends the mean-square-error bowl in noisy little steps of size μ; a larger step converges faster but rattles more around the minimum (misadjustment).

How it works

LMS approximates true gradient descent on the mean-square error by using a single sample’s error in place of a statistical average:

  • Form the filter output y = wᵀx from the current tap vector w and the recent input samples x.
  • Compute the error e = d − y against the desired signal d (a training reference, a past decision, or — in blind variants — a target property).
  • Update every tap: w ← w + μ·e·x*, where μ (the step size) sets how far each sample nudges the taps and x* is the complex conjugate of the input for I/Q data.

Because the gradient estimate is noisy, the taps never sit exactly at the optimum; they hover around it. That residual jitter, called misadjustment, grows with μ.

Convergence and stability

The single knob μ governs the whole trade-off:

  • Too large — the taps overshoot and the algorithm diverges. Stability requires roughly 0 < μ < 2/λ_max, where λ_max is the largest eigenvalue of the input autocorrelation (in practice μ < 2/(N·P) for N taps of input power P).
  • Too small — stable and low-misadjustment, but slow to converge and slow to track a moving channel.
  • Convergence speed depends on the input’s eigenvalue spread (ratio of largest to smallest autocorrelation eigenvalue): highly coloured inputs converge slowly, a known weakness LMS shares and RLS largely fixes.

Normalized LMS (NLMS) removes the dependence on input power by scaling the step by the current input energy, μ/(ε + ‖x‖²), making the choice of μ far less sensitive to signal level — the form used in most practical echo and equalizer designs.

Relevance to SDR

LMS and NLMS are the default engines behind adaptive channel equalization, acoustic and line echo cancellation, and adaptive interference cancellers. In a radio receiver an LMS equalizer trims residual multipath, tightening the constellation and lifting the effective SNR at the decision slicer, often running decision-directed once a training sequence has pulled it near lock. The blind CMA equalizer is an LMS-style stochastic-gradient rule with a constant-modulus cost instead of a reference error. GopherTrunk runs LMS in exactly the trained role described above: its TETRA traffic path trains a short LMS equalizer on each burst’s known midamble and applies the frozen taps to that one burst (SnapshotLMS — see snapshot equalization), while the C4FM-family decoders (P25, DMR, NXDN) still lean on matched filtering and timing/carrier recovery, whose modest symbol rates keep ISI small enough to skip equalization.

Sources

  1. Least mean squares filter — Wikipedia, on the LMS update rule, step-size stability bounds, and NLMS. ↩

See also