Part 12 of From Spec to Shipping, a 14-part series on how a protocol decoder actually gets written — from standards documents and independent references to code you can trust on air. Part 11 made captures first-class test fixtures. This part is the rule that governs what you do with them: a bug fix is one narrow commit plus a regression test that fails without the fix and passes with it — and if you can’t write the failing test, you haven’t reproduced the bug, so keep digging instead of guessing. Three worked examples show the failing test doing the diagnosis itself.
TL;DR:
CONTRIBUTING.mdstates the rule in one line — “regression test that fails without the fix and passes with it” — and the tree is full of tests built to that shape.TestDMTCHSpeechRoundTrip(internal/radio/tetra/dmo_decode_test.go) scrambles its encode side unconditionally, like a real transmitter: the old colour-0 descramble skip yields 0 speech frames; the #1003 fix, 2 CRC-valid frames.TestProcessDecodesRealAirFormat(internal/radio/motorola/process_test.go) feeds the real SmartNet air format to the decoder: the pre-#1143 fabricated framing decodes nothing. And threeinternal/scanner/ccdecoder/pipelines_dmo_test.gotests all fail against the old DMO grant path that opened recordings on noise. The failing test is the reproduction, and the reproduction is where the root cause surfaces.
Key takeaways
- A failing test is a reproduction you can re-run. Until something fails on demand, you have a theory about the bug, not the bug — and a fix built on a theory is a guess wearing a commit message.
- The hard part is making the test able to fail. All three examples needed the test’s world made faithful first: an unconditional scrambler, a real-air bit stream, a true 255-dibit slot grid.
- The inversion is the point. Reproduce → fail → fix → pass front-loads the diagnosis; guess → fix → close defers it to the reporter, who becomes your regression suite.
- When no failing test is possible, the fix does not land. The P25 C4FM weak-signal levers are diagnosed, designed — and parked until a capture can make a test fail — Part 14’s subject.
Cheat sheet
| Concern | What it does | Where it lives |
|---|---|---|
| The rule | bug fix = narrow commit + a test that fails without it | CONTRIBUTING.md (“How changes are scoped”) |
| Colour-0 regression | encode side scrambles unconditionally, like real air | internal/radio/tetra/dmo_decode_test.go (TestDMTCHSpeechRoundTrip) |
| SmartNet regression | real-air OSW stream → old decoder = zero decodes | internal/radio/motorola/process_test.go (TestProcessDecodesRealAirFormat) |
| Noise-grant trio | idle channel, lock ordering, re-arm — three failure modes | internal/scanner/ccdecoder/pipelines_dmo_test.go |
| Faithful synthetic transmitter | every burst on a true 255-dibit timeslot | pipelines_dmo_test.go (buildDMODibitStream) |
| When you can’t fail first | the issue stays open, gated on evidence | CLAUDE.md policy → Part 14 |
In this post
- The rule, and what it is actually for — a failing test is a reproduction, not paperwork.
- Zero frames, then two — the DMO colour-0 descramble regression.
- A decoder that decodes nothing — the SmartNet real-air-format test.
- Three tests against one pipeline — the noise-grant trio.
- Failing first as a diagnostic method — root causes surface in the reproduction.
- When you cannot fail first — the honest ending, the Part 14 tie-in.
The rule, and what it is actually for
CONTRIBUTING.md scopes a bug fix in one sentence: one commit, narrow diff,
regression test that fails without the fix and passes with it. The standing
guidance sharpens the contrapositive: if you can’t write a test that fails
first, you have not yet reproduced the bug — keep digging or ask for a
reproduction, don’t guess at a fix.
Read as process, that sounds like ceremony. Read as epistemology, it’s the cheapest verification gate in the series. A fix without a failing test makes two unproven claims at once: that you found the real cause, and that your change removes it. The failing test converts both into observations — it fails against the old code, so the reproduction is real; it passes against the new, so the change addresses that reproduction. The price of skipping them was paid in public: issue #764 was closed twice on fixes nobody had watched fail while the symptom stayed live (#771) — the story Part 14 turns into policy.
One earned caveat: a failing test proves you reproduced a bug; only outside evidence proves it’s the bug — the Part 7 villain fabricates a convincing failure as easily as a convincing pass.
Zero frames, then two: the colour-0 descramble
Part 10
told the DMO story from the verdict side: clear TETRA direct-mode voice sat
at a ~1/256 chance-floor CRC rate and got misread as encryption, when the
real cause was DMBurstTCHSpeech skipping descramble at colour code 0.
TETRA scrambling is not an identity at colour 0 — the LFSR seeds to
0xC0000000 (EN 300 392-2 §8.2.5.2), which is why the signalling paths, which
always descramble, always decoded.
The regression test is the interesting artifact, because the old suite had
checked this and passed: its round-trips scrambled and descrambled under the
same colour != 0 condition — self-consistent, green either way, blind to
the asymmetry. The fix’s test changes one thing: the encode side now behaves
like the air, unconditionally.
// internal/radio/tetra/dmo_decode_test.go (shape) — TestDMTCHSpeechRoundTrip
// The colour-0 iteration is the failing-first regression for issue #1003:
// real air scrambles TCH/S even at colour 0 (seed 0xC0000000, §8.2.5.2), so
// a decoder that skips descramble there fails the class-2 CRC.
for _, colour := range []uint32{0, 0x0AB1F} {
/* … encode two speech frames through EN 300 395-2 TCH/S coding … */
onair := framing.ScrambleTetra(type4, colour) // scrambled like real air, incl. colour 0
/* … frame as a DNB, run the real extractor … */
frames := DMBurstTCHSpeech(*dnb, colour)
if len(frames) != 2 {
t.Fatalf("colour %#x: got %d frames, want 2 (CRC failed?)", colour, len(frames))
}
}
Verified failing-first, numbers on record: the old code returns 0 frames at colour 0; the fixed code returns the 2 CRC-valid frames that went in. That asymmetric shape — pin one side to reality, let the other prove itself — is the whole difference from the green predecessor, and the constructive mirror of the self-consistent trap. Full saga: TETRA End to End Part 12.
A decoder that decodes nothing: the SmartNet regression
Part 8’s case study ended with a rebuilt Motorola SmartNet decoder — the original’s fabricated framing matched no real reference, so no real system could ever lock, while every synthetic test glowed green. The rebuild’s headline regression states the indictment as a runnable fact:
// internal/radio/motorola/process_test.go (shape) — TestProcessDecodesRealAirFormat
// The issue #1143 regression: a stream in the REAL SmartNet air format (OP25
// rx_smartnet framing) must lock and publish the grant. The pre-#1143
// decoder (24-bit sync 0xA4D7AA + BCH(64,16,11), a framing no real system
// transmits) decodes NOTHING from it — verified failing-first.
cc := New(Options{Bus: bus, SystemName: "Real", FrequencyHz: 854_562_500})
cc.Process(realAirStream(200, seq...), 0)
locked, grants := drainEvents(sub)
if len(locked) == 0 {
t.Fatal("no cc.locked from a real-air-format stream")
}
/* … grant must carry tg 0xB010, src 0x2E9A, 861.5375 MHz (channel 0x1A5) … */
Notice the input: not the old encoder’s output, but a stream built to the
framing two independent proven-on-air decoders transmit and consume —
OP25’s rx_smartnet and trunk-recorder’s parser, cross-checked as
Part 3
prescribes. That provenance is what makes the failure meaningful: the old
decoder produces zero locks and zero grants from it — the reporter’s
cchunt: hunt failed symptom, reproduced on the desk, no antenna required.
(Per the on-air gate, even the rebuild stays capture-verification-pending
until the reporter’s 854.5625 MHz capture replays through it.)
Three tests against one pipeline: the noise-grant trio
The richest example is the DMO grant path: one field report yielded three
distinct failing tests, each pinning a different way the old code was
wrong. The report: the daemon granted and opened a recording ~230 ms after
startup on a silent channel, then never granted again — the operator’s
real 10-second PTT, which genuinely locked (dsb_schs_crc=46/54), produced
nothing. The mechanism
(TETRA End to End Part 13):
the DNB burst correlator fires ~18 times per second on pure noise, and
the grant logic took raw detections as traffic.
Test (pipelines_dmo_test.go) |
The failure it pins | Old code’s behaviour |
|---|---|---|
TestTETRADMOPipelineIgnoresIdleChannel |
10 s of noise: no lock, no grant, zero qualified DNBs — while raw detections keep raining | grants in ~230 ms on nothing |
TestTETRADMOPipelineGrantsOnlyAfterLock |
no grant may precede cc.locked |
dnbSinceLock was named for a lock the guard never consulted |
TestTETRADMOPipelineRearmsBetweenTransmissions |
after a silent gap, a second PTT must grant again | re-arm needed a drought the 18/s noise made impossible — and only ran when a burst arrived |
All three fail against the old pipeline. But the load-bearing work happened
before the assertions could mean anything: the synthetic transmitter had to
become faithful. The old fixture laid bursts into the stream with arbitrary
filler between them — and against that layout the eventual fix
(tetra.DMSlotGrid, which qualifies bursts by voting their positions onto
one learned residue of the 255-dibit timeslot grid) is untestable, because
the “transmitter” itself didn’t keep slot time. buildDMODibitStream now
emits every burst on its own 255-dibit timeslot, as a real radio transmits
them — at which point the idle-channel test could demand dnbQualified == 0
against the detection rain and watch the old code fail.
Failing first is a diagnostic method, not a formality
The rule’s reputation is bureaucratic. In practice it’s where most of the diagnosis happens, because building a reproduction forces you to state, mechanically, what you believe the bug is — and mechanical statements get checked in ways prose theories don’t.
The DMO trio shows it best. “The grant fires on noise” was the theory; making
a test say it required numbers — how often does the correlator fire on
noise? Working that out (529 of 4^11 possible 11-dibit patterns match at
tolerance 2, times eight filters, at 18,000 dibits/s — ~18 false
detections/s; the operator’s log measured 18.7) didn’t just justify the test;
it dictated the fix’s shape. A threshold can’t beat an 18/s rain, but a
slot-grid residue vote can: one radio on one clock puts every burst on one
residue mod 255, noise is uniform over all 255. Reproduction and design came
out of the same arithmetic. Likewise the colour-0 test — writing “scramble
like real air” forced the question what does real air do at colour 0?,
whose answer (0xC0000000, not identity) was the bug.
And when you can’t make the test fail, that’s data too: whatever you reproduced isn’t what was reported — and you found out at your desk, not in the reporter’s follow-up.
When you cannot fail first
The rule’s honest edge case: some failures can’t be reproduced from what you
have. The P25 Phase 1 C4FM weak-signal gap is the standing example — the
diagnosis is structural and confident (no equalizer, no soft-decision FEC on
that path), the fix is designed, and nothing has landed, because without a
weak C4FM voice capture in samples/p25/ there is no test that fails first.
So the workflow ends the only honest way it can: the issue stays open, with a status comment saying what’s known and what’s blocking — usually a Part 11-shaped capture request. What it must not end with is a close. Closing is a claim that the problem is gone, and that claim has its own definition, its own policy, and its own guard hook. That is the finale’s subject.
Where this goes next
A failing test proves the fix worked once, on your fixture. Whether it keeps working in the field only the running system can say — so it has to be able to say things. Part 13 builds diagnostics as instruments: counters on every branch, verdict lines written for operators, WARNs that can tell a condition from a transient.
FAQ
What makes a regression test “failing-first” rather than just a test? Provenance: it ran against the pre-fix code, was observed to fail, and the observation is recorded in the test’s doc comment (“verified failing-first against the old code”). A test only ever seen green proves the code satisfies the test, never that the test detects the bug.
Isn’t this just test-driven development? It’s TDD’s contrapositive, applied to defects: TDD writes a failing test to specify behaviour that doesn’t exist yet; failing-first writes one to reproduce behaviour that shouldn’t. The extra burden is fidelity — the test’s world must match the air, the wire, or the protocol, or it reproduces nothing (the Part 7 problem).
What if writing the failing test takes longer than the fix? It usually does, and that time is the diagnosis, not overhead. The colour-0 fix was a guard removal; finding which guard took a colour sweep, a scrambler-seed check, and an encode-side rebuild of the test — while “fix it and see” had already produced a wrong answer (“encrypted”) from the same evidence.
How do I fail first when the bug only appears on air?
Get the air onto disk: an IQ capture replayed through an env-gated harness is
a failing test an operator can run (GT_TETRA_DMO_IQ=… go test -run
TestTETRADMOReplay). Part 11 covers the conventions. When even that isn’t
available: the fix waits, and the capture request is the work product.
Series navigation
Part 12 of 14 · ← Part 11: Capture-Driven Development · Next → Part 13: Instruments, Not Logs