Part 14 — the last — of The Field Notebook, a 14-part operator’s
tutorial that reads GopherTrunk’s debug.log one line family at a time —
what each field measures, what a healthy rig prints, which number is a noise
meter and which one means traffic, and which deep dive to open when a line
goes wrong. Thirteen parts read the log from the startup banner
(Part 1)
to the bug report
(Part 13).
This closing part folds them onto one page: the card to keep beside the
terminal.
TL;DR: On any report, read five lines first: the
daemon:WARNs in the startup block, the protocol’scc lockedline, its periodic status line (tetra: decode status,widebandt2: channel decode activity,tetra dmo: decode status, the MRC health line), thesoapyremote: SDR overruns/ccdecoder: decode can't keep upWARNs, andrecorder: call ended … reason=. Then the family tables give each line’s healthy shape, its worry-when, and the part that explains it. Keep noise meters apart from traffic counters —dnb_totalvsdnb_qualified,bsch_okvssch_pdus,dibitsvssync_hits,coherencevslock_gate,updatesvsholds. And know what the log cannot yet say: the −20 kHz emitter, the cc=7 CSBK train, and the on-air gates (#836, #1003, #1187).
Key takeaways
- Five lines settle most reports. Startup WARNs, lock, status line, host WARNs, teardown reason — in that order, before any DSP theory.
- Every family has a noise meter and a traffic counter. The card names the pair; reason from the one that means decode.
- “Worry when” is relative, never absolute. Deaf is judged at the
channel’s own last-decoding level, coherence against its own
lock_gate. - The card ends where the log does. Some symptoms are open questions with an instrument waiting for a capture; the card says which.
Cheat sheet
| Concern | What it does | Where it lives |
|---|---|---|
| Startup & lock | banner, daemon: WARNs, cchunt: lines, cc locked |
Parts 1, 2 |
| TETRA TMO | tetra: decode status, the #815 offset WARN |
Parts 3, 4 |
| DMR | widebandt2: channel decode activity, deaf heal, idle beacon |
Part 5 |
| TETRA DMO | tetra dmo: decode status, seed lines, grant |
Part 6 |
| Host & front end | overruns, decode can't keep up, MRC health line |
Parts 7, 8 |
| Captures & recordings | siglab: capture …, recorder: call …, composer lines |
Parts 9, 10, 11 |
| Web & reporting | panel symptoms with log evidence; what to paste | Parts 12, 13 |
In this post
- The five lines to read first — the order that settles most reports.
- The card — one table per family: line → healthy → worry → open this.
- Noise meters vs traffic counters — the pairs that mislead.
- What the log still cannot tell you — the honest list.
The five lines to read first
Every report in this series’ source material was settled faster by this reading order than by any theory.
1. The startup WARNs. daemon: gain looks like dB, not tenths-of-dB —
radio may be effectively deaf explained a −70 dBFS control channel before
anyone measured anything. Two systems on one role: control tuner, dead
channels:, a missing band plan — all announced here.
2. The lock line. tetra cc locked freq= mcc= mnc= la=,
control channel locked nac= freq= (P25), dmr cc locked freq= cc= sysid=,
tetra dmo cc locked. Without one, nothing downstream can be judged;
cchunt: hunt failed — no control-channel lock with iq_observed= says
whether IQ even arrived.
3. The status line. The protocol’s periodic DEBUG line is the instrument. Read its trend across intervals, never one line.
4. The host WARNs. soapyremote: SDR overruns … host_drops= and
ccdecoder: decode can't keep up with real time … dropped_since_last= are
downstream signals — the consumer stopped draining
(Part 7).
5. The teardown reason. recorder: call ended … reason=timeout on
every call means the voice chain decoded nothing that refreshed liveness —
a decode problem in a recorder’s clothes
(Part 11).
Healthy, on a TETRA rig:
INF tetra cc locked freq=467912500 mcc=250 mnc=1 la=1021 system=Metro-TETRA
DBG tetra: decode status system=Metro-TETRA locked=true carrier_off_hz=-412.5 baud=18000 baud_dev_pct=0.01 sb_bursts=72 bsch_ok=72 bsch_fail=0 sysinfo=18 sch_pdus=340 sch_pdus_fail=6 frag_abandons=1 grants=3 colour_code=1
INF recorder: call ended device=cc:same-carrier:1 wav=../recordings/Metro-TETRA/2001/... duration=5.6s reason=normal
Unhealthy — three lines, three different parts:
WRN daemon: gain looks like dB, not tenths-of-dB — radio may be effectively deaf serial=32B0A4C configured=50 parsed_db=5 did_you_mean=500
DBG tetra: decode status system=Metro-TETRA locked=true carrier_off_hz=4950.0 baud=17998 baud_dev_pct=0.02 sb_bursts=70 bsch_ok=69 bsch_fail=1 sysinfo=0 sch_pdus=0 sch_pdus_fail=58 frag_abandons=0 grants=0 colour_code=1
INF recorder: call ended device=cc:same-carrier:1 wav=../recordings/Site-DMO/0/... duration=7.0s reason=timeout
The middle line is the 19 Aug “locked but deaf” signature — bsch_ok ~100 %,
sch_pdus=0, carrier_off_hz one alias bucket (4500 Hz) off
(Part 4).
The card
Every field below is a real slog key; every “open” is the part or deep
dive that explains the mechanism.
Startup & lock — all protocols
| Line / field | Healthy | Worry when | Open |
|---|---|---|---|
daemon: WARNs at startup |
none | any — gain in dB, dead channels:, two systems on one control tuner |
Part 1 |
cchunt: hunt failed — no control-channel lock iq_observed= |
absent | repeating — false means no IQ, true means IQ but no decode |
Part 2 |
ccdecoder: control carrier offset far from configured frequency … (issue #815) offset_hz= |
absent | persistent > 10 s — wrong site or mistuned; a one-off 5004 Hz was an alias bucket | Part 4 |
TETRA TMO
| Line / field | Healthy | Worry when | Open |
|---|---|---|---|
bsch_ok / bsch_fail |
~100 % ok | bsch_fail climbing, sb_bursts collapsing → resync storm, MarkLost |
TETRA 10 |
sch_pdus vs bsch_ok |
both advancing | bsch_ok high, sch_pdus=0 — locked but deaf (AFC alias) |
Part 3 |
tetra: payload drought persists across resyncs … declaring lock lost |
absent | present — the escape hatch fired; re-hunt follows | Part 3 |
TETRA DMO
| Line / field | Healthy | Worry when | Open |
|---|---|---|---|
dnb_total |
thousands, ~18/s on silence | never — a noise meter | Part 6 |
dnb_qualified, tch_crc |
rise together in a PTT | dsb_schs_crc climbs, tch_crc at the floor — seed not recovered |
Cookbook 5 |
seed_known / seed_verified / solve_rejects |
known + verified per PTT | composer: tetra DMO scramble seed changed flip-flopping — rejected solves are now counted, not adopted |
TETRA 13 |
DMR — wideband and conventional
| Line / field | Healthy | Worry when | Open |
|---|---|---|---|
dibits vs sync_hits |
both advancing | dibits up, sync_hits=0 for minutes at the channel’s normal level — the deaf tap |
DMR 10 |
csbk_crc_fail |
~0 | a 30 ms-cadence train at another colour code — real, vendor, unpinned | DMR 7 |
deaf_heals, widebandt2: conventional DMR tap deaf at its decoding level — resetting the receiver |
0 | every ~15 s with coarse_offset_hz far off — a false coarse engage, now rejected |
Part 5 |
Host & front end
| Line / field | Healthy | Worry when | Open |
|---|---|---|---|
soapyremote: SDR overruns … host_drops= |
absent | present — find what got slower downstream | Part 7 |
MRC coherence vs lock_gate |
above the gate, calibrated=true |
below — judge against the logged gate, never 0.5 | Weak-Signal 10 |
updates vs holds |
updates climbing, holds ~0 |
updates frozen while holds climbs — stale gain |
Part 8 |
Captures, recorder, voice chain
| Line / field | Healthy | Worry when | Open |
|---|---|---|---|
siglab: capture started center_hz vs requested_center_hz |
equal | differ — the slice was carved at the tuner centre | Part 9 |
composer: … voice follow ended speech_frames, bfi_count, vocoder_drops |
speech_frames ≫ bfi_count |
speech_frames=0 — the seed or the tap |
Part 11 |
Web
| Line / field | Healthy | Worry when | Open |
|---|---|---|---|
History vs recorder: call started timestamps |
agree in local time | offset by exactly your UTC offset — a render bug | Part 12 |
Noise meters vs traffic counters
Five pairs recur through the series; in each, the first moves on noise and the second means decode. Reason from the second.
dnb_totalvsdnb_qualified(DMO). The DNB correlator false-fires ~18/s on a silent channel by design arithmetic; only slot-grid-qualified bursts mean traffic.bsch_okvssch_pdus(TETRA). BSCH survives a 4-rotation search and heavy FEC, so it decodes through a wrong AFC alias bucket;sch_pdusdoes not.dibitsvssync_hits(DMR). Dibits prove the stream flows, not that any of it is DMR; the 12 Sep tap produced dibits for minutes withsync_hits=0.- Wideband
coherencevslock_gate(MRC). |ρ| is diluted by every hertz of noise-only bandwidth — 0.16 on a rig decoding 1425 CRC-clean BSCH — so it only means something against the logged gate. updatesvsholds(MRC).updatesfrozen withholdsclimbing is a combine running on a gain measured minutes ago.
Two rules of the same shape: never judge a channel by absolute dBFS (Analog Edge 13), and CRC yield is the only trustworthy metric; EVM is a trap (Weak-Signal 2).
What the log still cannot tell you
An honest card ends with the rows that have no “healthy” column yet.
- The −20 kHz emitter near 442.3675 MHz (15 Sep). The deaf heal rejects the false coarse engage it caused, but what transmits there is unknown; a ±100 kHz capture centred on 442.3875 MHz over one idle cycle is the instrument.
- The cc=7 CSBK-CRC-fail train on a cc=12 IPSC system (9 Sep). Parked
with
csbko/fid/info_hexin the DEBUG line. - Three on-air gates. DMR direct mode is capture-verified but not yet run live on the fixed build (#836); the DMO pre-roll and grant-on-adoption pair is capture-validated only (#1003); DMR Enhanced Privacy decrypts the known-key captures and the live call is still owed (#1187). Those rows read “capture-verified, live run pending”.
- P25 Phase 1 weak-signal voice. The one C4FM path with neither equalizer nor soft FEC; no fix without a marginal voice capture (P25 12).
How this card shapes operator practice
- Read in order, stop at the first failure. The checks are ranked by how often they ended a thread.
- Quote the pair, not the number.
bsch_ok=72 sch_pdus=0is a diagnosis;bsch_ok=72alone is a reassurance. - Judge every “worry” relatively. Against the channel’s own level, its own gate, its own previous interval.
- Treat the open rows as invitations. Each names the capture that would close it; Part 13 says how to make and paste it.
Where to go from here
The notebook read the log; the rest of the blog explains the machinery. The series index is the card’s table of contents. For the recipes that produce these logs, The Operator’s Cookbook; for when a fix is verified, From Spec to Shipping. Two series launch alongside this finale: DMR End to End follows one carrier from 4FSK to decrypted Enhanced Privacy, and Beyond Voice walks the eleven-place pattern behind every non-voice decoder. When a row sends you to a capture, the Analog Edge tells you how to record it well.
FAQ
What are the first lines to check in a GopherTrunk log?
In order: the daemon: WARNs in the startup block, the protocol’s
cc locked line, the trend of its periodic status line, the
soapyremote: SDR overruns / ccdecoder: decode can't keep up WARNs, and
recorder: call ended … reason=. The first one that fails names the family
table — and the part — to open.
Which log counters are noise meters rather than traffic?
dnb_total (DMO), dibits (DMR), bsch_ok on its own (TETRA), wideband
coherence read without its lock_gate, and calibrated=true without
updates moving. Their partners — dnb_qualified, sync_hits,
sch_pdus, lock_gate, updates — are the numbers that mean decode.
What does “locked but deaf” look like in the TETRA status line?
locked=true, bsch_ok near 100 % of sb_bursts, sch_pdus=0, and
carrier_off_hz about ±4500 Hz from where it sat — the AFC latched an alias
bucket. The payload heartbeat forces a resync after 12 s and escalates to
MarkLost after three fruitless ones.
Why does the card give no dBFS thresholds? Because every absolute-power gate in the project’s history became a gain-staging trap: a −56 dBFS control channel decoded every C_ALOHA while a WARN called it too weak, and MRC once demanded −40 dBFS to calibrate. The card judges everything relative to the channel’s own level or gate.
Which parts of GopherTrunk are still awaiting on-air verification?
DMR direct mode’s live run on the fixed build (#836), the DMO pre-roll and
grant-on-adoption pair in the live daemon (#1003), a live DMR Enhanced
Privacy call with encryption_keys (#1187), and P25 Phase 1 weak-signal
voice, which needs a marginal capture first.
Series navigation
Part 14 of 14 · ← Part 13: From Log to Issue — What to Paste, What to Capture · Back to the series index