summaryrefslogtreecommitdiff
path: root/.claude
diff options
context:
space:
mode:
authorhathach <[email protected]>2026-07-24 14:55:59 +0700
committerhathach <[email protected]>2026-07-24 14:55:59 +0700
commit21bbcb5bbff18454979a1dc9aaa210e938a004b5 (patch)
treed5953da04f7038e80ef8734eac8facbfbbbbec1f /.claude
parentb1becd8f5fcaf4a8a07aaa76fa68d3e9f7bb18a3 (diff)
docs(target-debug): vector catch, SWO trace, verifybin, FreeRTOS threads; table integration
- Vector catch + Cortex-M fault autopsy, verified with a deliberate bad-load on stm32f407disco: CFSR=0x8200 (BFARVALID|PRECISERR), BFAR = exact bad address, stacked pc addr2lined to the faulting line; gotchas recorded (stale FPB comparators fire phantom SIGTRAPs — scrub first; arm DEMCR after reset; loads precise / stores imprecise; ARMv6-M has no CFSR/BFAR) - SWO exception trace + hw PC sampling gate PASSED on F407: 680 KB of packets in 3 s (0x17 PC samples in flash range, 0x0E SysTick enter/exit); JLinkSWOViewerCL decodes stimulus only — raw SWORead is the recipe; SWOStart needs an explicit speed headless - verifybin 'Verify successful.'; FreeRTOS -rtos plugin lists all 6 cdc_msc_freertos tasks after a run->stop cycle (plain attach = 0xDEAD placeholder); semihosting anti-note; monitor-mode pointer (untested) - Intrusiveness table gains the new rows; agent playbook bullet updated; retrieval gate 5/5 with a fresh reader; executed plan committed
Diffstat (limited to '.claude')
-rw-r--r--.claude/agents/target-debugger.md3
-rw-r--r--.claude/skills/target-debug/SKILL.md176
2 files changed, 117 insertions, 62 deletions
diff --git a/.claude/agents/target-debugger.md b/.claude/agents/target-debugger.md
index 861600428..4d874e38c 100644
--- a/.claude/agents/target-debugger.md
+++ b/.claude/agents/target-debugger.md
@@ -13,7 +13,8 @@ truth; read the relevant SKILL.md BEFORE acting:
- `.claude/skills/target-debug/SKILL.md` — your primary playbook: technique
choice by intrusiveness, channel choice by link topology, capture recipes,
- GDB autopsy, all rig warnings.
+ breakpoint/watchpoint budget and cost model, vector catch + fault autopsy,
+ SWO trace, GDB autopsy, all rig warnings.
- `.claude/skills/hil/SKILL.md` — host/config selection, board lock protocol,
`hil_test.py` invocation.
- `.claude/skills/usbmon/SKILL.md` — Linux-host URB capture; exists only when a
diff --git a/.claude/skills/target-debug/SKILL.md b/.claude/skills/target-debug/SKILL.md
index 21dea38db..d0ceef2a9 100644
--- a/.claude/skills/target-debug/SKILL.md
+++ b/.claude/skills/target-debug/SKILL.md
@@ -18,11 +18,10 @@ Raspberry Pi). Pick capture channels by which end runs Linux, not by habit:
| `usb-sniffer` | what crossed the wire (PIDs, handshakes, resets) | hardware tap cabled in — role-agnostic |
For enumeration/transfer bugs the default posture is **dual-side capture** —
-both ends simultaneously, not one-side-first-then-escalate: usbmon plus a
-target channel when a Linux PC is the host. When TinyUSB is the host there is
-no usbmon on either end — pair the target channel with the wire
-(`usb-sniffer`) and, if the peer is a Linux gadget, `usb-kernel-debug` on the
-peer.
+both ends simultaneously: usbmon + a target
+channel when a Linux PC is the host; TinyUSB-as-host has no usbmon on either
+end — target channel + the wire (`usb-sniffer`), plus `usb-kernel-debug` on a
+Linux gadget peer.
## Rig discipline — lock first, always
@@ -52,9 +51,12 @@ that IS a finding (timing-sensitive): move down in intrusiveness, not up.
| Technique | Intrusiveness | Reach for it when |
|---|---|---|
| PC-sampling | none — no halt, no code change | core wedged/spinning somewhere unknown (rusb2 FRDY) |
+| SWO exception trace / hw PC-sample | none — needs SWO pin wired | ISR ordering/timing with zero code change |
+| Vector catch | none until a fault fires | crash-shaped wedges — autopsy AT the faulting pc |
| RAM ring-buffer | ~tens of cycles per event | ISR ordering/timing bugs (musb babble) |
-| TU_LOG (RTT) | µs per line | logic bugs that survive logging |
-| TU_LOG (UART) | ms per line — blocking write | same, when no J-Link on the board |
+| TU_LOG (RTT) | µs per line | logic bugs that survive logging (J-Link or OpenOCD rtt) |
+| TU_LOG (UART) | ms per line — blocking write | same, when no debug-probe RTT path |
+| dprintf / conditional breakpoint | halt+resume per hit (~ms) | low-rate probes post-wedge; never ISR-rate events |
| GDB halt / breakpoints | stops USB service entirely | post-mortem state autopsy once wedged |
## TU_LOG capture
@@ -73,15 +75,14 @@ stty -F /dev/ttyACM<N> 115200 raw && timeout 20s cat /dev/ttyACM<N> | tee /tmp/u
```bash
# OpenOCD RTT (any probe OpenOCD drives) — in telnet :4444 (or -c equivalents):
-rtt setup 0x20000000 0x8000 "SEGGER RTT" # search range = RAM ORIGIN + LENGTH (from the .ld / map file)
+rtt setup 0x20000000 0x8000 "SEGGER RTT" # RAM ORIGIN + LENGTH (from the .ld/map)
rtt start # after firmware booted; rerun after each reflash
rtt server start 19021 0
# then: timeout 20s nc localhost 19021 > /tmp/rtt.log
```
-OpenOCD polls the buffer: bursty logs can drop lines a J-Link would keep —
-prefer J-Link where both exist; the drain-model warning below applies
-unchanged.
+OpenOCD polls — bursty logs can drop lines; prefer J-Link where both
+exist. The drain-model warning below applies unchanged.
An RTT-built firmware that has since wedged still holds a log tail in RAM —
but ONLY what fits the drain model: the default SEGGER mode (NO_BLOCK_SKIP)
@@ -99,10 +100,8 @@ reads don't halt the target.
## GDB — state autopsy and watchpoints
Connect/load recipes per probe family (J-Link, OpenOCD for ST-Link /
-CMSIS-DAP / WCH-Link) are in CLAUDE.md "GDB Debugging". For scripted/batch
-sessions add `-singlerun` to JLinkGDBServer — the server exits with the
-connection; back-to-back server relaunches race the probe handle and hang at
-startup. Release builds keep DWARF (`MinSizeRel`), so `p`/struct access works
+CMSIS-DAP / WCH-Link) are in CLAUDE.md "GDB Debugging". Scripted sessions: JLinkGDBServer `-singlerun` (exits with the
+connection) — back-to-back relaunches race the probe handle and hang. Release builds keep DWARF (`MinSizeRel`), so `p`/struct access works
on HIL firmware.
**Autopsy of a wedged board: attach and halt ONLY** — skip CLAUDE.md's
@@ -121,55 +120,87 @@ watch xfer_status[2][1].total_len # HW watchpoint (Cortex-M: ~4); dwc2 names
break dcd_int_handler # works, but see warning below
```
-**Hardware budget — read it off the chip, not from memory** (verified: F407/M4
-= 6 bp + 4 wp, rp2040/M0+ = 4 + 2; M7 typically 8/4):
+**Hardware budget — read it off the chip** (verified: F407/M4 = 6 bp + 4 wp,
+rp2040/M0+ = 4 + 2; M7 typically 8/4):
```gdb
p ((*(unsigned*)0xE0002000)>>4) & 0xF # FPB NUM_CODE = hw breakpoints (M7 adds bits[14:12])
p (*(unsigned*)0xE0001000)>>28 # DWT_CTRL NUMCOMP = watchpoint comparators
```
-- `hbreak`/`thbreak` force a hardware breakpoint (code in flash can't take a
- software break unless the probe does flash breakpoints — J-Link does,
- OpenOCD needs `bp <addr> 2 hw`); `tbreak` = one-shot.
-- `watch -l <expr>` watches the *address* the expression evaluates to once —
- cheap and what you almost always want; `rwatch`/`awatch` trap reads/any
- access (hardware-only — they error rather than fall back). OpenOCD (telnet
- :4444) adds a data-VALUE match GDB cannot express: `wp <addr> 4 w <value>
- [mask]` — fires only when the written value matches (e.g. catch who writes
- 0 into a busy flag, ignoring writes of 1).
+- `hbreak`/`thbreak` force a hardware breakpoint (software breaks in flash
+ need flash-breakpoint support — J-Link has it; OpenOCD: `bp <addr> 2 hw`);
+ `tbreak` = one-shot.
+- `watch -l <expr>` watches the address expr evaluates to once — almost
+ always what you want; `rwatch`/`awatch` trap reads/any access (hardware-
+ only — they error, never fall back). OpenOCD adds a data-VALUE match GDB
+ can't express: `wp <addr> 4 w <value> [mask]` — catch who writes 0 into a
+ busy flag, ignoring writes of 1.
- **Demand the word "Hardware" in the confirmation.** `watch` silently falls
- back to a SOFTWARE watchpoint when no DWT comparator fits (expression too
- wide/complex, budget exhausted): GDB then single-steps the whole program —
- hundreds of times slower, certain USB death. `Watchpoint 2:` without
- "Hardware" = delete it; narrowing the expression (`watch -l`, cast to a
- 4-byte int) is the real fix.
-- Conditional breaks/watches (`break dcd_edpt_xfer if ep_addr==0x81`) are
- evaluated by GDB on the HOST with our stubs — neither JLinkGDBServer nor
- OpenOCD supports target-side agent expressions on Cortex-M — so every hit
- is a halt+resume (~ms) whether the condition matches or not: fine
- post-wedge or on cold paths, wrong under live USB traffic.
-- `commands <bpnum> ... end` auto-runs GDB commands at each hit (start with
- `silent`, end with `continue` for hands-free evidence collection) — same
- halt-per-hit cost.
-- `dprintf <loc>,"fmt",args` = printf without recompiling. Stay on the
- default `dprintf-style gdb` (host prints): the `call` style runs the
- target's own printf mid-halt and `agent` needs stub support — neither is
- viable on these probes. Same cost model as conditional breaks; for
- ISR-rate events use the RAM ring buffer instead.
-- Stepping while the USB ISR fires between every step is chaos: OpenOCD
- `cortex_m maskisr steponly` masks interrupts during single-steps only.
- The bus keeps running either way — the host may still reset a device that
- stops responding mid-step.
-- While halted you can poke state to test a hypothesis (`set var
- _usbd_dev.ep_status[2][1].busy = 0`) — but that invalidates the snapshot
- as post-mortem evidence; dump first, poke after.
+ back to a software watchpoint when no DWT comparator fits — GDB then
+ single-steps the whole program, hundreds of times slower: certain USB
+ death. Plain `Watchpoint 2:` = delete it and narrow the expression
+ (`watch -l`, cast to a 4-byte int).
+- Conditional breaks (`break ... if ep_addr==0x81`) and `dprintf
+ <loc>,"fmt",args` (printf without recompiling; keep `dprintf-style gdb`)
+ are host-evaluated — no Cortex-M agent expressions in our stubs: every hit
+ halts+resumes (~ms) even when the condition is false. Post-wedge/cold
+ paths only; ISR-rate events belong in the RAM ring.
+- `commands <bpnum> ... end` (start `silent`, end `continue`) auto-collects
+ evidence per hit — same halt-per-hit cost.
+- Stepping with the USB ISR firing between steps is chaos: OpenOCD
+ `cortex_m maskisr steponly`. The bus runs either way — the host may still
+ reset a halted-looking device.
+- Poking state while halted (`set var _usbd_dev.ep_status[2][1].busy = 0`)
+ tests a hypothesis but invalidates the post-mortem — dump first, poke after.
+- FreeRTOS examples: `-rtos GDBServer/RTOSPlugin_FreeRTOS` (OpenOCD: `-rtos
+ FreeRTOS`) → `info threads` lists every task with state/prio/frame
+ (verified: 6 tasks). It populates only after a
+ run→stop cycle — plain attach shows one 0xDEAD placeholder. Semihosting is
+ never the answer (traps + halts per call — RTT instead). **Monitor-mode
+ debugging** (J-Link, M3+) keeps chosen IRQs serviced at a breakpoint —
+ needs SEGGER's JLINK_MONITOR files + `SetMonModeDebug=1`; not set up here:
+ <https://kb.segger.com/Monitor_Mode_Debugging> (untested).
While halted the device answers **nothing**: host control transfers time out
in ~5 s and the OS may reset/re-enumerate — after `continue`, the bus traffic
shows recovery, not the original bug. Prefer one halt for a post-mortem dump
over stepping through live USB traffic.
+## Vector catch + fault autopsy — catch the crash, not the wedge
+
+A wedge that is really a fault (HardFault loop, lockup) autopsies best AT
+the faulting instruction. Two hardware-proven gotchas: **FPB/DWT comparators
+survive reflash and dead sessions** — a stale one fires as a phantom SIGTRAP
+at an unrelated line of NEW firmware — and J-Link's reset strategy manages
+vector-catch bits: scrub first, arm AFTER reset:
+
+```gdb
+# scrub: FP_COMP0..5 = 0xE0002008..201C, DWT_FUNCTIONn = 0xE0001028 + n*0x10
+set *(unsigned*)0xE0002008 = 0
+# ... (repeat per comparator; count from the budget reads above)
+# arm (after monitor reset; tool-agnostic — works via JLinkExe w4 too):
+set *(unsigned*)0xE000EDFC |= (1<<10)|(1<<9)|(1<<8)|(1<<7)|(1<<6)|(1<<5)|(1<<4)
+# = VC_HARDERR|INTERR|BUSERR|STATERR|CHKERR|NOCPERR|MMERR; bit0 VC_CORERESET halts at reset
+```
+
+OpenOCD native: `cortex_m vector_catch hard_err bus_err state_err chk_err mm_err`.
+It halts at exception ENTRY (pc = handler, LR = EXC_RETURN 0xFFFFFFFx); decode:
+
+```gdb
+p/x *(unsigned*)0xE000ED28 # CFSR — low byte MemManage, byte1 BusFault, top half UsageFault
+p/x *(unsigned*)0xE000ED2C # HFSR — bit30 FORCED = an escalated lower-priority fault
+p/x *(unsigned*)0xE000ED38 # BFAR — faulting address (valid if CFSR bit15 BFARVALID)
+x/8wx $msp # stacked frame: r0 r1 r2 r3 r12 lr pc xpsr — pc = culprit
+```
+
+`addr2line -e <elf> <stacked pc>` names the line (verified: CFSR 0x8200,
+BFAR = the bad address, stacked pc = the faulting ldr). Loads fault
+precisely; stores usually IMPRECISERR (BFAR invalid, pc late). ARMv6-M has no
+CFSR/BFAR, only VC_HARDERR|VC_CORERESET — stacked frame alone. Still a halt
+(host URB timeouts apply); clear DEMCR (`&= ~0x7F0`) before handing back;
+RISC-V: breakpoint the trap handler; mcause/mepc/mtval are the CFSR/BFAR analogs.
+
## RAM ring-buffer trace
The zero-print instrument (cracked the musb babble): a small event ring in the
@@ -218,11 +249,31 @@ arm-none-eabi-addr2line -e <firmware.elf> -f -a 0x<hot-pc> ... # PCs → funct
OpenOCD variant: repeat `mdw 0xE000101C` over telnet :4444. The histogram's
top entries are the spin site; a flat histogram = core is servicing normally.
+### SWO — hardware-timed trace on one pin (J-Link; verified on F407)
+
+If SWO (TRACESWO) is wired, DWT emits packets with ZERO code change:
+**exception trace** (DWT_CTRL bit16 — every IRQ enter/exit, timestamped) and
+**hardware PC sampling** (bit12), better histograms than DWT_PCSR polling.
+SWOViewer tools decode only ITM *stimulus* (TinyUSB emits none) — capture
+raw:
+
+```bash
+# JLinkExe -CommandFile:
+w4 E0001000, 0x00011401 # EXCTRCENA|PCSAMPLENA|SYNCTAP|CYCCNTENA
+SWOStart 4000000 # explicit speed — autodetect fails headless
+Sleep 3000
+SWORead # hex: 0x17+4B LE = PC sample, 0x0E+2B = IRQ enter/exit
+```
+
+Verified: 680 KB in 3 s (flash-range PC samples + SysTick enter/exit).
+SWORead stuck at 0 = SWO pin not wired (many boards route only SWDIO/SWCLK).
+Restore DWT_CTRL when done.
+
## Dual-side capture — the default for enumeration/transfer bugs
-Start both channels, then trigger the failing test (Linux-PC-host link shown;
-TinyUSB-as-host: swap the usbmon line for a `usb-sniffer` capture, plus
-`usb-kernel-debug` on the peer if it is a Linux gadget):
+Start both channels, then trigger the failing test (Linux-PC-host shown;
+TinyUSB-as-host: swap usbmon for `usb-sniffer`, + `usb-kernel-debug` on a
+Linux gadget peer):
```bash
.claude/skills/usbmon/scripts/usbcap.sh cafe: 30 /tmp/host.pcapng & # host URBs (usbmon skill)
@@ -239,14 +290,13 @@ the wire itself: `usb-sniffer` skill (hardware tap, PID-level).
## Manuals
-- J-Link / J-Trace User Guide (UM08001): <https://kb.segger.com/UM08001_J-Link_/_J-Trace_User_Guide> — flash breakpoints, RTT, SWO, monitor mode, Commander commands.
-- OpenOCD User's Guide: <https://openocd.org/doc/html/index.html> — `rtt`, `bp`/`wp`, `cortex_m vector_catch` / `maskisr`, `itm`/`tpiu`.
-- "Debugging with GDB" (the official manual; §5.1 covers break/watch/dprintf):
- calibre library first (`read-doc` skill) — use the **Tenth Edition (GDB 18)**
- copy, not the 2002 Ninth-Edition txt also present; fallback
+- J-Link (UM08001): <https://kb.segger.com/UM08001_J-Link_/_J-Trace_User_Guide> — flash breakpoints, RTT, SWO, monitor mode, Commander.
+- OpenOCD: <https://openocd.org/doc/html/index.html> — `rtt`, `bp`/`wp`, `cortex_m vector_catch`/`maskisr`, `itm`/`tpiu`.
+- "Debugging with GDB" (§5.1 = break/watch/dprintf): Tenth Edition (GDB 18)
+ via calibre/`read-doc` — NOT the 2002 Ninth-Edition txt also there — or
`curl -sL -o /tmp/gdb.pdf https://sourceware.org/gdb/current/onlinedocs/gdb.pdf`
- (the HTML mirror blocks fetchers; the PDF works). The installed
- `arm-none-eabi-gdb`'s `help <cmd>` is authoritative for what this rig runs.
+ (the HTML mirror blocks fetchers). Installed `arm-none-eabi-gdb`
+ `help <cmd>` is authoritative here.
## Warnings
@@ -268,6 +318,10 @@ the wire itself: `usb-sniffer` skill (hardware tap, PID-level).
freshly-reset chip (e.g. NVIC ISER = 0) reads like a smoking gun. Read DHCSR
(0xE000EDF0: bit 17 S_HALT, bit 25 S_RESET_ST) with every snapshot, and
cross-check against something the device demonstrably still does.
+- **"Flash OK" can lie** (silent no-op — old firmware keeps running). When
+ behavior contradicts the flashed code: `objcopy -O binary fw.elf
+ /tmp/fw.bin`, then `verifybin /tmp/fw.bin,<flash-base>` (J-Link, verified)
+ or `verify_image` (OpenOCD); on mismatch reflash before debugging further.
- **A marginal link can fake a deterministic firmware bug** — down to failing
the same test at the same iteration twice. "USB disconnect" in dmesg on a
freshly re-cabled port (high devnum = churn) means the plug, not the code: