diff options
| -rw-r--r-- | .claude/skills/hil/SKILL.md | 14 | ||||
| -rw-r--r-- | .claude/skills/usb-debug/SKILL.md | 36 | ||||
| -rwxr-xr-x | .claude/skills/usb-debug/scripts/usb_dyndbg.sh | 46 | ||||
| -rw-r--r-- | .claude/skills/usb-recover/SKILL.md | 91 | ||||
| -rwxr-xr-x | .claude/skills/usb-recover/scripts/usb_recover.sh | 111 | ||||
| -rw-r--r-- | .claude/skills/usbtest/SKILL.md | 123 |
6 files changed, 421 insertions, 0 deletions
diff --git a/.claude/skills/hil/SKILL.md b/.claude/skills/hil/SKILL.md index a7a916907..18e59c060 100644 --- a/.claude/skills/hil/SKILL.md +++ b/.claude/skills/hil/SKILL.md @@ -14,6 +14,20 @@ Run TinyUSB HIL tests on real boards. **Run `hostname` first** — it tells you Default to **local**. Use **remote** only when on `htpc` and the user says `remote`/`ci.lan`. Never attempt remote on `ci`. +## Stop the CI runner first (on `ci`) + +The `ci` rig also hosts a GitHub Actions runner that flashes boards and runs HIL as part of CI. If it fires while you are driving the hardware yourself — any HIL run, flashing, `test/hil/usbtest.py`, GDB, raw USB — it reflashes boards mid-test and churns the bus, producing spurious failures and even wedged devices. + +**Before touching hardware on `ci`, stop the runner; restart it when done.** `svc.sh` is run with `sudo` but must be run **from the runner root** (`~/actions-runner`, plural), else it errors "Must run from runner root": + +```bash +(cd ~/actions-runner && sudo ./svc.sh stop) # before any hardware/HIL action +# ... flash / run hil_test.py / usbtest.py / GDB ... +(cd ~/actions-runner && sudo ./svc.sh start) # ALWAYS restart when finished +``` + +Treat the restart as mandatory cleanup — leaving the runner stopped silently disables CI for the whole repo. Only applies on `ci` (htpc has no runner). Check state with `(cd ~/actions-runner && sudo ./svc.sh status)`. + ## Prerequisites Examples must be built for the target board(s) — see AGENTS.md "Build" → "All examples for a board" (produces `examples/cmake-build-<board>/`). `-B examples` points `hil_test.py` at that parent folder. diff --git a/.claude/skills/usb-debug/SKILL.md b/.claude/skills/usb-debug/SKILL.md new file mode 100644 index 000000000..20ab7d764 --- /dev/null +++ b/.claude/skills/usb-debug/SKILL.md @@ -0,0 +1,36 @@ +--- +name: usb-debug +description: Use when USB enumeration fails or misbehaves and usbmon alone can't explain WHY the host acted — port reset storms, repeated re-enumeration, address errors, xHCI ring/command errors, "device descriptor read error", babble, or when you need the host driver's own reasoning from dmesg on the ci HIL rig. +--- + +# usb-debug — host-side kernel dynamic debug for USB + +usbmon shows the URBs; kernel **dynamic debug** shows the host driver's +*reasoning* usbmon can't: port resets and their causes, enumeration retries, +address (re)assignment, EP halts, xHCI ring/command errors. + +Run this skill's `scripts/usb_dyndbg.sh` with `sudo` (abbreviated to +`usb_dyndbg.sh` in the examples below). It flips the dynamic-debug print flag +for an allowlisted set of USB host modules only: + +```bash +sudo usb_dyndbg.sh on usbcore xhci_hcd # enable +p; pick modules from `lsusb -t` Driver= +sudo usb_dyndbg.sh status [module] # list enabled print sites +sudo usb_dyndbg.sh off usbcore xhci_hcd # ALWAYS turn off when done — very noisy +``` + +Allowlisted modules: `usbcore xhci_hcd xhci_pci xhci_pci_renesas ehci_hcd +ehci_pci ohci_hcd ohci_pci uhci_hcd dwc2 cdc_acm usb_storage uas`. + +## Workflow + +1. `sudo usb_dyndbg.sh on usbcore <hcd-module>` — `usbcore` for enumeration/hub + logic, plus the controller module (`lsusb -t` shows the driver per bus). +2. Reproduce (replug / re-enumerate / rerun the failing test) while following + `sudo dmesg -w` (or grab `sudo dmesg | tail` afterwards). +3. `sudo usb_dyndbg.sh off ...` — leaving it on floods the log and skews timing. + +Pair with the `usbmon` skill: usbmon for what crossed the bus, dynamic debug for +why the host reacted. For a wedged device/bus use the `usb-recover` skill. + +Requires `CONFIG_DYNAMIC_DEBUG` and mounted debugfs (standard on distro kernels). diff --git a/.claude/skills/usb-debug/scripts/usb_dyndbg.sh b/.claude/skills/usb-debug/scripts/usb_dyndbg.sh new file mode 100755 index 000000000..0dc880469 --- /dev/null +++ b/.claude/skills/usb-debug/scripts/usb_dyndbg.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +# usb_dyndbg.sh — toggle kernel dynamic-debug on USB host drivers; run with sudo. +# Flips +p/-p only on an allowlisted set of USB modules, so it can't reach +# arbitrary kernel debug or unrelated subsystems. +# +# Usage: +# sudo usb_dyndbg.sh on <module>... # enable +p (e.g. usbcore xhci_hcd) +# sudo usb_dyndbg.sh off <module>... # disable -p +# sudo usb_dyndbg.sh status [module] # show enabled sites (or one module's sites) +set -euo pipefail + +CTL=/sys/kernel/debug/dynamic_debug/control +# Allowlist: USB host-controller + core + common host class drivers. +ALLOW='usbcore xhci_hcd xhci_pci xhci_pci_renesas ehci_hcd ehci_pci ohci_hcd ohci_pci uhci_hcd dwc2 cdc_acm usb_storage uas' + +die() { echo "usb_dyndbg: $*" >&2; exit 1; } +usage() { + echo "usage: usb_dyndbg.sh {on|off} <module>... modules: $ALLOW" >&2 + echo " usb_dyndbg.sh status [module]" >&2 + exit 2 +} +allowed() { local m; for m in $ALLOW; do [ "$m" = "$1" ] && return 0; done; return 1; } + +[ -e "$CTL" ] || die "dynamic_debug unavailable (need CONFIG_DYNAMIC_DEBUG + debugfs mounted)" + +action=${1:-}; shift || true +case "$action" in + on|off) + [ "$#" -ge 1 ] || usage + flag='+p'; [ "$action" = off ] && flag='-p' + for m in "$@"; do allowed "$m" || die "module not allowlisted: $m"; done + for m in "$@"; do echo "module $m $flag" > "$CTL"; echo "dynamic debug $action: $m"; done + ;; + status) + m=${1:-} + if [ -n "$m" ]; then + allowed "$m" || die "module not allowlisted: $m" + grep -E "\[$m\]" "$CTL" || echo "(no sites for $m)" + else + grep -E '=p( |$)' "$CTL" || echo "(no print sites enabled)" + fi + ;; + *) + usage + ;; +esac diff --git a/.claude/skills/usb-recover/SKILL.md b/.claude/skills/usb-recover/SKILL.md new file mode 100644 index 000000000..7f3e86632 --- /dev/null +++ b/.claude/skills/usb-recover/SKILL.md @@ -0,0 +1,91 @@ +--- +name: usb-recover +description: Use when a USB device or fixture on the ci HIL rig is stuck, hung, not enumerating, or wedged after a failed flash or test, or when processes touching USB (testusb, JLinkExe, uhubctl, libusb tools) start hanging in D state. +--- + +# USB Recovery on the HIL Rig + +Run this skill's `scripts/usb_recover.sh` with `sudo` (abbreviated to +`usb_recover.sh` in the examples below). It wraps four sysfs reset actions plus +a resolver: + +```bash +sudo usb_recover.sh resolve /dev/ttyACM3 # /dev node -> busport (e.g. 3-4.7); also ttyUSB*, sg* +sudo usb_recover.sh authorized <busport> # deauthorize+reauthorize: re-enumerate, no VBUS cut +sudo usb_recover.sh rebind <busport> # usb driver unbind+bind: re-probe +sudo usb_recover.sh pci-rebind <pciaddr> # whole HCD controller unbind+bind, e.g. 0000:02:00.0 +sudo usb_recover.sh pci-reset <pciaddr> # PCI function-level reset: kills URBs at HW level, no device lock +sudo usb_recover.sh pci-bind <pciaddr> [drv] # re-bind a DRIVERLESS controller (auto-tries xHCI drivers) +``` + +## Decide first: is anything stuck in D state? + +```bash +ps -eo pid,stat,wchan:30,cmd | awk '$2 ~ /D/' +``` + +**If yes** (uninterruptible sleep, typically a usbfs ioctl — e.g. testusb inside +`usb_sg_wait`): run `pci-reset` and NOTHING ELSE first: + +```bash +sudo usb_recover.sh pci-reset <pciaddr> +``` + +FLR kills the URBs at the hardware level without taking the per-device lock; +the ioctl then returns and the convoy unwinds on its own. + +**Not every controller supports FLR.** The Renesas uPD720201 (`0000:01:00.0`) +has no reset method — `pci-reset` fails with `Inappropriate ioctl for device` +(ENOTTY). On those, there is no clean D-state cure short of a **reboot**; do NOT +fall through to `pci-rebind` (see next). + +**`pci-rebind` can strand the controller driverless.** Its unbind succeeds but, +with a D-state process still holding a URB, the *re-bind* hangs — leaving the +PCI device with **no driver** (`/sys/bus/pci/devices/<addr>/driver` gone) and the +whole controller's fixtures offline. A second `pci-rebind` then dies with "no +driver bound". Recover with `pci-bind <addr>` (re-attaches the xHCI driver); +if that also hangs because the D-state URB is unkillable, **reboot** is the only +cure. The Renesas binds via `xhci-pci-renesas` (firmware loader), others via +`xhci_hcd` — `pci-bind` auto-tries both, or pass the driver explicitly. + +**Ordering is critical.** `authorized`/`rebind`/`pci-rebind` all take the +per-device lock the stuck ioctl holds — they block and join the convoy, and +soon every libusb tool (uhubctl, JLinkExe) hangs too. Worse, a blocked +`pci-rebind` grabs the PCI device lock on its way in, which `pci-reset` also +needs: once a rebind has been attempted and is stuck, even FLR deadlocks and +**only a rig reboot recovers**. pci-reset first (if supported), and never +`pci-rebind` a D-state wedge. + +**If no** (device merely dead or silent), escalate gently: + +1. `authorized <busport>` — re-enumerates just that device +2. `rebind <busport>` — re-probe; also worth trying on the parent hub's busport +3. `pci-rebind <pciaddr>` — last resort: bounces every fixture on that controller + +## Finding targets + +```bash +grep -l <SERIAL> /sys/bus/usb/devices/*/serial # serial -> busport (dir name) +readlink -f /sys/bus/usb/devices/usb<N> # bus N -> its PCI addr in the path +``` + +Rig layout: buses 3+4 = `0000:02:00.0` (main fixture tree: J-Links, ST-Links, +WCH-Links, DUTs); buses 9+12 = `0000:01:00.0`, the only ones with uhubctl port +power (ganged VBUS: `sudo uhubctl -l 9 -a cycle`). Hubs on buses 1-4 have no +port power switching — uhubctl reports "No compatible devices" there. + +## Common mistakes + +- `resolve` takes a **/dev node**, not a busport or serial ("no such device node"). +- `authorized`/`rebind` take a **busport** (`3-4.7`); `pci-rebind`/`pci-reset` + take a **PCI addr**. +- Command produces no output and doesn't return → it is blocked on the device + lock: a D-state holder exists; see above. +- Trying `pci-rebind` on a D-state hang — its re-bind hangs and strands the + controller **driverless**; recover with `pci-bind <addr>`, or reboot if the + D-state URB is unkillable. Use `pci-reset` (if supported) for D-state, never + `pci-rebind`. +- Running `pci-reset` on a controller without FLR support (Renesas) → ENOTTY; + no recovery but reboot. +- A J-Link reset (`r; go`) does not disconnect a wedged DUT from the host: the + DWC2 soft-connect pullup stays up through a core halt, so stuck URBs stay stuck. diff --git a/.claude/skills/usb-recover/scripts/usb_recover.sh b/.claude/skills/usb-recover/scripts/usb_recover.sh new file mode 100755 index 000000000..35bd4c784 --- /dev/null +++ b/.claude/skills/usb-recover/scripts/usb_recover.sh @@ -0,0 +1,111 @@ +#!/usr/bin/env bash +# usb_recover.sh — USB recovery helper for the HIL rig; run with sudo. Writes only +# to the specific sysfs control files below; arg regexes block path traversal. +# +# Usage: +# sudo usb_recover.sh authorized <busport> # e.g. 3-2 -> deauthorize+reauthorize (re-enumerate, NO VBUS cut) +# sudo usb_recover.sh rebind <busport> # e.g. 3-2 -> usb driver unbind+bind (re-probe) +# sudo usb_recover.sh pci-rebind <pciaddr> # e.g. 0000:01:00.0 -> HCD unbind+bind (WHOLE controller) +# sudo usb_recover.sh pci-reset <pciaddr> # e.g. 0000:01:00.0 -> PCI function-level reset: kills URBs at +# # HW level WITHOUT the device lock; the only cure when a process +# # is stuck in D state (usbfs ioctl) and unbind paths would convoy +# sudo usb_recover.sh pci-bind <pciaddr> [driver] # bind a DRIVERLESS controller (e.g. after a pci-rebind +# # whose re-bind hung and left it unbound). Auto-tries the xHCI +# # drivers (xhci-pci-renesas, xhci_hcd) unless one is named. +# sudo usb_recover.sh resolve <devnode> # e.g. /dev/ttyACM3 -> print its <busport> (no privilege needed) +set -euo pipefail + +USBPATH_RE='^[0-9]+-[0-9]+(\.[0-9]+)*$' +PCI_RE='^[0-9a-fA-F]{4}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}\.[0-9]$' +DRIVER_RE='^[A-Za-z0-9_-]+$' + +die() { echo "usb_recover: $*" >&2; exit 1; } +usage() { grep -E '^# sudo usb_recover' "$0" >&2; exit 2; } + +# Refuse to touch a PCI function that is not a USB controller (class 0x0c03xx), so a stray or +# mistyped BDF can't unbind/reset an unrelated device (storage, NIC) on a shared HIL host. +require_usb_controller() { + local addr=$1 cls + cls=$(cat "/sys/bus/pci/devices/$addr/class" 2>/dev/null) || die "no such pci device: $addr" + [[ "$cls" =~ ^0x0c03 ]] || die "$addr is not a USB controller (class $cls); refusing" +} + +# Resolve a /dev node (ttyACMx, ttyUSBx, sgN, ...) up to its USB device busport. +resolve() { + local node=$1 syspath dev + [ -e "$node" ] || die "no such device node: $node" + syspath=$(udevadm info -q path -n "$node" 2>/dev/null) || die "udevadm failed for $node" + dev="/sys$syspath" + while [ "$dev" != "/sys" ] && [ -n "$dev" ]; do + if [ -e "$dev/busnum" ] && [ -e "$dev/devnum" ] && [ -e "$dev/authorized" ]; then + basename "$dev"; return 0 + fi + dev=$(dirname "$dev") + done + die "could not find parent USB device for $node" +} + +action=${1:-}; target=${2:-} +[ -n "$action" ] && [ -n "$target" ] || usage + +case "$action" in + resolve) + resolve "$target" + ;; + authorized) + [[ "$target" =~ $USBPATH_RE ]] || die "bad usb path: $target" + d="/sys/bus/usb/devices/$target" + [ -e "$d/authorized" ] || die "no such usb device: $target" + echo 0 > "$d/authorized"; sleep 1; echo 1 > "$d/authorized" + echo "re-authorized $target" + ;; + rebind) + [[ "$target" =~ $USBPATH_RE ]] || die "bad usb path: $target" + [ -e "/sys/bus/usb/devices/$target" ] || die "no such usb device: $target" + echo "$target" > /sys/bus/usb/drivers/usb/unbind; sleep 1 + echo "$target" > /sys/bus/usb/drivers/usb/bind + echo "rebound $target" + ;; + pci-rebind) + [[ "$target" =~ $PCI_RE ]] || die "bad pci addr: $target" + require_usb_controller "$target" + [ -e "/sys/bus/pci/devices/$target/driver" ] || die "no driver bound to $target" + drv=$(basename "$(readlink -f "/sys/bus/pci/devices/$target/driver")") + echo "$target" > "/sys/bus/pci/drivers/$drv/unbind"; sleep 1 + echo "$target" > "/sys/bus/pci/drivers/$drv/bind" + echo "rebound pci $target ($drv)" + ;; + pci-bind) + # Re-attach a driver to a controller left DRIVERLESS (e.g. a pci-rebind whose re-bind hung). + [[ "$target" =~ $PCI_RE ]] || die "bad pci addr: $target" + require_usb_controller "$target" + [ -e "/sys/bus/pci/devices/$target" ] || die "no such pci device: $target" + [ -e "/sys/bus/pci/devices/$target/driver" ] && die "$target already has a driver bound" + drv=${3:-} + if [ -n "$drv" ]; then + [[ "$drv" =~ $DRIVER_RE ]] || die "bad driver name: $drv" + [ -e "/sys/bus/pci/drivers/$drv/bind" ] || die "no such pci driver: $drv" + echo "$target" > "/sys/bus/pci/drivers/$drv/bind" + echo "bound pci $target ($drv)" + else + # Auto-try the xHCI drivers (Renesas uPD720201 uses xhci-pci-renesas; others xhci_hcd). + for cand in xhci-pci-renesas xhci_hcd; do + [ -e "/sys/bus/pci/drivers/$cand/bind" ] || continue + if echo "$target" > "/sys/bus/pci/drivers/$cand/bind" 2>/dev/null; then + echo "bound pci $target ($cand)"; exit 0 + fi + done + die "could not bind $target with a known xHCI driver; pass the driver explicitly" + fi + ;; + pci-reset) + [[ "$target" =~ $PCI_RE ]] || die "bad pci addr: $target" + require_usb_controller "$target" + [ -e "/sys/bus/pci/devices/$target/reset" ] || die "no reset support on $target" + echo 1 > "/sys/bus/pci/devices/$target/reset" + echo "flr-reset pci $target" + ;; + *) + usage + ;; +esac diff --git a/.claude/skills/usbtest/SKILL.md b/.claude/skills/usbtest/SKILL.md new file mode 100644 index 000000000..8146d94e4 --- /dev/null +++ b/.claude/skills/usbtest/SKILL.md @@ -0,0 +1,123 @@ +--- +name: usbtest +description: Use when running, debugging, or porting the Linux usbtest/testusb battery (examples/device/usbtest, cafe:4010) — device "did not bind", SET_CONFIGURATION fails, a case fails with errno 110/32/5/71, toggle-clear/halt/unlink/iso failures, iso packets dropped, or a new MCU/DCD needs the full 30/30 sign-off. +--- + +# usbtest — porting & debugging the Linux kernel USB battery + +## Overview + +`examples/device/usbtest` is the device-side peer of the Linux kernel's `usbtest.ko`/`testusb` +(gadget-zero source/sink protocol): 30 cases over bulk, EP0, interrupt, and isochronous, including +halt, data-toggle, and unlink storms. It is the most adversarial exerciser a DCD gets — every port +so far surfaced at least one real driver bug. Host runner: `test/hil/usbtest.py`; HIL integration +runs it per board and reports `✅ 30/30` cells. + +**Core principle: the battery is a DCD test, not a firmware test.** When a case fails, suspect the +DCD path it exercises (table below), reproduce that one case, and root-cause on hardware before +changing anything (`superpowers:systematic-debugging`). One variable at a time; a fix is proven by +the failing case passing *and* the full battery still at 30/30 across reflash cycles. + +## Run + +```bash +# build (cmake); descriptor sizes auto-adapt per MCU via src/usb_descriptors.h + src/tusb_config.h +cd examples/device/usbtest && cmake -B build -DBOARD=<board> -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel && cmake --build build +# flash, wait ~3-5 s for enumeration to settle, then: +python3 test/hil/usbtest.py --serial <uid> --keep-binding # full battery for the advertised tier +python3 test/hil/usbtest.py --serial <uid> --keep-binding --tests 29 # one case +``` + +- **Always `--keep-binding`**: the cleanup unbind path has wedged host xHCIs (`usb_hcd_alloc_bandwidth`). +- Always settle a few seconds after flashing — enumeration can bounce once; testusb into the gap sees + the device drop mid-case. +- On a CI rig: stop the actions runner before touching hardware; restart after. Never run two + batteries concurrently (hil_test.py serializes them; concurrent batteries have hard-frozen a rig + via a fatal PCIe error on a VFIO-passed xHCI). + +## Porting ladder — new MCU/DCD to 30/30 + +1. **Tier 1 (bulk)**: set `USBTEST_TIER 1`, get enumeration + cases 0,9,10 (EP0) + 1–8,17–20,27,28 + solid. EP0 correctness first — everything else reports through it. +2. **Tier 2 (ctrl_out 14/21)**, **tier 3 (interrupt 25/26)**, **tier 4 (iso 15/16/22/23)** — raise + the tier only when the layer below is clean; run the *full* battery after each layer. +3. **Fit the endpoints**: tier 4 needs 6 endpoints + EP0. Small parts need per-MCU mps/epbuf + overrides in `src/usb_descriptors.h` (`USBTEST_INT/ISO_EP_MPS_FS`) and `src/tusb_config.h` + (`CFG_TUD_VENDOR_TX_EPSIZE`) — follow the existing CH32/LPC11 patterns. Parts that can't fit go + in `skip.txt`. +4. **Sign-off = reliability, not one pass**: 3–10 full flash→battery cycles. One 30/30 proves + nothing on a flaky bring-up; deterministic partial counts (e.g. exactly 1-in-8 lost) are a + signature, not noise — chase them. +5. Register the board in `test/hil/tinyusb.json` so the HIL suite runs it. + +## Case → DCD subsystem map + +| Failing case(s) | Exercises | First suspect | +|---|---|---| +| 9, 10 | EP0 control storms | EP0 state machine, ZLP/status stage, control starvation under load | +| 1–8, 17–20, 27, 28 | bulk source/sink, sg, perf | FIFO handling, multi-packet, ZLP tolerance | +| 11, 12, 24 | URB unlink mid-transfer | abort/close paths leaving state half-armed | +| 13 | set/clear halt | stall must kill the transfer; halt on armed IN must flush the TX FIFO | +| **29** | clear-halt on an **armed, un-halted** ep | **the classic**: `dcd_edpt_clear_stall` resets toggle but disarms the queued receive → NAKs forever, errno 110. Fix: reset toggle to DATA0 *and* re-arm/preserve the pending transfer. Found independently on rp2040, fsdev, ch32_usbhs, rusb2 | +| 14, 21 | vendor EP0 write/readback | multi-packet control-OUT chunking, DCP flow control | +| 25, 26 | interrupt src/sink | usually free once bulk works | +| 15, 16, 22, 23 | isochronous | see iso rules below | + +## Iso rules (most-violated contract) + +- **DATA0-only in BOTH directions** at FS — never run bulk-style toggle logic on an iso endpoint + (manual-toggle parts: skip the ISR toggle flip for iso IN *and* the toggle-mismatch drop for iso + OUT). Symptom of violating it: exactly every-other packet lost. +- **No handshake** — iso never NAKs/STALLs; parts with response fields use their "no response" + encoding (e.g. NYET on WCH). +- `dcd_edpt_iso_alloc`/`iso_activate` **must not be stubs returning false** — usbd fails the + interface open and the kernel logs "did not bind"/SET_CONFIG times out. If a DCD refuses iso + "because the hardware can't", **verify against the datasheet — the manual outranks the code + comment** (two "no iso support" claims in this tree were false, incl. a per-endpoint exception + the RM documents for one endpoint number only). +- A multi-packet iso IN submit is legal: the DCD streams it one packet per frame, refilling in the + ISR. Slow cores may need double-buffered iso to make the frame deadline. + +## Debug ladder (escalate in order) + +| errno | Meaning | +|---|---| +| 110 | timeout — endpoint NAKing forever / device wedged | +| 32 | EPIPE — unexpected STALL | +| 5 | EIO — iso packet errors (check `dmesg`: "N errors out of M") | +| 71 | EPROTO — device answered wrong / too slow (after HC retries) | + +1. `usbtest.py` per-case output + its captured `dmesg` (`TEST n` markers bracket each case). +2. **usbmon** (`usbmon` skill): URB-level ground truth. **It cannot show data toggles or NAKs** — + a toggle desync and a dead endpoint look identical (Submits without Completes); distinguish + device-side with GDB. +3. **On-device gdb/openocd**: read the EP control registers and DCD structs at the hang. +4. Heisenbugs (vanish under logging): RAM ring-buffer trace dumped over openocd; for silent lockups + JLink PC-sampling (`halt`+`regs` repeatedly — a pinned PC names the spin). +5. **Cross-check the reference manual** (calibre library) before changing any register-level code — + per CLAUDE.md, and because comments/assumptions in DCDs have been wrong about hardware caps. +6. Check the vendor's **silicon errata** early for timing/DMA hangs (an unimplemented erratum + workaround caused a case-10 hang on one port). + +## Traps that pass gcc/desk review but fail elsewhere + +- `TUD_OPT_HIGH_SPEED` is a **compile-time capability, not the live speed**: the FS config + descriptor (and OTHER_SPEED) must use FS-legal sizes (int ≤ 64, iso IN+OUT ≤ 1023 B/frame) even + on HS builds — use separate `_FS`/`_HS` descriptor macros. +- Unused `static inline` helpers: clang `-Wunused-function` and IAR `Pe177` error where gcc stays + quiet → `TU_ATTR_UNUSED`. +- A symbol referenced only inside naked asm is invisible to LTO and gets dropped in `-flto` make + builds → keep a `TU_ATTR_USED` C reference to it. +- Nested USB IRQs on cores with hardware context stacks (QingKe HWSTK): plain + `__attribute__((interrupt))` corrupts the return — use naked handlers relying on the HW stack. +- Dedicated USB RAM budgets (PMA/USB-RAM) differ per part *and* per build system section placement: + check the link map, not just that it builds. + +## Red flags — stop and re-examine + +- "One pass = done" → run reflash cycles. +- "The DCD comment says the hardware can't" → open the datasheet. +- "usbmon shows no toggle problem" → usbmon can't see toggles. +- "It works on gcc" → clang/IAR/LTO/make still pending. +- "Fixed iso IN" → apply the same exemption to iso OUT (toggle logic is symmetric). +- A clean single-board run does not validate concurrent/fleet behavior — batteries serialize. |