diff options
326 files changed, 27808 insertions, 3975 deletions
diff --git a/.circleci/config.yml b/.circleci/config.yml index 66799910d..48fa87899 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -15,7 +15,7 @@ jobs: - run: name: Set matrix command: | - MATRIX_JSON=$(python .github/workflows/ci_set_matrix.py) + MATRIX_JSON=$(python .github/scripts/ci_set_matrix.py) echo "MATRIX_JSON=$MATRIX_JSON" BUILDSYSTEM_LIST=( diff --git a/.claude/agents/driver-reviewer.md b/.claude/agents/driver-reviewer.md index f45eca03e..9ce8b621f 100644 --- a/.claude/agents/driver-reviewer.md +++ b/.claude/agents/driver-reviewer.md @@ -1,7 +1,7 @@ --- name: driver-reviewer description: Review one TinyUSB driver directory or one diff against one review dimension (correctness, ISR safety, datasheet/errata conformance, style) with coverage-first structured findings; or adversarially verify a single finding / fix. Read-only. -tools: Bash, Read, Grep, Glob +tools: Bash, Read, Grep, Glob, Skill model: opus --- @@ -9,7 +9,7 @@ You review exactly the scope given in your prompt (one driver directory, or one ## Datasheets & errata -For register-use review, find the MCU/USB-IP reference manual in `$HOME/Documents/calibre-library` — and ALSO search the library for the part's errata / silicon-bug sheets (search terms: "errata" plus the MCU or USB-IP name). When the code touches behavior an erratum covers, verify the driver implements the documented workaround; a missing erratum workaround IS a finding (severity by impact — the nRF52 erratum-199 DMA class is major). If a needed document is absent, mark affected findings `confidence: "low"` and name the missing document in `why`. +For register-use review, find the MCU/USB-IP reference manual with the `read-doc` skill — `python3 .claude/skills/read-doc/search.py <keywords>`, never `find`/`grep` over the library tree — and ALSO search for the part's errata / silicon-bug sheets (search terms: "errata" plus the MCU or USB-IP name). When the code touches behavior an erratum covers, verify the driver implements the documented workaround; a missing erratum workaround IS a finding (severity by impact — the nRF52 erratum-199 DMA class is major). If a needed document is absent, mark affected findings `confidence: "low"` and name the missing document in `why`. ## Reporting discipline diff --git a/.claude/agents/hil-operator.md b/.claude/agents/hil-operator.md index c48ceb8bd..ebc9251cc 100644 --- a/.claude/agents/hil-operator.md +++ b/.claude/agents/hil-operator.md @@ -7,9 +7,9 @@ model: sonnet You operate physical USB test hardware. These repo skills are your source of truth — read the relevant one BEFORE acting: -- `.claude/skills/hil/SKILL.md` — run `hostname` first (host `ci` = local mode with `test/hil/tinyusb.json`; host `htpc` = local `local.json` or remote via `test/hil/hil_ci.sh`); the board lock protocol; exact `hil_test.py` invocations. -- `.claude/skills/usb-recover/SKILL.md` — only when a device/fixture is wedged or processes hang in D state. -- `.claude/skills/usb-debug/SKILL.md` — only when you need to explain WHY the host rejected a device (dmesg analysis). +- `.claude/skills/hil/SKILL.md` — run `hostname` first (host `ci` = local mode with `test/hil/tinyusb.json`; host `tusb` = local mode with `test/hil/hfp.json`; any other host (dev PC) = local `local.json` or remote via `test/hil/hil_ci.sh`); the board lock protocol; exact `hil_test.py` invocations. +- `.claude/skills/usb-kernel-recover/SKILL.md` — only when a device/fixture on the rig's Linux host is wedged or processes hang in D state. +- `.claude/skills/usb-kernel-debug/SKILL.md` — only when you need to explain WHY the Linux kernel rejected a device (dmesg analysis). ## Board lock protocol (CI runs concurrently — NEVER stop the actions-runner) @@ -18,19 +18,23 @@ The GitHub Actions runner keeps running during your work. Per-board flock locks - `python3 test/hil/hil_test.py ...` runs: do NOT pre-hold those boards — `hil_test.py` self-locks each board for its flash+test and would fail fast with `board locked` against your own hold. - ANY other hardware action (JLinkExe/openocd/GDB, manual flash, usbtest.py, serial poking): hold first, release when done — release is mandatory cleanup (a crashed holder auto-releases via kernel flock, but do not rely on it): ```bash - python3 test/hil/board_lock.py hold <board...> --reason "<task>" + python3 test/hil/helper/hil_lock.py hold <board...> --reason "<task>" # ... hardware work ... - python3 test/hil/board_lock.py release <board...> + python3 test/hil/helper/hil_lock.py release <board...> ``` -- Rig-wide operations (uhubctl power cycling, pci-rebind — they renumber buses): `python3 test/hil/board_lock.py hold --all --reason "<why>"` first. -- If a lock is already held by someone else: report holder/reason (`board_lock.py status`) — never force, never kill the holder. If the holder's reason is `hil_test.py`, that is a concurrent CI job mid-test on the board: waiting a few minutes and retrying once is appropriate when your task allows; otherwise return the holder info so the orchestrator can ask the user. +- Rig-wide operations (uhubctl power cycling, controller resets — they renumber buses): `python3 test/hil/helper/hil_lock.py hold --all --reason "<why>"` first. +- If a lock is already held by someone else: report holder/reason (`hil_lock.py status`) — never force, never kill the holder. If the holder's reason is `hil_test.py`, that is a concurrent CI job mid-test on the board: waiting a few minutes and retrying once is appropriate when your task allows; otherwise return the holder info so the orchestrator can ask the user. - You cannot ask the user anything. Bypassing a lock (`HIL_NO_BOARD_LOCK=1`, or proceeding with manual hardware work despite a held lock) is allowed ONLY when your prompt explicitly states the user authorized forcing. ## Hard rules -- HIL runs take 2–5 min per board: use Bash timeouts >= 20 min (1200000 ms) and NEVER cancel early. +- HIL runs take 2-5 min per board, but a stuck fleet runs to `HIL_POOL_TIMEOUT` — 60 min + unless the env pins it; the run logs its guard in the startup line. That far exceeds the + Bash tool's 10 min foreground cap: run it in the background and wait + for the completion notification. A foreground timeout kills the run before hil_test.py + can write its report. NEVER cancel early. - One hardware action at a time. You are never run concurrently with another hil-operator. -- On test failure: retry once with `-v -r 1` appended (one verbose attempt for diagnosis — the first run already did the flake-retries). If a board/fixture stops enumerating or tools hang in D state, consult usb-recover and capture `dmesg | tail -50` into `detail`; set `wedged` true. +- On test failure: retry once with `-v -r 1` appended (one verbose attempt for diagnosis; a usbtest battery that produced per-case verdicts is NOT auto-retried, so its result already stands). If a board/fixture stops enumerating, or a tool of YOURS hangs in D state, consult usb-kernel-recover and capture `dmesg | tail -50` into `detail`; set `wedged` true. A `> **Rig note.**` banner reporting someone else's D-state process is not that — see the hil skill's banner list. ## Output contract diff --git a/.claude/agents/port-dev.md b/.claude/agents/port-dev.md index 76bafb39b..77a28bafa 100644 --- a/.claude/agents/port-dev.md +++ b/.claude/agents/port-dev.md @@ -16,7 +16,7 @@ You implement exactly one specified change in one assigned scope (a directory un ## Datasheets -When changing dcd/hcd register logic, cross-check the MCU reference manual / datasheet / programming guide in `$HOME/Documents/calibre-library` (search by MCU or USB-IP name). If the document is missing, say so in `notes` and do NOT guess register semantics. +When changing dcd/hcd register logic, cross-check the MCU reference manual / datasheet / programming guide with the `read-doc` skill — `python3 .claude/skills/read-doc/search.py <MCU or USB-IP name>`, never `find`/`grep` over the library tree. If the document is missing, say so in `notes` and do NOT guess register semantics. ## Finish checklist (in order) diff --git a/.claude/agents/pr-monitor.md b/.claude/agents/pr-monitor.md index 7dba91fea..77777b0fb 100644 --- a/.claude/agents/pr-monitor.md +++ b/.claude/agents/pr-monitor.md @@ -9,7 +9,7 @@ You triage exactly one PR (number given in your prompt) using `gh`. You never mo ## CI triage -1. `gh pr checks <N>`. If checks are running and your prompt says to wait, use `gh pr checks <N> --watch` with a Bash timeout >= 30 min. +1. `gh pr checks <N>`. If checks are running and your prompt says to wait, run `gh pr checks <N> --watch` as a BACKGROUND Bash task (the foreground timeout is capped at 10 min). 2. For each failing check, find its run and read the failure: `gh run view <run-id> --log-failed | head -150`. 3. Classify each failure: - **infra/flake**: runner lost communication, network/DNS timeouts, artifact 404, docker pull/rate-limit errors, cancelled-by-timeout with no test output. diff --git a/.claude/agents/target-debugger.md b/.claude/agents/target-debugger.md new file mode 100644 index 000000000..1b6931307 --- /dev/null +++ b/.claude/agents/target-debugger.md @@ -0,0 +1,73 @@ +--- +name: target-debugger +description: Root-cause one USB misbehavior on real HIL hardware by instrumenting the TinyUSB target — device or host stack — with TU_LOG/RTT, RAM ring-buffer trace, GDB autopsy, J-Link PC-sampling, correlated with capture from the link's other end (Linux PC host, another TinyUSB board, or a Linux gadget peer) and the wire. Long serial debug loop under one held board lock; strictly one instance. Produces a diagnosis with on-target evidence (plus a candidate fix when one emerges), never a merged patch. +model: opus +--- + +You debug one failing USB behavior on one physical board until you can name the +mechanism — or report exactly what you ruled out. The target may run the device +stack, the host stack, or both; its link peer may be the Linux PC, another +TinyUSB board, or a Linux gadget (e.g. a Raspberry Pi) — pick capture channels +by which end runs Linux, not by habit. Resolve the board's family first +(`ls -d hw/bsp/*/boards/<board>`): Espressif boards are a different backend +entirely — esp-target-debug is your primary playbook there; every other +family uses target-debug's probe recipes directly. These repo skills (each at +`.claude/skills/<name>/SKILL.md`) are your source of truth; read the relevant +one BEFORE acting: + +| Skill | Use for | +|--------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| target-debug | primary playbook — technique choice by intrusiveness, channel choice by link topology, capture recipes, bp/wp budget + cost model, vector catch + fault autopsy, SWO trace, GDB autopsy, rig warnings | +| hil | host/config selection, board lock protocol, `hil_test.py` invocation | +| esp-target-debug | PRIMARY playbook for Espressif boards — built-in USB-Serial-JTAG attach, the PHY map that decides whether JTAG exists, FreeRTOS threads via ESP_RTOS; target-debug still supplies the methodology | +| usbmon | Linux-host URB capture; only when a Linux PC is the link's host (default posture: dual-side, both ends simultaneously) | +| usb-sniffer | wire-level capture (hardware tap): host can't see the bus, usbmon vs target logs disagree, or TinyUSB is the host (no usbmon anywhere) | +| etm-trace | instruction-level ETM trace via SEGGER J-Trace (exact execution history, profile, coverage) when sampled PCs and logs cannot resolve the mechanism. Requires the J-Trace physically wired to THIS board (supported boards: the skill's boards.md) — use only when your prompt states the board is trace-wired or the user asked for it; otherwise name it in `notes` as the next technique | +| usb-kernel-debug | why the Linux kernel acted (dmesg/dynamic debug); PC host or a Linux gadget peer's device side | +| usb-kernel-recover | only when the DUT or fixture wedges the rig PC's Linux host stack | + +## The loop (deliberately serial — no fan-out) + +hypothesis → least-intrusive technique that can test it → instrument → build → +flash → trigger the failing case → capture both sides → correlate → refine. +One hypothesis per cycle. A disproven hypothesis is progress — record it and +what disproved it. If instrumentation makes the bug vanish, that IS a finding +(timing-sensitive): move DOWN in intrusiveness, not up. + +## Diagnosis standard + +A theory becomes a diagnosis only when (a) captured evidence directly shows the +mechanism, or (b) a change validated against the ORIGINAL failing case flips it +on hardware. A plausible fix that "should" explain it counts for nothing until +the original case passes with it and fails without it. Stop and hand back a +partial diagnosis when two consecutive instrument→capture cycles yield no new +evidence: report what was ruled out, the strongest surviving hypothesis, and +the next technique you would try. + +## Lock discipline + +- Hold the board lock for the WHOLE session (`python3 test/hil/helper/hil_lock.py + hold <board> --reason "target debug: <bug>"`). Multi-hour holds are fine; never stop the + actions-runner. Locks held by others: report holder/reason, never force + unless your prompt states the user authorized it. +- `hil_test.py` self-locks: release your hold before any `hil_test.py` run, + re-hold immediately after. +- You cannot ask the user anything mid-session. + +## Hard rule — fix stays, probe goes, re-verify clean + +Instrumentation is temporary. Before releasing the lock at session end: +1. Revert every instrumentation change (ring buffers, extra logging, temporary + tier/skip edits). The candidate fix, if one emerged, stays in the working + tree — uncommitted. +2. Rebuild clean (fix only, no probes) and re-run the original failing case on + it — `fixVerified` means verified on THIS build, not an instrumented one. +3. Reflash pristine firmware so the next CI run inherits nothing. +Anything you could not revert or verify goes in `notes`, explicitly. + +## Output contract + +Your final message is parsed by a program. Return ONLY this JSON — no prose, +no code fences: + +{"board": "...", "bug": "<one-line original failing case>", "diagnosis": "<mechanism, or strongest surviving hypothesis>", "confirmed": true, "ruledOut": ["<hypothesis — what disproved it>"], "evidence": ["<artifact path or capture — what it shows>"], "fixDiffstat": "<git diff --stat, or empty>", "fixVerified": false, "instrumentationReverted": true, "lockReleased": true, "notes": "..."} diff --git a/.claude/skills/code-size/SKILL.md b/.claude/skills/code-size/SKILL.md index f3c51ccfa..0a5305221 100644 --- a/.claude/skills/code-size/SKILL.md +++ b/.claude/skills/code-size/SKILL.md @@ -13,14 +13,7 @@ Compare TinyUSB code size between a base ref (default `master`) and the current | **all examples, one board** | Per-board regression sweep | `-b BOARD` | | **all examples, all CI families (combined)** | Pre-merge full check | `--ci` | -The script handles the full base-vs-branch dance: -1. Creates a temporary git worktree of the base ref under `cmake-metrics/_worktree/`. -2. Builds the base in `cmake-metrics/<board>/base/`. -3. Builds the current tree in `cmake-metrics/<board>/build/`. -4. Runs `tools/metrics.py compare` and writes `cmake-metrics/<board>/metrics_compare.md`. -5. Removes the worktree on exit. - -`--combined` (auto-set by `--ci`) also produces `cmake-metrics/_combined/metrics_compare.md` aggregating across all boards. +The script does the whole base-vs-branch dance itself: a temporary git worktree of the base ref under `cmake-metrics/_worktree/` (removed on exit), base + branch builds under `cmake-metrics/<board>/{base,build}/`, then `tools/metrics.py compare` (report paths under Outputs). ## Choosing arguments @@ -35,29 +28,20 @@ Infer from the user's request: ## Common invocations ```bash -# Single example, one board (linkermap, fastest): +# Single example, one board (linkermap, fastest; add --bloaty for section/symbol breakdown): python3 tools/metrics_compare_base.py -b raspberry_pi_pico -e device/cdc_msc -# Same with bloaty for section/symbol breakdown: -python3 tools/metrics_compare_base.py -b raspberry_pi_pico -e device/cdc_msc --bloaty - -# All examples for one board: +# All examples for one board (repeat -b for several boards): python3 tools/metrics_compare_base.py -b raspberry_pi_pico -# Multiple boards, one combined report: -python3 tools/metrics_compare_base.py -b raspberry_pi_pico -b raspberry_pi_pico2 --combined - # Full CI sweep (first board per arm-gcc family, combined): python3 tools/metrics_compare_base.py --ci - -# Compare against a tag/commit instead of master: -python3 tools/metrics_compare_base.py -b raspberry_pi_pico --base-branch v0.18.0 ``` ## Outputs - **Per-board:** `cmake-metrics/<board>/metrics_compare.md` (and `_<example>.md` when `-e` is set) -- **Combined (with `--combined`/`--ci`):** `cmake-metrics/_combined/metrics_compare.md` +- **Combined (`--combined`, auto-set by `--ci`):** `cmake-metrics/_combined/metrics_compare.md`, aggregating all boards - **Bloaty:** printed to stdout as section + symbol diffs ## Timing diff --git a/.claude/skills/esp-target-debug/SKILL.md b/.claude/skills/esp-target-debug/SKILL.md new file mode 100644 index 000000000..1af9fcf7f --- /dev/null +++ b/.claude/skills/esp-target-debug/SKILL.md @@ -0,0 +1,110 @@ +--- +name: esp-target-debug +description: Use when debugging TinyUSB firmware on Espressif boards (ESP32-S3/P4 on the rig — dcd_dwc2 examples, idf.py builds) with the chips' built-in USB-Serial-JTAG — attach/halt/backtrace, breakpoints, FreeRTOS task lists, console capture — or when JTAG "could not find or open device", the 303a:1001 port vanishes, or the S3's debug port turns into the TinyUSB device. +--- + +# esp-target-debug — Espressif built-in USB-JTAG backend + +Methodology — intrusiveness ladder, board locks, dual-side capture, diagnosis +standards — lives in `target-debug`; this skill is the Espressif backend: a +different gdb, a different openocd (fork), no probe serial (the debugger IS a +USB device), and a PHY story that decides whether JTAG exists at all. +Built-in USB-Serial-JTAG only; external JTAG is a TODO (no rig adapter). + +## The PHY map — decides everything (verified on the rig) + +| Board | USB-SJ vs TinyUSB OTG | JTAG while USB device runs? | +|--------------------------|------------------------------------------------------|-----------------------------| +| espressif_p4_function_ev | separate pins: USB-SJ GPIO24/25 (FS), OTG own HS PHY | **yes — coexist** (verified: 303a:1001 + cafe:4008 simultaneously, gdb attach during live CDC traffic) | +| espressif_s3_devkitm | ONE shared PHY/port | **no** — the same hub port flips 303a:1001 → cafe:4008 as the app boots; openocd fails `esp_usb_jtag: could not find or open device!` | + +Flashing works in ANY PHY state: the rig flashes via the boards' CP2102N UART +bridges (hence `tinyusb.json` esptool uids are CP210x serials, not MACs). The +UART side is also the remote reset: `esptool.py --after hard_reset read_mac`. + +### P4 (Function-EV) notes + +- The board has **no USB-SJ connector** — GPIO24 (D−, white) / GPIO25 (D+, + green) / GND are broken out from header J1 to a rig hub port. Swapped + D+/D− enumerates as `new low-speed USB device` + error -71; correct shows + `new full-speed`. + +### S3 (DevKitM) notes + +- Debugging windows: non-USB firmware (`board_test` — attach/halt/symbol + resolution verified; `usb_new_phy` is absent from the ELF when + `CFG_TUD/TUH_ENABLED` are 0), bootloader/ROM (always stable), or external + JTAG (TODO). +- **Keep-alive quirk (verified)**: with app firmware running and nothing + attached, USB-SJ drops ~4 s after boot (device-side disconnect, then + half-dead `-71` setup failures until reset). Attach a client inside the + window — once it survives the window it stays up. Recover via the UART + reset above. +- PHY mux reference: `RTC_CNTL_RTC_USB_CONF_REG` (0x60008120) bits + `SW_HW_USB_PHY_SEL`/`SW_USB_PHY_SEL` (TRM 10.56); 0 = eFuse/hardware + control (default). `esptool.py read_mem/write_mem` peeks and pokes + registers over plain UART with the chip in download mode. + +## Attach + +```bash +. $HOME/code/esp-idf/export.sh # openocd-esp32, riscv32-/xtensa-esp32s3-elf-gdb, esptool +openocd -c 'set ESP_RTOS FreeRTOS' -f board/esp32p4-builtin.cfg \ + -c 'adapter serial <MAC-with-colons>' & # S3: board/esp32s3-builtin.cfg +riscv32-esp-elf-gdb -batch -ex 'target extended-remote :3333' \ + -ex 'tbreak tud_task_ext' -ex continue -ex bt -ex 'info threads' -ex detach <elf> +# S3 is Xtensa: use xtensa-esp32s3-elf-gdb with the same arguments +``` + +- `adapter serial` = the chip MAC **with colons** — the USB-SJ device's + iSerial exactly as `lsusb -v -d 303a:1001` or + `/dev/serial/by-id/usb-Espressif_USB_JTAG_serial_debug_unit_<MAC>-if00` + prints it. (The `tinyusb.json` esptool uids are the CP2102N *flasher* + serials — a different port; never pass those to openocd.) +- `set ESP_RTOS FreeRTOS` must precede the board cfg: with it, `info threads` + lists every task with name/state/CPU (verified: usbd Running @CPU0, IDLE1 + @CPU1, ...); without it, one bare "Remote target". +- The ELF: `idf.py -B <builddir> -DBOARD=<board> build` under the example + (CLAUDE.md Espressif notes) — symbolized app backtraces verified + (`tud_task_ext` ← `usb_device_task` ← `vPortTaskWrapper`). +- **Attach may reset the target** (a boot-fresh FreeRTOS tick observed on a + minutes-old session). Until pinned down, do NOT trust built-in-JTAG attach + for post-mortem autopsy of a wedged board (`target-debug`'s + attach-and-halt-only rule); capture state via console or treat the reset + as part of the reproduce cycle. +- Halting still stops USB service: the host may drop the DUT during long + halts; after detach the device may need the UART reset to re-enumerate. + +## Scripted-session gotchas (verified) + +- xtensa-gdb batch `continue`/`interrupt` is async-flaky — for scripted + state reads, halt via openocd telnet :4444 first, then attach gdb to the + stopped target. Interactive sessions are unaffected. +- cpu1 debug-logic examination can fail (`OCD_ID = 00000000`) — + `-c 'set ESP_ONLYCPU 1'` degrades to cpu0-only debugging. +- ROM-frame backtraces (`0x4004xxxx` on S3, `0x4fc0xxxx` on P4, all `??`) + mean the core idles in ROM — break in app code (`tbreak tud_task_ext`) + for symbolized frames. + +## Technique mapping (vs the `target-debug` arsenal) + +| target-debug technique | Espressif backend | +|------------------------|-------------------| +| GDB autopsy, bp/wp | same flow via openocd-esp32 :3333; RISC-V triggers (P4) / Xtensa 2 bp + 2 wp (S3) | +| Vector catch | none — breakpoint the panic handler; `mcause`/`mepc`/`mtval` on P4 | +| SWO / DWT data trace | none — apptrace over JTAG is the analog (untested: needs CONFIG_APPTRACE + app init) | +| RTT / TU_LOG | console on **UART0 = the CP2102 flasher tty** by default (verified); USB-SJ console needs sdkconfig `ESP_CONSOLE_USB_SERIAL_JTAG` (untested) | +| FreeRTOS threads | native — `set ESP_RTOS FreeRTOS` (see Attach) | +| verifybin | `esptool.py verify_flash` (untested) | + +## Rig deltas + +- Locks per `hil` skill; reflash-pristine before release applies unchanged. +- One client per USB-SJ: openocd and a terminal on the USB-SJ CDC side + conflict the same way J-Link clients do. + +## TODO — external JTAG (needs hardware) + +S3 JTAG pins GPIO39–42 (MTCK/MTDO/MTDI/MTMS) + any adapter openocd-esp32 +supports (ESP-Prog/FT2232-class); would give S3 debugging under live USB +traffic. Mind `EFUSE_DIS_PAD_JTAG` / JTAG-source strapping. Unverified. diff --git a/.claude/skills/etm-trace/SKILL.md b/.claude/skills/etm-trace/SKILL.md new file mode 100644 index 000000000..43cf6a2a5 --- /dev/null +++ b/.claude/skills/etm-trace/SKILL.md @@ -0,0 +1,203 @@ +--- +name: etm-trace +description: Use when you need instruction-level execution data from real hardware via a SEGGER J-Trace — cycle-accurate hot-function profiling, on-target code coverage, or raw instruction history (e.g. what ran right before a fault/hang) — beyond what logs, GDB, or DWT PC-sampling can answer. Covers unattended (headless) capture and analysis on ETM-capable TinyUSB boards. +--- + +# etm-trace — unattended ETM instruction trace via J-Trace + Ozone + +Streams full instruction (ETM) trace from a board wired to a SEGGER J-Trace, +headlessly: no GUI, scripted end to end. Produces hot-function profile, code +coverage, and optionally the raw instruction history. + +| Skill | Answers | +|-----------------|----------------------------------------------------------------------------| +| `usbmon` | what the host actually exchanged (URBs) | +| `target-debug` | what the target did (logs, driver state, sampled PCs) | +| `usb-sniffer` | what crossed the wire | +| **`etm-trace`** | **exactly which instructions executed, when** (profile, coverage, history) | + +Use `target-debug`'s DWT PC-sampling for a quick statistical profile; use +this skill for exact counts, coverage, or instruction-by-instruction history. + +## Requirements + +- J-Trace on USB (`lsusb -d 1366:1020`) wired to the board's trace header; + select it by J-Link USB **nickname** (this rig: `jtrace`) — never commit + serials. +- **Physical setup is per-board and exclusive** (one J-Trace, moved between + boards; some rigs are fly-wired): unless the user just asked for trace on + this board or your task states it is wired, **confirm with the user** that + the J-Trace is connected to the target before flashing or capturing. +- `ozone` on PATH (≥ V3.38 for the automation socket) and `xvfb-run`. +- Firmware built with **`-DTRACE_ETM=1`** (BSP trace-pin + trace-clock init). +- Boards with a reference `hw/bsp/*/boards/<board>/ozone/*.jdebug` work out of + the box (`ls` that glob for the list); others fall back to `JLINK_DEVICE` + from `board.cmake` + default trace config. Verified boards: `boards.md` in + this skill directory. + +## Rig discipline + +- One probe, one client: quit interactive Ozone/JLinkExe/GDB on the probe + first. Kill only processes you started — if it's held by someone else's + session (check `fuser /dev/bus/usb/<bus>/<dev>`), surface it and ask. The + capture script uses automation port **19201**, never an interactive Ozone's + 19200. +- Hold the board lock (see the `hil` skill): + `python3 test/hil/helper/hil_lock.py hold <board> --reason "etm capture"`. +- Committed `hw/bsp/**/ozone/*.jdebug` are the maintainer's interactive + projects — automation never opens them (Ozone rewrites project files); the + script generates a throwaway project. +- The default capture reflashes and resets the target (`--attach` doesn't). + +## Capture and analyze + +```bash +# 1. Build with trace support: +cd examples && cmake -B cmake-build-<board> -DBOARD=<board> -G Ninja \ + -DCMAKE_BUILD_TYPE=MinSizeRel -DTRACE_ETM=1 . \ + && cmake --build cmake-build-<board> --target <example> + +# 2. Capture (all options + defaults: etm_capture.py --help): +python3 .claude/skills/etm-trace/scripts/etm_capture.py \ + --board <board> --probe jtrace --duration-ms 10000 --out <dir> + +# 3. Analyze (hot functions, coverage, hottest lines, optimization hints): +python3 .claude/skills/etm-trace/scripts/etm_profile.py <dir> --elf <elf> +``` + +Every capture — TinyUSB firmware or vendor demo — goes through +`etm_capture.py`; extend it when a board needs something new, never +hand-roll Ozone drivers. + +Choosing capture flags (semantics in `--help`): +- fresh-boot profile/coverage: defaults (flash + reset + trace from startup) +- narrowing debug on a LIVE target: `--attach` — no reflash/reset (flashed + firmware must match `--elf` and be TRACE_ETM-built) +- raw history: `--trace-csv` (~80 MB/1M instructions) — when sequence/timing + matters, e.g. feeding `--isr` +- stream dies (overflow/unknown-packet): `--no-timestamps`, then reduce the + core clock (`boards.md`); marginal wiring: sweep `--trace-timing`, + isolate lines with `--trace-width`. "capture OK" requires nonzero profile + totals — silence (no trace at all) fails with its own error +- deeper data: `--profile-lines-csv` (hottest lines), `--profile-insts-csv` + (branch bias), `--sample "expr,.."` (data sampling), `--power` (probe-powered + targets only), `--os-plugin` (RTOS timeline), `--trace-only` (experimental, + see Warnings) +- non-TinyUSB targets: `--device` + `--elf`, plus `--jlink-script` when the + firmware doesn't init the trace pins + +First trace on a board — or after any rewiring — is a bring-up, not a plain +capture: follow "Adding a new board" below (vendor example first). + +Analyzer: `--isr ENTRY[,BODY..]` gives ISR min/median/avg/worst from a +`--trace-csv` capture with timestamps (fast-enumerating boards need a short +no-eviction run); `--exclude REGEX` drops idle/poll loops from the load +ranking. + +Outputs in `<dir>`: `code_profile.txt` (run/fetch counts + coverage); on +request `itrace.csv`, `profile_lines.csv`, `profile_insts.csv`, `samples.csv`, +`power.csv`; `session.log` / `ozone_console.log` / `jlink.log` as evidence. + +## Reading results + +- **Load %** = share of instruction **fetches** — Ozone has no per-function + time; time comes only from itrace timestamps (`--isr`, time-share table). +- ISR timing: sub-µs values are approximate (interpolated timestamps — hence + the SysTick calibration); instruction counts are exact. Time-share ≫ + instruction-share = stalled/waiting (e.g. slave-mode FIFO at wire pace). +- "Fully covered" needs both branch directions — 100% is not expected from an + idle run. +- itrace timestamps scale by `VAR_TRACE_CORE_CLOCK` (from the board reference; + `--core-clock` overrides): ordering is exact, absolute times approximate. +- A ms+ "largest gap" or `Trace overflow detected` beyond the startup burst = + lost packets — reduce the core clock or trace a quieter phase. +- One `Invalid trace timestamp` line at `Debug.Halt` is a normal decoder + artifact. +- `Unknown trace data packet … Trace collection stopped!` = stream dead from + that point (the script exits non-zero): retry with `--no-timestamps`, then + reduce the core clock. + +## Timing + +- Capture ≈ `--duration-ms` + 15 s overhead; add ~5 s per 1M instructions with + `--trace-csv`. Bash timeout: duration + 120000 ms. +- Analyzer: < 5 s for a 2M-row itrace.csv. + +## Warnings + +- **Trace starts at `trace_etm_init()`**, not at reset: earlier `board_init()` + code shows as never-executed and Ozone logs `No trace clock present` — both + expected. Trace-from-reset needs a SEGGER J-Link script (`.pex`) instead of + firmware init. +- Never commit capture output (`itrace.csv` can exceed 100 MB) — keep `--out` + in scratchpad/`/tmp`; `*.jdebug.user` files stay untracked. +- The automation socket can't evaluate symbolic constants (`EXPORT_AS_CSV`): + the scripts send numeric/plain commands only — keep it that way when + extending them (UM08025 §6.7). +- Without `xvfb-run`, Ozone opens on `DISPLAY` and steals keyboard focus. + Ozone has no `--help`/`--version` — any such probe opens the GUI; check + with `which ozone` only. +- **`--trace-only` is experimental**: ETM start/stop comparators are scarce + and erratic — low-rate handler windows may silently not record, adjacent + instructions leak in, timestamps are invalid across gaps, and the profile + becomes share-of-traced-stream. Use only for instruction-exact inventories + of high-rate symbols; for ISR timing use full trace + `--isr`. + +## Adding a new board + +Bring-up ladder — each step gates the next: + +1. **Docs before hardware** (`read-doc` skill first, then vendor site): board + manual, schematics, MCU reference manual. Establish the trace clock + source and max — chip side and probe side (J-Trace PRO Cortex-M tops out + at a 150 MHz trace clock) — the pins carrying TRACE_CLK/D0-D3 (read the board's + debug-connector table — boards often route trace on alternate pins), and + required rework (jumpers, solder bridges, 0 Ω resistors to add/remove). + Hunt shared-net hazards: PHYs or other active drivers on trace nets, + boot straps, connector stubs. +2. **Confirm with the user before any hardware change**: present the rework + findings as **[ACTION]** items and wait — the user solders/jumpers, you + verify afterward. +3. **Vendor example before TinyUSB**: fetch SEGGER's trace example for the + same/similar MCU + (<https://www.segger.com/products/debug-probes/j-trace/technology/tested-devices/>) + and run it with `--device <MCU> --elf <demo ELF> --jlink-script + <demo .pex>`. Streaming proves the physical path — and only that: demo + firmware often runs reset-default clocks (the RA6M5 one traces at a few + MHz), so its success says nothing about your target's trace rate. The + example may target a different board (the LPC4357 one is tested on a + Keil MCB4300), so silence isn't final proof — but its J-Link + script/config is often borrowable. +4. **TinyUSB support**: `trace_etm_init()` in the family BSP — mux trace + pins AFTER the final core-clock switch, enable the trace clock, enable + any funnel between ETM and TPIU; committed `ozone/*.jdebug` reference, + plus a `.JLinkScript` declaring off-ROM-table CSTF/TMC/TPIU (addresses + from the vendor demo's script); build with `TRACE_ETM=1`, validate with + `--board <board>`. +5. **Still silent or corrupt?** In order: chip-side register audit (pinmux, + TPIU, ETM, DEMCR — and EVERY funnel in the path; an unprogrammed funnel + reads register-perfect and eats the stream), physically re-seat both + connector ends, then SEGGER's procedure (UM08001): find a stable + `--trace-timing` at `--trace-width 1`, step up to 2, then 4 (sampling + default is +2 ns) — then search the + MCU vendor's application notes and community forums for the chip's trace + recipe: more than one board's fix lived only in a forum thread. +6. **Board note**: add the table row (core clock, TRACECLK pin + max, + width, timing, physical setup, TODO for anything left unvalidated) plus + a caveat bullet — both in `boards.md`. + +## Per-board notes + +Every validated board has a row (config: core clock, TRACECLK, width, timing, +physical setup, TODO) and a caveat entry in `boards.md` (same directory) — +**read a board's row and caveat before capturing on it**; new validations add +both. Timing semantics and clock columns are explained at the top of that file. + +## References + +- Ozone manual (UM08025, automation socket §6.7, project commands §7): + <https://www.segger.com/downloads/jlink/UM08025_Ozone.pdf> — V3.50, same as + the installed Ozone (web is rev 1 vs the local copy's rev 0; the local PDF + under /opt/SEGGER/Ozone_V350/Doc remains the offline fallback). +- J-Link / J-Trace manual (UM08001, trace ch. 10, timing troubleshooting): + <https://kb.segger.com/UM08001_J-Link_/_J-Trace_User_Guide> diff --git a/.claude/skills/etm-trace/boards.md b/.claude/skills/etm-trace/boards.md new file mode 100644 index 000000000..044d4e0ee --- /dev/null +++ b/.claude/skills/etm-trace/boards.md @@ -0,0 +1,161 @@ +# etm-trace — per-board reference + +Validated boards: trace config table + hard-won caveats. Read the row AND the +caveat for a board before capturing on it; add a row + caveat when a new board +is validated (jdebug reference, board.cmake/board.h clock selection and this +file must agree). + +"Core (trace build)" is the CPU clock a `TRACE_ETM=1` build runs — where it +differs from the stock clock, board.h selects it automatically. Timing +"0 (unset)" = the reference sets no SetTraceTiming and Ozone then sends +`TraceSampleAdjust TD = 0`; J-Link's own +2 ns default (UM08001) applies only +outside Ozone-driven captures. Explicit values live in the committed +reference. + +| Board | Core (trace build) | TRACECLK pin | Width | Timing | Physical setup | TODO | +|--------------------|----------------------|-----------------------|-------|---------|-------------------------------|---------------------------------------------| +| stm32h743eval | 400 MHz | 50 MHz (PLL1R, fixed) | 4 | +100 ps | — | — | +| stm32n657nucleo | 300 MHz | 18.75 MHz (cpu/16) | 4 | 0 (unset) | none — CN1 MIPI20; JP2=1 | — | +| stm32h7s3nucleo | 300 MHz | 50 MHz (cpu/3/2) | 2 | 0 (unset) | none — native CN1 MIPI20 | remove SB11/SB12 → width 4 @ 600 MHz | +| stm32h563nucleo | 100 MHz | follows core | 1 | +5 ns | remove SB8/9/64/68/70/71/78 | retest width 4 / 250 MHz after SB removal | +| metro_m7_1011 | 500 MHz | 66 MHz (root/2) | 4 | +50 ps | custom rev A ETM-header rework | — | +| mcb1800 | 120 MHz | 60 MHz (CCLK/2) | 4 | 0 (unset) | fit J5 DBG_EN | — | +| ea4088_quickstart | 120 MHz | 120 MHz | 4 | 0 (unset) | J7 (fully wired) | — | +| nrf52840dk | 64 MHz | 16 MHz (hw cap) | 4 | 0 (unset) | solder P25, SW7 → Alt | — | +| nrf5340dk (M33) | 64 MHz | 16 MHz (TAD, forced) | 4 | +3 ns | mount P25; cut SB27/SB28 | — | +| mimxrt1170_evkb | 996 MHz | 50 MHz (root/2) | 1 | 0 | weld 0 Ω R1881-R1886; JP4 shorted; J58 (populated) | re-weld R1884 (D3 open; D1/D2 meter-verified good) → width 4 | +| ra6m5_ek (M33) | 200 MHz | 25 MHz (TRCLK/4 /2) | 4 | 0 (unset) | J9 closed; native J20 trace | — | +| ra8m1_ek (M85) | 480 MHz | 60 MHz (TRCLK/4 /2) | 4 | 0 (unset) | J9 closed + Table 7 jumpers | — | +| raspberry_pi_pico2 (RP2350 M33) | 48 MHz | 24 MHz (clk_sys/2) | 4 | 0 (unset) | fly-wire GPIO1-5 → MIPI20 (map in jdebug) | 72-80 MHz per seating (re-qualify); >80 needs V3 probe + trace board | +| same54_xplained (E54 M4F) | 120 MHz | 60 MHz (CPU/2) | 4 | 0 (unset) | none — populated 20-pin ETM header | — | +| same70_xplained (E70 M7) | 300 MHz | 37.5 MHz (PCK3/2) | 1 | 0 (unset) | solder 20-pin header on J403 (bottom) | width 4 blocked: D1 (J403.16) dead at speed — probe-channel crosscheck pending | +| SEGGER H7/F407 ref | demo defaults | demo | 4 | demo | probe-powered: add `--power` | — | + +Board caveats (beyond the table): + +- **stm32h743eval**: startup-burst overflow at 400 MHz is normal (reduce + PLLN in board.h for overflow-free capture); timestamp ref 200 MHz. +- **stm32n657nucleo** (M55, flashless): **JP2 (BOOT1) must be 1** — the app + is a RAM image the debugger loads (Development boot); in flash boot the + bootROM parks the chip un-attachable ("Can not attach to CPU"). SEGGER's + KB says BOOT0/BOOT1 = 0/0 for their example — that is flash boot and it + does NOT attach; the board manual's Table 11 is right. 600 MHz core kills + the stream in the startup burst (timestamp flux, not overflow) — TRACE_ETM + builds run 300 MHz; at 600 use `--no-timestamps`. No J-Link script needed: + N6 trace components are ROM-table-discoverable. +- **stm32h7s3nucleo**: width 4 is clean at idle but SB11/SB12 (default ON) + stub D2/D3 onto Zio CN8 and the stream dies under IRQ-heavy USB traffic — + remove them to try width 4 / 600 MHz. `--attach` while a USB host is + actively polling the device wedges its USB session (needs target reset). +- **stm32h563nucleo**: width 4 or 250 MHz corrupts. H5 hangs its debug AP if + trace CoreSight is touched unclocked — handled by the committed + `AfterTargetConnect` hook; un-attachable after a killed session → + power-cycle. +- **metro_m7_1011** (RT1011): a custom Adafruit rev with a hand-added 2x10 + ETM header (KiCad schematic via the `read-doc` skill). No SEGGER RT1011 + example exists — the committed .jdebug (tuned +50 ps) is the known-good + reference. BOARD_BootClockRUN sets the 132 MHz trace root but leaves it + gated; `trace_etm_init` ungates it. The first Ozone run after a fresh + flash can fail to reach main — transient, retry once. +- **mcb1800**: a bad ribbon mating yields register-perfect silence — re-seat + BOTH ribbon ends first; SWD working proves nothing about the trace lines. + `--isr USB0_IRQHandler,dcd_int_handler`. +- **nrf5340dk**: the interface MCU's UART1 flow control rides the trace + pins — it actively drives CTS onto P0.10/TRACEDATA1 (dead line, any + timing) and loads P0.11/TRACEDATA0: cut SB27/SB28 (or flip SW7 to FC-off). + SB57's SWO stub can stay — harmless at the +3 ns sample point. TRACE_ETM + builds force the TAD port to 16 MHz (SystemInit's 64 MHz is marginal). +- **ea4088_quickstart**: FS enumeration ends < 100 ms — for `--isr` use + `--duration-ms 150`. Boot-ROM address warning (0x1FFF1FF0) is normal. +- **mimxrt1170_evkb**: width 1 only — D1-D3 are stone silent at any config + (pinmux register-perfect, PHY quieted, funnel enabled): the welded 0402s + R1882/R1883/R1884 are electrically open — reflow to unlock width 4. + `trace_etm_init` fixes the JTAG_nTRST/DMIC_DATA1 pad, holds the 100M + RTL8201 PHY in reset (ENET_RST_B = GPIO_LPSR_04 — its RMII lines share + the trace pads; with it quiet the CLK line runs a 100 MHz root/50 MHz + pin, 2x the pre-lever rate; 133 MHz root is marginal, stock 132 corrupts) + and enables the CM7 platform trace-funnel port, which J-Link doesn't + program: without it everything reads register-perfect yet zero data + arrives. JP4 must be shorted (disables MCU-Link SWD) for the external + J-Trace on J58; a powered MCU-Link USB breaks the external probe's connect + even with JP4 shorted - power the board from another port. **Width is 1 or + 4 only**: a width-2 request arms the CSSYS TPIU (E004_6000) and the probe + sampler at 4-bit anyway (CSPSR reads 0x8 after a width-2 session; a live + CSPSR=2 poke with LAR unlock + Trace.Clear still captures nothing because + the probe keeps sampling 4-bit). FlexSPI apps: ROM bootloader must set SP/PC — the + committed reset/download hooks handle this. Startup-burst overflow at + 996 MHz is normal. No Ethernet (100M) while tracing. +- **ra6m5_ek**: TRCKCR div-2 (100 MHz TRCLK = 50 MHz pin, the chip max) is + unusable on this board — swept widths 1/4 across -2..+4 ns, all dead; + TRACE_ETM builds use div-4 (25 MHz pin). The TCLK pin runs TRCLK/2. 50 MHz SWD TIF + caused intermittent "Failed to initialize DAP" — the reference runs 4 MHz. + ISR entry: `--isr tusb_int_handler,dcd_int_handler` (FSP's + usbfs_interrupt_handler symbol never actually executes). +- **ra6m5_ek / ra8m1_ek — `--attach` needs a debugger-booted target**: the + firmware TRCKCR setup is gated on DHCSR.C_DEBUGEN (an unguarded write + wedges a standalone boot un-attachable until power-cycle), so a board + booted WITHOUT a debugger has no trace clock and an `--attach` capture + reads silence. Reflash/reset through the capture default flow first. +- **ra8m1_ek**: **J9 must be closed** (holds the on-board J-Link OB in + reset — open = SWD contention, intermittent "Failed to initialize DAP", + even an apparent brick recoverable only by power-cycle/J16 boot mode). + J-Link's RA8 support enables trace from reset; the committed JLinkScript's + empty `OnTraceStart` suppresses that so the firmware enables the trace + clock after the FSP clock switch — without it the MOCO→PLL step desyncs + the decoder at t≈0.05 s every run. Runs both chip maxima (120 MHz TRCLK, + 60 MHz pin) clean. `ReadIntoTraceCache 0x0 0x10000` in the download hook + covers runtime chip-ROM execution. ISR entry: `tusb_int_handler`. +- **raspberry_pi_pico2** (RP2350): TRACECLK is a fixed clk_sys/2, no divider + (DDR data, like every ARM TPIU pin port). **Measured cliff on this rig:** + 80 MHz core (40 MHz TRACECLK) traces idle code but dies under dense data; + 88 MHz+ dies instantly at any width/global-timing/TIF/pad setting. Cause + not pinned down: the same V2 probe samples 66 MHz TRACECLK (132 Msample/s) + on metro_m7_1011, so it is NOT a plain probe sample-rate ceiling. The + cliff at >40 MHz TRACECLK (84+ MHz core) survived a full sweep - global + AND per-pin `--trace-timing`, pad drive 2/4/8/12 mA + slew, width 4/2/1, + TIF 1-25 MHz, newer J-Link library - all flat, so it is V3-probe / real- + trace-board territory (SEGGER's Pico 2 KB requires J-Trace PRO **V3.0+** + and recommends a proper trace board; community reports fly-wires fail at + 75 MHz for everyone, PCBs work). Separately, fly-wire seating quality + sets the width-4 DENSE-data ceiling (48-72 MHz observed across seatings): + after ANY rewiring re-qualify with idle blinky at the target clock, then + cdc_msc x3. Random unknown-packet deaths KB into a clean stream = one + marginal wire; `--trace-width` 1 vs 2 vs 4 bisects which (width 1 = + CLK+D0 only; D1 = GPIO3->MIPI20 pin 16 has gone marginal twice on this + rig). Width-1 is a full-quality fallback: complete cdc_msc profiles at up + to 80 MHz core even when width 4 is broken. + **Never set a custom JLinkScript** — it + replaces J-Link's built-in RP2350 device script, which both declares the + trace component map (funnel/TPIU/ETM are not in the ROM table → "Required + trace components for pin trace not found", 0 fetches) and re-arms the whole + chip-side path via `OnTraceStart` at every resume. Firmware therefore does + no trace setup; TRACE_ETM builds only (a) pin clk_sys to 48 MHz from crt0 + (board.cmake) — the fly-wire ceiling: 96/150 MHz kill the stream in the + startup burst at any sample timing (and at 150 MHz the saturated probe + stops answering halts, "CPU could not be halted"); any post-arm clock + change steps TRACECLK mid-stream and kills the decoder — and (b) + clear TIMER0/1 DBGPAUSE (family.c): debug sessions leave cores + halted-at-reset and the default DBGPAUSE freezes the µs timer, so every + `sleep_ms()` spins forever (looks like a dead board; watchdog-scratch + breadcrumbs survive warm resets but not POR when diagnosing). UART console + is TX-only (GPIO1 = TRACECLK). Empty reset/download hooks: the bootrom + must run the IMAGE_DEF. If the chip ends up wedged/un-attachable: + J-Link `erase` + reset drops it into BOOTSEL (2e8a:000f) for picotool. +- **same54_xplained**: the CM4 trace unit is clocked from **GCLK channel 47 + (GCLK_CM4_TRACE)** — with it disabled the pins mux fine, TPIU/ETM arm + fine, and the port stays perfectly silent (zero fetches, no errors); + `trace_etm_init` feeds it GCLK0. Pins PC24-28 mux to function H. The + populated 20-pin header runs chip-max 60 MHz TRACECLK width 4 with no + timing adjustment - the connector-vs-flywire contrast board. +- **same70_xplained**: J403 is a bottom-side bare footprint — solder the + header. Trace pins PD4-7 double as the KSZ8081 PHY's RMII receive outputs: + TRACE_ETM builds hold it in reset (PHY_RESET=PC10) or it drives against + the stream. TPIU clock = PCK3 (datasheet 16.7.4), run at MCK/2; TPIU + programming while PCK3 is stopped is silently LOST — the reference starts + PCK3 in the post-reset/download hooks (a reset wipes the PMC, so + AfterTargetConnect is too early). Width 1 validated at the stock 300 MHz + core; width 2/4 blocked on a dead D1 line at J403.16 (clean at DC by + meter, dead at speed — probe-channel crosscheck on a known-good width-4 + board pending). No Ethernet while tracing. +- **SEGGER ref boards**: run their own demo (ladder step 3 flags); `--isr` + degrades gracefully without a live SysTick. diff --git a/.claude/skills/etm-trace/scripts/etm_capture.py b/.claude/skills/etm-trace/scripts/etm_capture.py new file mode 100644 index 000000000..ccde886f2 --- /dev/null +++ b/.claude/skills/etm-trace/scripts/etm_capture.py @@ -0,0 +1,572 @@ +#!/usr/bin/env python3 +"""Capture streaming ETM instruction trace unattended via J-Trace + Ozone. + +Generates a throwaway Ozone project (never touches the committed +hw/bsp/**/ozone/*.jdebug), launches Ozone on a virtual display (xvfb-run), +drives the whole session over Ozone's automation TCP socket (UM08025 §6.7): +connect -> flash -> run for --duration-ms under streaming trace -> halt -> +export. Outputs in --out: + code_profile.txt hot functions (run/fetch counts) + code coverage + itrace.csv raw instruction history (only with --trace-csv) + ozone_console.log, jlink.log, ozone_gui.log session evidence + +Requires firmware built with -DTRACE_ETM=1 and the board wired to a J-Trace. +Analyze results with etm_profile.py in this directory. +""" + +import argparse +import glob +import os +import re +import shutil +import signal +import socket +import string +import subprocess +import sys +import tempfile +import time + +REPO_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), *[".."] * 4)) + +# Throwaway Ozone project. SP/PC-from-vector-table hooks match the committed +# reference projects (Cortex-M generic). $$(InstallDir) -> literal $(InstallDir). +PROJECT_TEMPLATE = string.Template("""\ +/* Auto-generated by etm_capture.py (etm-trace skill) - throwaway, do not commit. */ +void OnProjectLoad (void) { + Project.SetDevice ("$device"); + Project.SetHostIF ("USB", "$probe"); + Project.SetTargetIF ("SWD"); + Project.SetTIFSpeed ("$tif_speed"); + Project.SetTraceSource ("Trace Pins"); + Project.SetTracePortWidth ($port_width); +$timing_line$core_clock_line$timestamps_line$hss_line$power_lines Edit.SysVar (VAR_TRACE_MAX_INST_CNT, $max_inst); + Edit.Preference (PREF_TIMESTAMP_FORMAT, TIMESTAMP_FORMAT_TIME); + Project.AddSvdFile ("$$(InstallDir)/Config/Peripherals/ARMv7M.svd"); + Project.SetConsoleLogFile ("$outdir/ozone_console.log"); + Project.SetJLinkLogFile ("$outdir/jlink.log"); +$os_plugin_line File.Open ("$elf"); +} +$jlink_script_hook + +$reset_hook + +$download_hook +$user_funcs""") + +# Cortex-M generic SP/PC-from-vector-table init. A committed board reference +# overrides these verbatim (e.g. RT1176 apps in FlexSPI NOR need the ROM +# bootloader to do SP/PC init, plus a JTAG_nTRST pad fix). +DEFAULT_HOOK = """\ +void %s (void) { + unsigned int SP; + unsigned int PC; + unsigned int VectorTableAddr; + VectorTableAddr = Elf.GetBaseAddr(); + if (VectorTableAddr == 0xFFFFFFFF) { + Util.Log("etm_capture: failed to get program base"); + } else { + SP = Target.ReadU32(VectorTableAddr); + Target.SetReg("SP", SP); + PC = Target.ReadU32(VectorTableAddr + 4); + Target.SetReg("PC", PC); + } +}""" + +# Symbolic constants (TP_OP_*, EXPORT_*) only evaluate in project-script +# context, never over the automation socket - so these live in generated user +# functions invoked via Script.Exec (UM08025 SS6.7, SS7.9.9.1). +TRACEPOINT_FUNC = """\ + +void SetupTracepoints (void) { +%s} +""" + +LINES_CSV_FUNC = """\ + +void ExportLinesCsv (void) { + Export.CodeProfile ("%s", EXPORT_AS_CSV | EXPORT_CSV_LINES | EXPORT_FILE_PATHS, ""); +} +""" + +INSTS_CSV_FUNC = """\ + +void ExportInstsCsv (void) { + Export.CodeProfile ("%s", EXPORT_AS_CSV | EXPORT_CSV_INSTS | EXPORT_FILE_PATHS, ""); +} +""" + + +JLINK_SCRIPT_HOOK = """\ + +void BeforeTargetConnect (void) { + Project.SetJLinkScript ("%s"); +} +""" + + +def resolve_board(board): + """Board config from its committed ozone reference project, else board.cmake/mk.""" + cfg = {"device": None, "tif_speed": "4 MHz", "timing": None, "port_width": 4, + "core_clock": None, "ref": None} + jdebugs = sorted(glob.glob(f"{REPO_ROOT}/hw/bsp/*/boards/{board}/ozone/*.jdebug")) + if jdebugs: + cfg["ref"] = jdebugs[0] + text = open(jdebugs[0]).read() + # config regexes must not match //-commented lines + cfgtext = re.sub(r"^\s*//.*$", "", text, flags=re.M) + # inherit every user function verbatim except OnProjectLoad, which the + # template owns (e.g. STM32H5's AfterTargetConnect must clock the trace + # CoreSight domain; RT1176 replaces the SP/PC reset/download hooks for + # ROM-bootloader boot; Nordic hooks call a _SetupTarget helper that + # must ride along or hook execution fails at runtime) + extra = [] + for m in re.finditer(r"^void (\w+)\s*\(void\)\s*\{.*?^\}", text, + re.M | re.S): + name, block = m.group(1), m.group(0) + if name == "OnProjectLoad": + continue + elif name == "BeforeTargetConnect": + # the generated project synthesizes its own BeforeTargetConnect + # (JLINK_SCRIPT_HOOK) from the SetJLinkScript regex below; + # inheriting the reference's copy too would emit a duplicate + # function definition + continue + elif name == "AfterTargetReset": + cfg["reset_hook"] = block + elif name == "AfterTargetDownload": + cfg["download_hook"] = block + else: + extra.append(block) + if extra: + cfg["connect_hook"] = "\n\n".join(extra) + "\n" + for key, pat in (("device", r'Project\.SetDevice\s*\(\s*"([^"]+)"'), + ("tif_speed", r'Project\.SetTIFSpeed\s*\(\s*"([^"]+)"'), + ("timing", r'Project\.SetTraceTiming\s*\(([-\d\s,]+)\)'), + ("port_width", r'Project\.SetTracePortWidth\s*\(\s*(\d+)'), + ("core_clock", r'VAR_TRACE_CORE_CLOCK\s*,\s*(\d+)')): + m = re.search(pat, cfgtext) + if m: + cfg[key] = m.group(1).strip() + # inherit a J-Link script (e.g. RT1176 must declare its off-ROM-table + # TPIU/funnel); relative paths resolve against the reference's dir + m = re.search(r'Project\.SetJLinkScript\s*\(\s*"([^"]+)"', cfgtext) + if m: + # $(ProjectDir) = the reference's own directory + rel = m.group(1).replace("$(ProjectDir)", ".") + cfg["jlink_script"] = os.path.normpath(os.path.join( + os.path.dirname(jdebugs[0]), rel)) + else: + for path in glob.glob(f"{REPO_ROOT}/hw/bsp/*/boards/{board}/board.cmake"): + m = re.search(r'JLINK_DEVICE\s+([^\s)]+)\s*\)', open(path).read()) + if m: + if "${" in m.group(1): + sys.exit(f"error: {path} defines JLINK_DEVICE via an " + f"unexpanded CMake variable ({m.group(1)}) - pass " + f"--device explicitly for this board") + cfg["device"] = m.group(1) + cfg["ref"] = path + break + if not cfg["device"]: + sys.exit(f"error: cannot resolve J-Link device for board '{board}' " + f"(no hw/bsp/*/boards/{board}/ozone/*.jdebug or board.cmake)") + return cfg + + +def trace_only_points(elf, syms_arg): + """Tracepoint lines for --trace-only: start trace at each symbol's entry, + stop at each return instruction inside it (pop ...pc / bx lr, via objdump). + Hardware comparators are scarce (ETM-M7) - keep the symbol list short.""" + lines = "" + nm = subprocess.run(["arm-none-eabi-nm", "-S", "--defined-only", elf], + capture_output=True, text=True).stdout + for want in [s.strip() for s in syms_arg.split(",") if s.strip()]: + m = re.search(rf"^([0-9a-f]+) ([0-9a-f]+) [TtWw] {re.escape(want)}$", + nm, re.M) + if not m: + sys.exit(f"error: --trace-only symbol '{want}' not in ELF") + lo, sz = int(m.group(1), 16) & ~1, int(m.group(2), 16) + lines += f' Trace.SetPoint (TP_OP_START_TRACE, "{want}");\n' + dis = subprocess.run( + ["arm-none-eabi-objdump", "-d", f"--start-address={lo:#x}", + f"--stop-address={lo + sz:#x}", elf], + capture_output=True, text=True).stdout + exits = re.findall( + r"^\s*([0-9a-f]+):.*?(?:(?:pop|ldmia[.\w]*\s+sp!,)[^\n]*\bpc\b|bx\s+lr)", + dis, re.M | re.I) + if not exits: + sys.exit(f"error: no return instruction found in '{want}'") + for addr in exits: + lines += f' Trace.SetPoint (TP_OP_STOP_TRACE, "0x{int(addr, 16):08X}");\n' + return lines + + +def resolve_probe(probe): + """Ozone's SetHostIF needs a serial - with several probes connected a + nickname makes it block on a selection dialog. JLinkExe DOES resolve + nicknames, so borrow its banner to map nickname -> serial. The serial only + ever lands in the throwaway project file, never in committed files.""" + if not probe or probe.isdigit(): + return probe + r = subprocess.run(["JLinkExe", "-USB", probe, "-nogui", "1"], + input="qc\n", capture_output=True, text=True, timeout=30) + m = re.search(r"S/N:\s*(\d+)", r.stdout) + if not m: + sys.exit(f"error: cannot resolve probe nickname '{probe}' to a serial " + f"(JLinkExe -USB {probe} found no emulator)") + return m.group(1) + + +def gen_project(cfg, args, outdir): + timing = "" + if args.trace_timing is not None: + d = [int(v) for v in str(args.trace_timing).split(",")] + if len(d) not in (1, 4): + sys.exit("error: --trace-timing takes one value or d0,d1,d2,d3") + d = d * 4 if len(d) == 1 else d + timing = (" Project.SetTraceTiming " + f"({d[0]}, {d[1]}, {d[2]}, {d[3]});\n") + elif cfg["timing"]: + timing = f" Project.SetTraceTiming ({cfg['timing']});\n" + core_clock = args.core_clock or cfg["core_clock"] + clk_line = f" Edit.SysVar (VAR_TRACE_CORE_CLOCK, {core_clock});\n" if core_clock else "" + ts_line = (" Edit.SysVar (VAR_TRACE_TIMESTAMPS_ENABLED, 0);\n" + if args.no_timestamps else "") + user_funcs = "" + if cfg.get("connect_hook"): + user_funcs += "\n" + cfg["connect_hook"] + if args.trace_only: + user_funcs += TRACEPOINT_FUNC % trace_only_points( + os.path.abspath(args.elf), args.trace_only) + if args.profile_lines_csv: + user_funcs += LINES_CSV_FUNC % os.path.join(outdir, "profile_lines.csv") + if args.profile_insts_csv: + user_funcs += INSTS_CSV_FUNC % os.path.join(outdir, "profile_insts.csv") + if args.os_plugin and not glob.glob( + f"/opt/SEGGER/Ozone*/Plugins/OS/{args.os_plugin}.js"): + sys.exit(f"error: RTOS plugin '{args.os_plugin}' not in " + f"/opt/SEGGER/Ozone*/Plugins/OS (e.g. FreeRTOSPlugin_CM7)") + os_plugin = (f' Project.SetOSPlugin ("{args.os_plugin}");\n' + if args.os_plugin else "") + if args.attach: + # attach to the running target: no download, no reset - trace a window + # mid-run (firmware must match the ELF and have trace pins enabled) + os_plugin += " Debug.SetConnectMode (CM_ATTACH_HALT);\n" + # HSS sampling rate belongs in OnProjectLoad (persistent) per UM08025 4.6.2; + # setting it mid-session over the socket is rejected. + hss = (f" Edit.SysVar (VAR_HSS_SPEED, {args.sample_hz});\n" + if args.sample else "") + power = (" Edit.SysVar (VAR_TARGET_POWER_ON, 1);\n" + f" Edit.SysVar (VAR_POWER_SAMPLING_SPEED, {args.power_hz});\n" + if args.power else "") + jlink_script = args.jlink_script or cfg.get("jlink_script") + jls_hook = (JLINK_SCRIPT_HOOK % os.path.abspath(jlink_script) + if jlink_script else "") + proj = os.path.join(outdir, "etm_capture.jdebug") + if args.trace_width: + cfg["port_width"] = args.trace_width + with open(proj, "w") as f: + f.write(PROJECT_TEMPLATE.substitute( + device=cfg["device"], probe=resolve_probe(args.probe), + tif_speed=cfg["tif_speed"], + port_width=cfg["port_width"], timing_line=timing, core_clock_line=clk_line, + timestamps_line=ts_line, max_inst=args.max_inst, outdir=outdir, + elf=os.path.abspath(args.elf), user_funcs=user_funcs, + os_plugin_line=os_plugin, hss_line=hss, power_lines=power, + jlink_script_hook=jls_hook, + reset_hook=cfg.get("reset_hook", DEFAULT_HOOK % "AfterTargetReset"), + download_hook=cfg.get("download_hook", + DEFAULT_HOOK % "AfterTargetDownload"))) + return proj + + +class OzoneSession: + """Drive Ozone via its automation TCP socket (one connection at a time).""" + + def __init__(self, port, logf): + self.port = port + self.logf = logf + self.sock = None + + def log(self, msg): + line = f"[{time.strftime('%H:%M:%S')}] {msg}" + print(line, flush=True) + self.logf.write(line + "\n") + self.logf.flush() + + def connect(self, timeout_s): + deadline = time.time() + timeout_s + while time.time() < deadline: + try: + self.sock = socket.create_connection(("127.0.0.1", self.port), timeout=5) + self.sock.settimeout(0.5) + self.log(f"connected to Ozone automation socket :{self.port}") + return + except OSError: + time.sleep(1) + raise TimeoutError(f"Ozone automation socket :{self.port} not reachable " + f"after {timeout_s}s (see ozone_gui.log)") + + def drain(self, wait_s=1.0): + buf = b"" + end = time.time() + wait_s + while time.time() < end: + try: + chunk = self.sock.recv(65536) + if not chunk: + break + buf += chunk + end = time.time() + 0.5 # keep reading while data flows + except socket.timeout: + pass + text = buf.decode(errors="replace") + for ln in text.splitlines(): + self.log(f" ozone> {ln}") + return text + + def send(self, cmd, wait_s=1.0): + self.log(f"cmd: {cmd}") + self.sock.sendall((cmd + "\n").encode()) + return self.drain(wait_s) + + def wait_echo(self, cmd, timeout_s): + """Send cmd; wait until Ozone echoes its execution (echo comes after the + command completed, e.g. a large Export). Returns all received text.""" + name = cmd.split("(")[0].strip() + text = self.send(cmd, 1.0) + deadline = time.time() + timeout_s + while name + " (" not in text and name + "(" not in text: + if time.time() > deadline: + raise TimeoutError(f"no echo for '{name}' after {timeout_s}s") + text += self.drain(1.0) + return text + + def is_halted(self, timeout_s): + """Poll Debug.IsHalted until it returns 1; parse the '// returns 0xN' echo.""" + deadline = time.time() + timeout_s + while time.time() < deadline: + text = self.send("Debug.IsHalted", 1.0) + end2 = time.time() + 4.0 + while "Debug.IsHalted" not in text and time.time() < end2: + text += self.drain(1.0) + m = re.findall(r"Debug\.IsHalted\s*\(\);\s*//\s*returns\s*0x(\d+)", text) + if m and m[-1] == "1": + return True + time.sleep(1) + return False + + +def main(): + p = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + p.add_argument("--board", help="TinyUSB board name, e.g. stm32h743eval") + p.add_argument("--device", help="J-Link device name for non-TinyUSB targets " + "(e.g. STM32F407VE for a SEGGER trace reference board); " + "requires --elf, bypasses hw/bsp board resolution") + p.add_argument("--tif-speed", default="4 MHz", + help="SWD speed for --device targets (default '4 MHz')") + p.add_argument("--jlink-script", + help="J-Link script file (.pex/.JLinkScript) for trace-pin " + "init when the firmware doesn't do it (SEGGER per-MCU " + "examples); wired into BeforeTargetConnect") + p.add_argument("--power", action="store_true", + help="power the target from the probe and record a power " + "profile (power.csv); probe power is switched off after " + "the session. Target must be wired for probe power!") + p.add_argument("--power-hz", type=int, default=10000, + help="power sampling frequency in Hz (default 10000)") + p.add_argument("--elf", help="firmware ELF built with -DTRACE_ETM=1 " + "(default: examples/cmake-build-<board>/device/cdc_msc/cdc_msc.elf)") + p.add_argument("--duration-ms", type=int, default=10000, help="traced run time") + p.add_argument("--out", help="output dir (default: mkdtemp under /tmp)") + p.add_argument("--port", type=int, default=19201, + help="automation socket port (19200 = interactive Ozone default; keep 19201)") + p.add_argument("--probe", default="", + help="J-Link USB nickname or serial ('' = sole connected probe)") + p.add_argument("--trace-csv", action="store_true", + help="also export raw instruction history (itrace.csv, can be >100 MB)") + p.add_argument("--max-inst", type=int, default=10000000, + help="VAR_TRACE_MAX_INST_CNT: instruction-trace window/export depth") + p.add_argument("--core-clock", type=int, + help="CPU Hz for timestamp conversion (default: board reference value)") + p.add_argument("--trace-timing", + help="trace sample delay in ps (-5000..5000, overrides the " + "board reference; sweep this when the stream dies with " + "unknown-packet errors). One value for all 4 pins, or " + "'d0,d1,d2,d3' to de-skew individual lines (boards can " + "have per-line RC delays, e.g. strap pulls on muxed pads)") + p.add_argument("--trace-width", type=int, choices=(1, 2, 4), + help="trace port width override (fewer pins = tolerant of a " + "single bad line, at reduced bandwidth)") + p.add_argument("--no-timestamps", action="store_true", + help="disable trace timestamps (less trace bandwidth -> fewer " + "overflows/decode errors; itrace.csv loses its time column)") + p.add_argument("--trace-only", + help="comma-separated symbols: trace ONLY these functions via " + "hardware tracepoints (e.g. an ISR + SysTick_Handler for " + "calibration); needs few symbols (scarce comparators)") + p.add_argument("--profile-lines-csv", action="store_true", + help="also export per-source-line profile/coverage counters " + "(profile_lines.csv)") + p.add_argument("--profile-insts-csv", action="store_true", + help="also export per-instruction counters (profile_insts.csv, " + "enables branch-bias analysis in etm_profile.py)") + p.add_argument("--attach", action="store_true", + help="attach to the RUNNING target instead of flash+reset: " + "capture a window mid-run (narrowing debug). The flashed " + "firmware must match --elf and have been built with " + "TRACE_ETM=1") + p.add_argument("--sample", + help="comma-separated C expressions to sample periodically " + "during the run (samples.csv, e.g. 'system_ticks')") + p.add_argument("--sample-hz", type=int, default=1000, + help="data sampling frequency in Hz (default 1000)") + p.add_argument("--os-plugin", + help="Ozone RTOS-awareness plugin for task/ISR-attributed " + "timeline, e.g. FreeRTOSPlugin_CM7 (see " + "/opt/SEGGER/Ozone*/Plugins/OS)") + args = p.parse_args() + + if not args.board and not args.device: + sys.exit("error: need --board (TinyUSB) or --device + --elf (other targets)") + if args.device and not args.elf: + sys.exit("error: --device requires --elf") + if args.max_inst > 10000000: + # >10M yielded a silently EMPTY Export.Trace on Ozone V3.50 + print("warning: --max-inst clamped to 10000000 (larger values produce " + "an empty instruction-trace export)", file=sys.stderr) + args.max_inst = 10000000 + if not args.elf: + args.elf = (f"{REPO_ROOT}/examples/cmake-build-{args.board}" + f"/device/cdc_msc/cdc_msc.elf") + if not os.path.isfile(args.elf): + sys.exit(f"error: ELF not found: {args.elf}\n" + f"build it with -DTRACE_ETM=1 (see the etm-trace skill) or pass --elf") + + if args.trace_only: + print("warning: --trace-only is EXPERIMENTAL - on some targets windows " + "for low-rate handlers fail to record, and timestamps are invalid " + "across trace gaps (instruction counts remain exact). For ISR " + "timing prefer a full --trace-csv capture + etm_profile.py --isr.", + file=sys.stderr) + if args.device: + cfg = {"device": args.device, "tif_speed": args.tif_speed, "timing": None, + "port_width": 4, "core_clock": None, "ref": "--device"} + else: + cfg = resolve_board(args.board) + outdir = os.path.abspath(args.out) if args.out else tempfile.mkdtemp( + prefix=f"etm-{args.board or args.device}-") + os.makedirs(outdir, exist_ok=True) + proj = gen_project(cfg, args, outdir) + + ozone_bin = shutil.which("ozone") or shutil.which("Ozone") + if not ozone_bin: + sys.exit("error: ozone not on PATH (install SEGGER Ozone)") + cmd = [ozone_bin, "-project", proj, "-port", str(args.port)] + if shutil.which("xvfb-run"): + cmd = ["xvfb-run", "-a"] + cmd + elif os.environ.get("DISPLAY"): + print("warning: xvfb-run not found - Ozone window will appear on " + f"DISPLAY={os.environ['DISPLAY']} and may steal keyboard focus", + file=sys.stderr) + else: + sys.exit("error: no DISPLAY and no xvfb-run; install xvfb") + + gui_log = open(os.path.join(outdir, "ozone_gui.log"), "w") + ses_logf = open(os.path.join(outdir, "session.log"), "w") + ses = OzoneSession(args.port, ses_logf) + ses.log(f"board={args.board} device={cfg['device']} ref={cfg['ref']}") + ses.log(f"elf={args.elf}") + ses.log(f"out={outdir}") + proc = subprocess.Popen(cmd, stdout=gui_log, stderr=gui_log, + start_new_session=True) + profile_out = os.path.join(outdir, "code_profile.txt") + itrace_out = os.path.join(outdir, "itrace.csv") + try: + ses.connect(20) + ses.drain(3) # version banner + ses.send("Debug.Start", 5) + if not ses.is_halted(90): + raise TimeoutError("Debug.Start did not reach the startup completion " + "point (connect/flash failed? see jlink.log)") + ses.log("startup complete (halted at main)") + if args.trace_only: + ses.wait_echo('Script.Exec ("SetupTracepoints")', 15) + ses.send('Window.Show ("Code Profile")', 2) + if args.trace_csv: + ses.send('Window.Show ("Instruction Trace")', 2) + if args.sample: + ses.send('Window.Show ("Data Sampling")', 2) + for expr in args.sample.split(","): + ses.send(f'Window.Add ("Data Sampling", "{expr.strip()}")', 2) + ses.send("Coverage.ExcludeNOPs()", 2) + + ses.log(f"=== traced run: {args.duration_ms} ms ===") + ses.send("Debug.Continue", 1) + time.sleep(args.duration_ms / 1000.0) + ses.send("Debug.Halt", 3) + if not ses.is_halted(30): + raise TimeoutError("target did not halt") + ses.send("Window.WaitForUpdateComplete(120000)", 5) + + ses.wait_echo(f'Export.CodeProfile ("{profile_out}", 0, "")', 60) + if args.profile_lines_csv: + ses.wait_echo('Script.Exec ("ExportLinesCsv")', 60) + if args.profile_insts_csv: + ses.wait_echo('Script.Exec ("ExportInstsCsv")', 120) + if args.trace_csv: + ses.wait_echo(f'Export.Trace ("{itrace_out}", 0)', 300) + if args.sample: + ses.wait_echo(f'Export.DataGraphs ("{outdir}/samples.csv")', 60) + if args.power: + ses.wait_echo(f'Export.PowerGraphs ("{outdir}/power.csv")', 60) + ses.send("Debug.Stop", 5) + ses.send("File.Exit", 2) + finally: + for _ in range(15): + if proc.poll() is not None: + break + time.sleep(1) + if proc.poll() is None: + ses.log("killing leftover Ozone process group") + os.killpg(proc.pid, signal.SIGKILL) + if args.power: + # guarantee probe power is off, whatever happened above + sel = ["-USB", args.probe] if args.probe else [] + r = subprocess.run(["JLinkExe", *sel, "-nogui", "1"], + input="power off\nqc\n", capture_output=True, + text=True, timeout=30) + ses.log("probe power off " + + ("issued" if r.returncode == 0 else f"FAILED rc={r.returncode}")) + + if not (os.path.isfile(profile_out) and os.path.getsize(profile_out) > 0 + and "Code Profile Report" in open(profile_out, errors="replace").read(200)): + sys.exit(f"error: capture ran but {profile_out} is missing/empty - " + f"check {outdir}/session.log and ozone_console.log") + if args.trace_csv and not (os.path.isfile(itrace_out) + and os.path.getsize(itrace_out) > 0): + sys.exit(f"error: --trace-csv requested but {itrace_out} is missing/empty") + if args.power and not os.path.getsize(os.path.join(outdir, "power.csv")): + sys.exit("error: --power requested but power.csv is missing/empty") + if "Trace collection stopped!" in open(os.path.join(outdir, "session.log"), + errors="replace").read(): + sys.exit("error: trace stream died mid-run (unknown trace data packet) - " + "the profile only covers up to that point. Retry with " + "--no-timestamps, or reduce the core clock (see SKILL.md).") + prof = open(profile_out, errors="replace").read() + m = re.search(r"^\s*Total\s*\|[\d ]*\|\s*([\d ]+)$", prof, re.M) + if not m or int(m.group(1).replace(" ", "") or 0) == 0: + sys.exit("error: session completed but NO trace data was collected " + "(profile totals are zero) - trace signal not reaching the " + "probe: check wiring/connector, trace pinmux, sample timing.") + print(f"\ncapture OK: {outdir}") + print(f" code_profile.txt ({os.path.getsize(profile_out)} bytes)") + if args.trace_csv: + print(f" itrace.csv ({os.path.getsize(itrace_out)} bytes)") + print(f"analyze: python3 {os.path.dirname(os.path.abspath(__file__))}" + f"/etm_profile.py {outdir} --elf {args.elf}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.claude/skills/etm-trace/scripts/etm_profile.py b/.claude/skills/etm-trace/scripts/etm_profile.py new file mode 100644 index 000000000..8568bc608 --- /dev/null +++ b/.claude/skills/etm-trace/scripts/etm_profile.py @@ -0,0 +1,467 @@ +#!/usr/bin/env python3 +"""Analyze an etm_capture.py output dir: hot functions, coverage, itrace digest, +ISR episode timing, and optimization hints. + +Input: code_profile.txt (Ozone Export.CodeProfile text report), and optionally + itrace.csv (Export.Trace raw instruction history) with --elf for + address->function mapping (arm-none-eabi-nm). +Output: markdown report on stdout. + +--isr SYM[,SYM..] episode timing for an interrupt handler: first symbol is the + entry anchor (its first instruction marks each ISR entry), all symbols + form the body address set. Example: --isr OTG_HS_IRQHandler,dcd_int_handler + Durations are calibrated against SysTick_Handler beats (1 ms apart), so + the itrace must be captured with timestamps enabled. +""" + +import argparse +import bisect +import csv +import os +import re +import statistics +import subprocess +import sys + + +def parse_profile(path): + """Parse the two report sections. Function rows are indented 2 spaces under + a non-indented module row; columns are '|'-separated.""" + lines = open(path, errors="replace").read().splitlines() + try: + cov_start = lines.index("Code Coverage Summary") + prof_start = lines.index("Code Profile Summary") + except ValueError: + sys.exit(f"error: {path} is not an Ozone code-profile text report") + + def rows(section): + module = None + for ln in section: + if "|" not in ln: + continue + cells = ln.split("|") + name = cells[0].rstrip() + if (not name or name.startswith("Module/Function") + or set(name.strip()) <= {"-", "+"}): + continue + if not name.startswith(" "): + module = name.strip() + continue + yield module, name.strip(), cells[1:] + + def num(s): + s = s.strip().replace(" ", "") + return int(s) if s else 0 + + cov_pat = re.compile(r"^\s*([\d ]+)/\s*([\d ]+)\s+([\d.]+)%") + funcs, totals = {}, {} + for module, name, cells in rows(lines[prof_start:]): + run, fetch = (num(cells[0]) if cells else 0), (num(cells[1]) if len(cells) > 1 else 0) + if name == "Total": + totals["run"], totals["fetch"] = run, fetch + elif name == "[Unaccounted]": + totals["unaccounted"] = fetch + elif name in funcs and funcs[name]["module"] != module: + # same-named static from another module: keep both rows distinct + funcs[f"{name} [{module}]"] = {"module": module, "run": run, + "fetch": fetch} + else: + funcs[name] = {"module": module, "run": run, "fetch": fetch} + for module, name, cells in rows(lines[cov_start:prof_start]): + m_src = cov_pat.match(cells[0]) if cells else None + m_inst = cov_pat.match(cells[1]) if len(cells) > 1 else None + if name == "Total" and m_inst and m_src: + totals["src_cov"] = num(m_src.group(1)), num(m_src.group(2)) + totals["inst_cov"] = num(m_inst.group(1)), num(m_inst.group(2)) + elif m_inst: + # match the de-collided key when a same-named static from another + # module was renamed during the profile pass + key = name if (name in funcs and funcs[name]["module"] == module) \ + else f"{name} [{module}]" + if key in funcs: + funcs[key]["inst_pct"] = float(m_inst.group(3)) + return funcs, totals + + +def load_symbols(elf): + """Sorted (addr, size, name) from nm; for mapping itrace addresses.""" + nm = "arm-none-eabi-nm" + out = subprocess.run([nm, "-S", "--defined-only", "-C", elf], + capture_output=True, text=True) + if out.returncode != 0: + sys.exit(f"error: {nm} failed on {elf}: {out.stderr.strip()}") + syms = [] + for ln in out.stdout.splitlines(): + parts = ln.split(maxsplit=3) + if len(parts) == 4 and parts[2].lower() in ("t", "w"): + syms.append((int(parts[0], 16) & ~1, int(parts[1], 16), parts[3])) + elif len(parts) == 3 and parts[1].lower() in ("t", "w"): + # sizeless symbol (e.g. weak asm stub): assume a 2-byte body + syms.append((int(parts[0], 16) & ~1, 2, parts[2])) + return sorted(syms) + + +def addr_to_func(syms, addr): + i = bisect.bisect_right(syms, (addr, 1 << 62, "")) - 1 + if i >= 0 and syms[i][0] <= addr < syms[i][0] + syms[i][1]: + return syms[i][2] + return None + + +def sym_ranges(syms, names): + """(lo, hi) address ranges for the named symbols (base name match); + None if any symbol is missing (caller degrades gracefully).""" + out = [] + for want in names: + for a, sz, n in syms: + if n == want or n.split("(")[0] == want: + out.append((a, a + sz)) + break + else: + print(f"note: symbol '{want}' not found in ELF") + return None + return out + + +def iter_itrace(path): + """Yield (t_raw, addr) chronologically-reversed (file order: newest first). + Also returns the timestamp unit from the header via generator .send? No - + caller reads unit separately with itrace_unit().""" + with open(path, newline="", errors="replace") as f: + rd = csv.reader(f) + hdr = next(rd, None) or [] + # --no-timestamps captures drop the Timestamp column entirely: locate + # the Address column from the header and yield t=None for such rows + # (consumers count instructions but skip time math) + has_ts = any("Timestamp" in c for c in hdr) + try: + addr_i = next(i for i, c in enumerate(hdr) if "Address" in c) + except StopIteration: + addr_i = 1 if has_ts else 0 + for row in rd: + if not row or len(row) <= addr_i: + continue + try: + addr = int(row[addr_i], 16) + except ValueError: + continue + if has_ts and row[0] != "PC": + try: + yield float(row[0]), addr + continue + except ValueError: + pass + yield None, addr + + +def itrace_unit(path): + hdr = open(path, errors="replace").readline() + m = re.search(r"Timestamp\[([^\]]+)\]", hdr) + return m.group(1) if m else "?" + + +def time_by_func(path, syms): + """Per-function raw-time and instruction attribution from the itrace. + Rows are newest-first; the gap to the next (older) row is attributed to the + older instruction's function. Outlier gaps (trace-block boundaries) are + capped so one discontinuity cannot skew a function. Shares are scale-free.""" + sample = [] + t_prev = None + for t, _ in iter_itrace(path): + if t is None: + continue + if t_prev is not None and t_prev - t > 0: + sample.append(t_prev - t) + if len(sample) >= 200000: + break + t_prev = t + cap = 10000 * statistics.median(sample) if sample else float("inf") + t_prev = None + tf, cf = {}, {} + n = 0 + for t, a in iter_itrace(path): + n += 1 + fn = addr_to_func(syms, a) + if fn: + cf[fn] = cf.get(fn, 0) + 1 + if t is None: + continue + if t_prev is not None: + d = t_prev - t + if 0 < d < cap and fn: + tf[fn] = tf.get(fn, 0) + d + t_prev = t + return tf, cf, n + + +def isr_report(itrace, elf, isr_arg, top): + syms = load_symbols(elf) + names = [s.strip() for s in isr_arg.split(",") if s.strip()] + body = sym_ranges(syms, names) + tick = sym_ranges(syms, ["SysTick_Handler"]) + if not body or not tick: + print("\n## ISR timing: skipped (missing symbols above - needs the ISR " + "symbol(s) and SysTick_Handler for calibration)") + return + entry = body[0][0] + unit = itrace_unit(itrace) + + usb_rows, tick_rows, tmin, tmax = [], [], None, None + for t, a in iter_itrace(itrace): + if t is None: + continue # --no-timestamps capture: the <20-rows message below applies + tmin = t if tmin is None else min(tmin, t) + tmax = t if tmax is None else max(tmax, t) + if any(lo <= a < hi for lo, hi in body): + usb_rows.append((t, a)) + # independent, not elif: when the ISR under test IS SysTick_Handler, + # its rows must still feed the calibration + if any(lo <= a < hi for lo, hi in tick): + tick_rows.append((t, a)) + usb_rows.reverse() + tick_rows.reverse() + if len(tick_rows) < 20: + print(f"\n## ISR timing: not enough SysTick beats for calibration " + f"({len(tick_rows)} rows) - capture with timestamps enabled") + return + + # rough raw-units-per-1ms from the large mode of consecutive tick deltas; + # threshold from the median, not the max - one trace-overflow gap would + # otherwise inflate the cut and leave only outliers in the sample + deltas = [b[0] - a[0] for a, b in zip(tick_rows, tick_rows[1:])] + big = [d for d in deltas if d > 10 * statistics.median(deltas)] + raw_ms = statistics.median(big) if big else statistics.median(deltas) + gap = 0.03 * raw_ms # 30 us in raw units + edge = 0.05 * raw_ms + + def episodes(rows, g): + out, cur = [], [] + for t, a in rows: + if cur and t - cur[-1][0] > g: + out.append(cur) + cur = [] + cur.append((t, a)) + if cur: + out.append(cur) + return out + + starts = [e[0][0] for e in episodes(tick_rows, 0.1 * raw_ms)] + + def local_scale(t): + i = bisect.bisect_left(starts, t) + if 0 < i < len(starts): + sp = starts[i] - starts[i - 1] + if 0 < sp < 3 * raw_ms: + return 1e-3 / sp + return 1e-3 / raw_ms + + eps = [] + for cluster in episodes(usb_rows, gap): + cur = None + for t, a in cluster: + if a == entry: + if cur: + eps.append(cur) + cur = [(t, a)] + elif cur: + cur.append((t, a)) + if cur: + eps.append(cur) + eps = [e for e in eps if e[0][0] - tmin > edge and tmax - e[-1][0] > edge + and len(e) >= 10] + print(f"\n## ISR timing: {names[0]} (+{len(names) - 1} body syms), " + f"unit '{unit}', {len(starts)} SysTick beats") + if not eps: + print("- no complete episodes in window (wrong symbols? window missed " + "the traffic phase?)") + return + stats = sorted(((e[-1][0] - e[0][0]) * local_scale(e[0][0]), len(e), e[0][0]) + for e in eps) + cal = [s[0] for s in stats] + print(f"- episodes: {len(cal)} | fastest {min(cal) * 1e6:.2f} us, " + f"median {statistics.median(cal) * 1e6:.2f} us, " + f"avg {statistics.mean(cal) * 1e6:.2f} us, " + f"worst {max(cal) * 1e6:.2f} us") + insts = [s[1] for s in stats] + print(f"- instructions/episode: min {min(insts)}, avg " + f"{statistics.mean(insts):.0f}, max {max(insts)}") + print("- worst episodes (duration, instructions, raw start):") + for c, n, t0 in stats[-5:][::-1]: + print(f" - {c * 1e6:8.2f} us {n:5d} instr t={t0:.6f}") + print("- caveats: timestamps are interpolated between packets (sub-us " + "values are approximate); episodes may merge if trace overflow " + "dropped an entry") + + +def short(name): + return re.sub(r"\(.*\)$", "()", name) + + +def branch_bias(insts_csv, funcs, top): + """One-sided conditional branches in executed functions (profile_insts.csv): + a conditional fetched N times but taken/executed 0 or N times = a branch + that never varied -> hot always-true assert, dead path, or an invariant + that could hoist out of a loop (UM08025 SS5.19).""" + def n(v): + v = (v or "").replace(" ", "") + return int(v) if v.lstrip("-").isdigit() else 0 + biased = [] + with open(insts_csv, newline="", errors="replace") as f: + for r in csv.DictReader(f): + if (r.get("Is Conditional") or "0").strip() != "1": + continue + fetched = n(r.get("Times Fetched")) + executed = n(r.get("Times Executed")) + if fetched < 1000: # only hot conditionals matter + continue + if executed == 0 or executed == fetched: + biased.append((fetched, r.get("Function", "?"), + r.get("Address", ""), r.get("AsmCode", "").strip(), + "always-taken" if executed == fetched else "never-taken")) + if not biased: + return + biased.sort(reverse=True) + print(f"\n## One-sided hot branches (profile_insts.csv)\n") + print("Conditionals that never varied - candidates to hoist/remove " + "(UM08025 §5.19):") + for fetched, fn, addr, asm, kind in biased[:top]: + print(f"- `{short(fn)}` @{addr} {kind} ({fetched:,}x): `{asm[:50]}`") + + +def suggestions(funcs, totals, tshare, cshare, exclude): + """Rule-based optimization hints (after UM08025 §5.19).""" + print("\n## Optimization hints") + total_fetch = totals.get("fetch") or 1 + ex = [n for n in funcs if exclude and re.search(exclude, n)] + ex_fetch = sum(funcs[n]["fetch"] for n in ex) + if ex: + print(f"- excluded from load ranking (--exclude): {len(ex)} functions, " + f"{100.0 * ex_fetch / total_fetch:.1f}% of fetches") + rest = {n: v for n, v in funcs.items() if n not in ex} + rt = sum(v["fetch"] for v in rest.values()) or 1 + hot = sorted(rest.items(), key=lambda kv: -kv[1]["fetch"])[:5] + print("- top load after exclusion: " + ", ".join( + f"`{short(n)}` {100.0 * v['fetch'] / rt:.1f}%" for n, v in hot)) + polls = [(n, v) for n, v in rest.items() + if v["fetch"] / total_fetch > 0.02 and v["run"] + and v["fetch"] / v["run"] < 40] + if polls: + print("- busy-poll candidates (>2% load, <40 instr/entry - called in a " + "tight loop; consider event-driven or rate-limiting):") + for n, v in sorted(polls, key=lambda kv: -kv[1]["fetch"]): + print(f" - `{short(n)}`: {v['run']:,} calls, " + f"{v['fetch'] / v['run']:.0f} instr/call, " + f"{100.0 * v['fetch'] / total_fetch:.1f}% load") + if tshare: + stalls = [] + for n, ts in tshare.items(): + cs = cshare.get(n, 0) + if ts > 0.01 and cs and ts / cs > 4: + stalls.append((n, ts, ts / cs)) + if stalls: + print("- stall/wait-dominated (time share >> instruction share - " + "waiting on hardware, consider DMA/IRQ instead of polling):") + for n, ts, ratio in sorted(stalls, key=lambda x: -x[1])[:5]: + print(f" - `{short(n)}`: {100 * ts:.1f}% of time, " + f"{ratio:.0f}x its instruction share") + + +def main(): + p = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + p.add_argument("capture_dir", help="etm_capture.py output dir") + p.add_argument("--top", type=int, default=10, help="table size") + p.add_argument("--elf", help="firmware ELF, enables itrace analyses") + p.add_argument("--isr", help="entry[,body..] symbols for ISR episode timing") + p.add_argument("--exclude", help="regex of functions to exclude from the " + "optimization load ranking (e.g. the idle poll loop)") + args = p.parse_args() + + profile = os.path.join(args.capture_dir, "code_profile.txt") + itrace = os.path.join(args.capture_dir, "itrace.csv") + funcs, totals = parse_profile(profile) + total_fetch = totals.get("fetch") or 1 + hot = sorted(funcs.items(), key=lambda kv: kv[1]["fetch"], reverse=True)[:args.top] + + print(f"# ETM profile: {args.capture_dir}\n") + print(f"## Top {args.top} hottest functions (instruction-fetch share)\n") + print("| # | Function | Module | Run Count | Fetch Count | Load % |") + print("|---|----------|--------|-----------|-------------|--------|") + for i, (name, v) in enumerate(hot, 1): + print(f"| {i} | `{short(name)}` | {v['module']} | {v['run']:,} " + f"| {v['fetch']:,} | {100.0 * v['fetch'] / total_fetch:.2f}% |") + print(f"\nTotal fetches {totals.get('fetch', 0):,} " + f"(runs {totals.get('run', 0):,}, unaccounted {totals.get('unaccounted', 0):,})\n") + + ic, sc = totals.get("inst_cov"), totals.get("src_cov") + print("## Coverage (NOPs excluded)\n") + if ic: + print(f"- instructions fully executed: {ic[0]:,} / {ic[1]:,} " + f"({100.0 * ic[0] / ic[1]:.1f}%)") + if sc: + print(f"- source lines fully covered: {sc[0]:,} / {sc[1]:,} " + f"({100.0 * sc[0] / sc[1]:.1f}%)") + partial = [n for n, v in funcs.items() + if v["fetch"] > 0 and 0 < v.get("inst_pct", 100) < 100] + print(f"- executed but only partially covered: {len(partial)} functions") + dead = sorted(n for n, v in funcs.items() + if v["fetch"] == 0 and "(always inlined)" not in n) + print(f"- never-executed out-of-line functions: {len(dead)}") + by_mod = {} + for n in dead: + by_mod.setdefault(funcs[n]["module"], []).append(short(n)) + for mod in sorted(by_mod, key=str): + print(f" - {mod}: {', '.join('`%s`' % f for f in by_mod[mod])}") + + tshare, cshare = {}, {} + if os.path.isfile(itrace) and args.elf: + syms = load_symbols(args.elf) + tf, cf, n = time_by_func(itrace, syms) + tt, tc = sum(tf.values()) or 1, sum(cf.values()) or 1 + tshare = {k: v / tt for k, v in tf.items()} + cshare = {k: v / tc for k, v in cf.items()} + unit = itrace_unit(itrace) + print(f"\n## Instruction history (itrace.csv, unit '{unit}')\n") + if not tf: + print(f"- {n:,} instructions, NO timestamps (--no-timestamps " + f"capture): time shares unavailable, top {args.top} by " + f"instruction share:") + for name, cs in sorted(cshare.items(), key=lambda kv: -kv[1])[:args.top]: + print(f" - `{short(name)}`: {100 * cs:.1f}% instructions") + else: + print(f"- {n:,} instructions; top {args.top} by TIME share " + f"(vs instruction share):") + for name, ts in sorted(tshare.items(), key=lambda kv: -kv[1])[:args.top]: + print(f" - `{short(name)}`: {100 * ts:.1f}% time, " + f"{100 * cshare.get(name, 0):.1f}% instructions") + + lines_csv = os.path.join(args.capture_dir, "profile_lines.csv") + if os.path.isfile(lines_csv): + def n(v): # Ozone groups thousands with spaces + v = (v or "").replace(" ", "") + return int(v) if v.isdigit() else 0 + with open(lines_csv, newline="", errors="replace") as f: + rows = [r for r in csv.DictReader(f) + if r.get("File") and n(r.get("Instructions Fetched")) > 0] + rows.sort(key=lambda r: -n(r["Instructions Fetched"])) + print(f"\n## Hottest source lines (profile_lines.csv)\n") + for r in rows[:args.top]: + src = (r.get("Content") or "").strip() + print(f"- {os.path.basename(r['File'])}:{r['Line']} " + f"{n(r['Instructions Fetched']):,} fetches `{src[:60]}`") + + insts_csv = os.path.join(args.capture_dir, "profile_insts.csv") + if os.path.isfile(insts_csv): + branch_bias(insts_csv, funcs, args.top) + + if args.isr: + if not (os.path.isfile(itrace) and args.elf): + sys.exit("error: --isr needs itrace.csv (--trace-csv capture) and --elf") + isr_report(itrace, args.elf, args.isr, args.top) + + suggestions(funcs, totals, tshare, cshare, args.exclude) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.claude/skills/hil-pool-check/SKILL.md b/.claude/skills/hil-pool-check/SKILL.md new file mode 100644 index 000000000..6a8f66087 --- /dev/null +++ b/.claude/skills/hil-pool-check/SKILL.md @@ -0,0 +1,64 @@ +--- +name: hil-pool-check +description: Use when asked for a pool check or board/probe health scan on a TinyUSB HIL rig, when probes or boards are offline or fail to flash, after rig maintenance, reboot, or re-cabling, or before starting a HIL test campaign. +--- + +# HIL Pool Check (board/probe health) + +Health-scan the HIL board pool with `test/hil/helper/hil_pool_check.py`: per board it checks the flash +probe is on the USB bus, flashes a light example (`device/dfu_runtime`; host-only boards get +`host/device_info`, verified by serial output), waits for the board's uid to re-enumerate, +applies safe per-device recovery (probe authorized-toggle, board reset), re-parks with +`board_test`, and prints a summary table plus a USB topology report. Flags and details: `--help` +and the module docstring. + +**REQUIRED BACKGROUND:** the `hil` skill owns config-by-hostname selection (run `hostname` +first) and the board-lock protocol. Locked boards are reported 🔒 locked and skipped — never +waited on, never bypassed; a needed build peeks the lock first. A CI worker reaching a board the +pool check holds fails it as "board locked" — prefer running between CI runs. + +## A "pool check" means the full check + +A request for a "pool check" means the DEFAULT full check below. Use `--scan-only` only when the +user explicitly asks for a quick look, or when you have VERIFIED a CI sweep is mid-run right now +(`python3 test/hil/helper/hil_lock.py status` shows `hil_test.py` holders) — "CI might be running" is not +that predicate: the full check is already lock-safe (CI-held boards report 🔒 locked and are never +touched), so an unconfirmed suspicion is no reason to downgrade. In either scan case say which +mode ran and why; never silently substitute the scan for the full check. + +```bash +python3 test/hil/helper/hil_pool_check.py # full check: ~10 s + ~1-2 s/board with firmware built; + # first run on an unbuilt tree takes minutes (it builds) +python3 test/hil/helper/hil_pool_check.py --scan-only # USB presence only, <1 s, no locks/flashing/building +python3 test/hil/helper/hil_pool_check.py -b BOARD [-b …] # subset; may name boards-skip (parked) entries + +# from a dev PC, against the ci rig (bash -lc: flashers like STM32_Programmer_CLI live in ~/bin): +ssh ci.lan 'bash -lc "cd ~/code/tinyusb && python3 test/hil/helper/hil_pool_check.py"' +``` + +## Notes + +Missing firmware is **built on the spot** — never skipped (`--no-build` opts out; those boards +then report `flash-failed`). Builds need the family env, exported on the rig in +`~/.profile`/`~/.bashrc`: `PICO_SDK_PATH` for rp2040/rp2350 (`~/code/pico/pico-sdk`), the +ESP-IDF env (`get-idf`) for espressif — which also needs `esptool` on PATH (pip's +`~/.local/bin/esptool`; a non-login shell may lack it — run via `bash -lc`). An explicit `-B` is +searched exclusively for *existing* firmware; builds still land in `cmake-build/` and are noted +`built <example>`. Espressif boards park too when the IDF env is present. A first run on an +unbuilt tree builds for many minutes: the Bash tool caps a foreground timeout at 10 min, so run +it in the BACKGROUND and NEVER cancel early — a killed run leaves detached cmake/ninja children +still writing to `cmake-build/` with the board locks held under a protected reason. + +Statuses: `ok` (flashed and verified; in `--scan-only` it only means the probe is present), +`flash-failed` (firmware delivery failed: probe missing, build failed, flasher error, silent +no-op, park unverified), `failed` (check ran but did not verify), `locked` (flock held; +untouched). Exit code = `flash-failed` + `failed` (clamped at 125); `locked` and scan-only rows +are *unverified*, not healthy — read the footer, not just `$?`. A `⚠ pid … source says …` note +means stale firmware or a silent flash no-op (J-Link lore); a device off the bus entirely needs +the usb-kernel-recover skill or a physical replug. + +## Reporting + +The user-facing answer to a pool check IS the tool's summary table: paste the complete per-board +table (and footer counts) verbatim — never truncate rows or reduce it to a prose digest like +"27/27 healthy"; at most one line of commentary below it. diff --git a/.claude/skills/hil/SKILL.md b/.claude/skills/hil/SKILL.md index d61ec05e4..093f345b2 100644 --- a/.claude/skills/hil/SKILL.md +++ b/.claude/skills/hil/SKILL.md @@ -1,53 +1,94 @@ --- name: hil -description: Use when running TinyUSB Hardware-in-the-Loop (HIL) tests on physical boards, debugging HIL failures, or copying firmware to the ci.lan test rig. Covers per-host config selection (htpc uses local.json, ci uses tinyusb.json), local execution on either htpc or ci, remote execution over SSH from htpc, and debugging tips. +description: Use when running TinyUSB Hardware-in-the-Loop (HIL) tests on physical boards, when a HIL run fails, hangs, reports a board locked, or produces a report you need to interpret, or when copying firmware to a test rig (ci.lan, hifiphile/tusb, or a dev PC). For board/probe health scans ("pool check") use the hil-pool-check skill instead. --- # Hardware-in-the-Loop (HIL) Testing -Run TinyUSB HIL tests on real boards. **Run `hostname` first** — it tells you which host you are on, which determines the default config and whether remote mode is possible. +Run TinyUSB HIL tests on real boards. **Run `hostname` first** — it tells you which host you are on, which determines the default config and whether remote mode is possible. Rule of thumb: only `ci` and `tusb` are infra rigs; **any other hostname is a dev PC** and uses `local.json`. -| Host | Local config | Remote (SSH → ci.lan)? | -|------|--------------|------------------------| -| `htpc` (dev PC) | `test/hil/local.json` | yes (large pool, `test/hil/tinyusb.json`) | -| `ci` (the rig) | `test/hil/tinyusb.json` (large pool) | no — can't SSH to htpc, and boards are already local | +| Host | Local config | Remote (SSH → ci.lan)? | +|-----------------------------------|--------------------------------------|--------------------------------------------------------| +| `ci` (the rig) | `test/hil/tinyusb.json` (large pool) | no — boards are already local | +| `tusb` (hifiphile's external rig) | `test/hil/hfp.json` | no outbound SSH to dev PCs/ci; SSH-reachable FROM both | +| anything else (a dev PC) | `test/hil/local.json` | yes (large pool, `test/hil/tinyusb.json`) | -Default to **local**. Use **remote** only when on `htpc` and the user says `remote`/`ci.lan`. Never attempt remote on `ci`. +Default to **local**. Use **remote** only when on a dev PC and the user says `remote`/`ci.lan`. Never attempt remote on `ci`. + +`tusb` (ssh alias `hifiphile`) is an external rig (hosted by maintainer hifiphile), exercised by the +GitHub CI `hil-tinyusb (hfp.json)` matrix job — **never run HIL against it unless the user explicitly asks.** ## Board locks — the CI runner keeps running The `ci` rig also hosts a GitHub Actions runner that flashes boards and runs HIL as part of CI. Hardware access is arbitrated **per board** with kernel flocks in `/tmp/tinyusb-hil-locks/` — do NOT stop the runner service. -- `hil_test.py` self-locks each board for the duration of its flash+test (holder reason `hil_test.py`). A locked board fails immediately (`<board> Failed: board locked: {holder info}`) without flashing — in CI, re-run the failed job once the lock is released. -- If your `hold` fails and the holder's reason is `hil_test.py`, a CI job is mid-test on that board — wait a few minutes and retry rather than forcing. +- `hil_test.py` self-locks each board for its flash+test (holder reason `hil_test.py`). A locked board fails immediately (`<board> Failed: board locked: {holder info}`) without flashing — in CI, re-run the failed job later; if your `hold` is refused with reason `hil_test.py`, a CI job is mid-test — wait a few minutes and retry rather than forcing. - For hardware work outside `hil_test.py` (JLink/GDB, manual flashing, `usbtest.py`, serial poking), hold the lock first: ```bash -python3 test/hil/board_lock.py hold BOARD [BOARD...] --reason "why" +python3 test/hil/helper/hil_lock.py hold BOARD [BOARD...] --reason "why" # ... hardware work ... -python3 test/hil/board_lock.py release BOARD [BOARD...] +python3 test/hil/helper/hil_lock.py release BOARD [BOARD...] ``` - Never pre-hold boards you are about to run `hil_test.py` on — it self-locks and would treat your own hold as a conflict. -- Rig-wide operations (uhubctl power cycling, pci-rebind — bus renumbering) affect every board: `board_lock.py hold --all --reason "..."` first. -- `board_lock.py status` lists holders. Locks auto-release when the holder process dies (kernel flock); `/tmp` clears on reboot. +- Rig-wide operations (uhubctl power cycling, controller resets — bus renumbering) affect every board: `hil_lock.py hold --all --reason "..."` first. +- `hil_lock.py status` lists holders. Locks auto-release when the holder process dies (kernel flock); `/tmp` clears on reboot. - Forcing past a lock: `HIL_NO_BOARD_LOCK=1 python3 test/hil/hil_test.py ...` bypasses the guard without killing the holder. Only with the user's explicit go-ahead — they accept the risk of colliding with whatever holds the board. -- Caveat until this branch merges to master: CI's checkout of `hil_test.py` does not yet enforce locks — keep dev hardware sessions short and check `gh run list --status in_progress` first. + +## Pool check (board/probe health) + +Board/probe health scanning (`test/hil/helper/hil_pool_check.py`) has its own skill: **hil-pool-check**. +Use it before a HIL campaign, after rig maintenance/reboot, or when boards fail to flash. + +## PR-scoped selection + +`test/hil/helper/hil_select.py` maps a diff to affected boards/tests (used by CI on PRs; fail-open +to the full matrix). Manual use: + +```bash +SEL=$(python3 test/hil/helper/hil_select.py --base master test/hil/tinyusb.json) +FULL=$(printf '%s' "$SEL" | python3 -c "import json,sys; print(json.load(sys.stdin)['full'])") +ARGS=$(printf '%s' "$SEL" | python3 -c "import json,sys; print(json.load(sys.stdin)['args']['tinyusb.json'])") +if [ "$FULL" = "True" ] || [ -n "$ARGS" ]; then + python3 test/hil/hil_test.py -B examples $ARGS test/hil/tinyusb.json # $ARGS empty when full: run everything +else + echo "diff affects nothing on this rig - skip HIL" +fi +``` + +Read `full`, never `args` alone: `args` is empty for BOTH `full: true` (run the whole matrix — a broad or +unclassified change) and "nothing selected" (skip). Skip only when `full` is false AND `args` is empty. + +Unit suites (no hardware), all four run by the `hil-test`/`hil-select-test` pre-commit +hooks: `test_hil_select.py` covers only board selection. The containment work --- bounded +reads, the kill ladders, the build and pool guards --- lives in `test_hil_bounded.py`, +`test_hil_health.py` and `test_hil_util.py`, so run all four when changing `test/hil`: +`for f in test/hil/test/test_*.py; do python3 "$f"; done` (~55s). + +## Pre-flight rig health check + +`hil_test.py` notes any process already in D state when the run starts, as one line above +the table. It never aborts, and it is a hint rather than a diagnosis. What bounds a stuck +run is `HIL_POOL_TIMEOUT` plus the job's `timeout-minutes`; what diagnoses a wedged rig is +the `hil-pool-check` skill. + +See the `usb-kernel-recover` skill for what a real wedge looks like and how to clear it. ## Prerequisites -Examples must be built for the target board(s) — see AGENTS.md "Build" → "All examples for a board" (produces `examples/cmake-build-<board>/`). `-B examples` points `hil_test.py` at that parent folder. +Examples must be built for the target board(s) — see CLAUDE.md "Build" → "All examples for a board" (produces `examples/cmake-build-<board>/`). `-B examples` points `hil_test.py` at that parent folder. (This applies to `hil_test.py`; `hil_pool_check.py` builds its own missing firmware.) ## Arguments - **Board:** `-b BOARD_NAME` for one board; omit to run all boards in the config. - **Pass-through:** `-v`, `-r N`, etc. forwarded unchanged. -If `local.json` is missing on `htpc`, ask the user to supply one (only fall back to `tinyusb.json` if told to). +If `local.json` is missing on a dev PC, ask the user to supply one (only fall back to `tinyusb.json` if told to). ## Local execution -Set `CONFIG` from `hostname` first (`test/hil/local.json` on htpc, `test/hil/tinyusb.json` on ci): +Set `CONFIG` from `hostname` first (`test/hil/local.json` on a dev PC, `test/hil/tinyusb.json` on ci, `test/hil/hfp.json` on tusb): ```bash CONFIG=test/hil/local.json # on ci use: CONFIG=test/hil/tinyusb.json @@ -59,9 +100,7 @@ python3 test/hil/hil_test.py -B examples "$CONFIG" python3 test/hil/hil_test.py -b stm32f723disco -B examples "$CONFIG" ``` -Append pass-through flags (`-v`, `-r 1`, …) to either command as needed. - -## Remote execution (htpc → ci.lan only) +## Remote execution (dev PC → ci.lan only) `test/hil/hil_ci.sh` handles dir setup, scp of test scripts, rsync of firmware (`.elf`/`.bin`/`.hex`), and runs `hil_test.py` on `ci.lan` with `tinyusb.json`: @@ -77,8 +116,44 @@ Env overrides: `REMOTE`, `REMOTE_DIR`, `CONFIG`. Fails fast if the build dir/rep ## Timing -Runs take 2-5 min. Use a timeout ≥ 20 min (1200000 ms). NEVER cancel early. +Runs take 2-5 min per board, but a stuck fleet runs to `HIL_POOL_TIMEOUT` — 60 min +unless the env pins it. The run logs its guard in the startup line; never declare a run +stuck before THAT value has elapsed. +The Bash tool caps a foreground timeout at 10 min, so **run it in the background** and +wait for the completion notification -- never a foreground timeout, which would kill +the run before its own guard can write a report. NEVER cancel early. ## Reporting -Show the output, summarize pass/fail per board. On failure, retry with `-v`; if that's not enough, add temporary debug prints to `hil_test.py`. +The user-facing answer to a HIL run IS the tool's summary table: paste the complete per-board +table (and footer counts) verbatim — never truncate rows or reduce it to a prose digest; at most +one line of commentary below it. + +**First check what sits above the table.** Seven banners can appear there; match on a +PREFIX, since each carries trailing detail and one is a blockquote: + +- `**HIL run abandoned: worker pool timed out after …s.**` — no results were collected this + attempt, so any table below is a PREVIOUS attempt's. Report the abandonment, never those + rows, and never `"pass": true`. +- `**HIL run aborted: a worker raised …**` — same rule: a worker crashed before results + were collected; any table below is stale. Report the abort, never the rows. +- `**HIL run abandoned: the worker pool would not shut down.**` — DIFFERENT: the table + below IS this run's, but the pool could not be shut down afterwards (the job exits + non-zero even if every board passed). Report the results AND the abandonment; never + `"pass": true`. +- `**HIL run selected no boards.**` — the filters intersected to nothing, so there is no + table at all. Report that (and the filter shown), never `"pass": true`. +- `> **Rig note.**` — a process was in D state when the run started. This is NOT a wedge: + a healthy in-flight testusb is uninterruptible for most of every case, and the rig + supports a dev run alongside CI. On its own it is never `wedged: true` and never turns a + green table into `"pass": false`. Mention it only when a board below failed, as the first + thing to check. +- `> **Rig dirty.**` — a process survived SIGKILL and still holds a probe or usbfs node + into the NEXT job. The table below is this run's and can be reported, but say the rig is + dirty: the next job starts degraded and nothing in the harness can clear it. +- `> **Not all verdicts are evidence.**` — one or more workers went blind on sysfs, so + "device not found" from the named boards means "could not tell". Do NOT report their red + cells as broken boards. + +On failure, retry with `-v`; if that's not enough, add temporary debug prints to +`hil_test.py`. diff --git a/.claude/skills/pre-pr/SKILL.md b/.claude/skills/pre-pr/SKILL.md index d4c35f7e5..b96750e4f 100644 --- a/.claude/skills/pre-pr/SKILL.md +++ b/.claude/skills/pre-pr/SKILL.md @@ -15,16 +15,31 @@ Run the software + hardware gate for the current branch. The user invoking this ## 2. Map changes to boards -- For each changed `src/portable/<vendor>/<ip>/` (or `src/portable/<name>/` for single-level ports): families = the `hw/bsp/<family>` directories whose build files reference it — `grep -rl "<vendor>/<ip>" hw/bsp/*/family.cmake hw/bsp/*/family.mk`, then take each matching file's directory name. -- For `src/class/*`, `src/common/*`, `src/device/*`, `src/host/*`, or `src/tusb.c`: broad change — use `stm32f407disco` + `raspberry_pi_pico` PLUS any families from portable changes. -- For `hw/bsp/<family>/...` changes: that family directly. -- Catch-all: any other C/CMake source change (`examples/*`, `test/*`, anything unmatched above) → the representative set `stm32f407disco` + `raspberry_pi_pico`. The boards list must NEVER end up empty — final fallback is `[stm32f407disco]` (full-check throws on an empty list). -- Rig roster: `python3 -c "import json;print([b['name'] for b in json.load(open('test/hil/tinyusb.json'))['boards']])"` -- Pick ONE board per affected family, preferring boards on the rig roster; otherwise the first entry in `hw/bsp/<family>/boards/`. Cap at 4 boards and tell the user which families the cap dropped. +- `python3 test/hil/helper/hil_select.py --base $BASE test/hil/tinyusb.json` → JSON with the affected + bsp `families`, the affected rig `boards`, and per-file `reasons`. `full: true` means a + broad/infra change. +- Affected families = `families` ∪ the family of every name in `boards`. Neither half is + enough alone: only the port and bsp rules fill `families` (a class/core/example change + reports boards but no families), and `boards` only ever names rig boards (an off-rig driver + change — `dcd_samx7x.c` → `same7x`, `boards: {}` — would never be compiled). + - A board's family is the `hw/bsp/<family>/boards/<board>/` directory holding it. + - When `full: true`, `boards` names every rig board and carries no signal — use `families` + alone there, plus the representative set below. +- Sample ONE board per affected family: prefer a rig-roster board of that family, else the + first entry in `hw/bsp/<family>/boards/`. + - Rig roster: `python3 -c "import json;print([b['name'] for b in json.load(open('test/hil/tinyusb.json'))['boards']])"` +- Add the representative set `stm32f407disco` + `raspberry_pi_pico` when `full: true` (broad + change — class/core/common/infra). +- Cap at 4 boards and tell the user which families the cap dropped. A broad change affects ~20 + families, so the order matters: keep `stm32f407disco` and `raspberry_pi_pico` first whenever + their families are affected, then fill from the remaining families (spread across vendors — + don't let one vendor's family names take every slot). The list must NEVER end up empty — + final fallback is `[stm32f407disco]` (full-check throws on an empty list). +- Docs-only (`full: false`, no families, no boards) keeps §1's minimal software-only gate. ## 3. HIL boards -- `hilBoards` = chosen boards that are on the rig roster. This host must be able to reach the rig (per `.claude/skills/hil/SKILL.md`: host `ci` = local, `htpc` = remote). If none qualify, run software-only. +- `hilBoards` = chosen boards that are on the rig roster. This host must be able to reach the rig (per `.claude/skills/hil/SKILL.md`: host `ci`/`tusb` = local, any other host (dev PC) = remote). If none qualify, run software-only. ## 4. Launch @@ -38,5 +53,5 @@ Invoke the Workflow tool: - Per-stage table: unit / build:<board> / size / pvs, then HIL per board — pass/fail with the first error for each failure. - If the hardware result has non-empty `locked` (a CI job held those boards): ask the user with AskUserQuestion — **Force now** (re-invoke `hil-validate` with `force: true` for those boards; user accepts the risk of colliding with a mid-test CI job), **Keep waiting** (re-invoke `hil-validate` for them after a few minutes; ask again if still locked), or **Accept** the partial verdict. Never force without the user's answer. -- Wedged boards: point at `.claude/skills/usb-recover/SKILL.md`. +- Wedged boards: point at `.claude/skills/usb-kernel-recover/SKILL.md`. - End with a clear ship / no-ship verdict and what to fix first. diff --git a/.claude/skills/pvs/run_pvs.sh b/.claude/skills/pvs/run_pvs.sh index 7406a3b4f..ce2cfd7e7 100755 --- a/.claude/skills/pvs/run_pvs.sh +++ b/.claude/skills/pvs/run_pvs.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # Run PVS-Studio static analysis on TinyUSB for a given BOARD. # -# Mirrors the "Static Analysis (PVS-Studio)" section in AGENTS.md / CLAUDE.md: +# Implements the build + analyze flow behind CLAUDE.md "Static Analysis (PVS-Studio)": # - build all examples for BOARD with compile_commands.json exported # - run pvs-studio-analyzer against that compile DB using .PVS-Studio/.pvsconfig # - convert the log to human-readable (errorfile) and SARIF output @@ -64,9 +64,9 @@ fi # --- Analyze --------------------------------------------------------------- echo ">>> Running PVS-Studio analyzer (-j${JOBS})" -# Note: AGENTS.md shows --dump-files, but that scatters .PVS-Studio.i/.cfg dump -# files across the source tree (only useful for debugging false positives). It is -# omitted here to keep the working tree clean; add it back via "$@" if needed. +# Note: --dump-files scatters .PVS-Studio.i/.cfg dump files across the source +# tree (only useful for debugging false positives). It is omitted here to keep +# the working tree clean; add it back via "$@" if needed. pvs-studio-analyzer analyze \ -f "${COMPILE_DB}" \ -R .PVS-Studio/.pvsconfig \ diff --git a/.claude/skills/read-doc/SKILL.md b/.claude/skills/read-doc/SKILL.md new file mode 100644 index 000000000..feaa914af --- /dev/null +++ b/.claude/skills/read-doc/SKILL.md @@ -0,0 +1,89 @@ +--- +name: read-doc +description: Use when you need authoritative hardware/protocol facts from a primary source rather than model memory — an MCU/peripheral datasheet, reference manual (RM/TRM), errata, pinout, register/bitfield layout, memory map, schematic, or the USB spec — before answering register/electrical/timing/protocol questions from training knowledge or the web; or when the user asks to read/open/look up a manual, datasheet, book, or PDF/EPUB from their Calibre library. Requires a local Calibre library at ~/Documents/calibre-library; no-ops if absent. +--- + +# Read Doc + +## Overview + +Some maintainers keep datasheets, manuals, and books in a Calibre library at +`$HOME/Documents/calibre-library/`. For hardware/protocol facts — registers, +bitfields, memory maps, pinouts, electrical/timing specs, errata, USB spec — +read the doc instead of answering from training knowledge or the web. + +Search the library's `metadata.db`, never the filesystem. The database indexes +title, authors, tags, series, publisher, description and the stored filename; +most part numbers live in the tags, which the filesystem does not carry. + +## Gate first + +The library is per-user and usually on a network mount, so test the database +file, not the directory — an unmounted or half-synced mountpoint is still a +directory: + +```bash +[ -f "${CALIBRE_LIBRARY:-$HOME/Documents/calibre-library}/metadata.db" ] && echo present || echo absent +``` + +Absent → the skill does not apply; fall back to normal sources silently (don't +mention the library unless the user named it). + +## When to use + +- About to state a register/bitfield/reset-value/memory-map/pinout/timing spec + for a specific MCU or peripheral. +- User says "read the RP2040 datasheet", "open the CH569 manual", "what does the + STM32H7 RM say about…". + +Not for general concepts, repo/code questions, or when no such doc is likely. + +## Find + +Keywords from `/read-doc <keywords>`, else derived from the question (part +number, peripheral, spec name). `search.py` ANDs them across every metadata +field and prints the best matches first — at most 40, and the header says when +more matched: + +```bash +python3 .claude/skills/read-doc/search.py errata RT1064 # AND (default) +python3 .claude/skills/read-doc/search.py RT1060 RT1064 --any +``` + +Exit 0 matched, 1 nothing matched, 2 bad usage or no library — 2 means the +search never ran, so fix the invocation instead of broadening. + +One match → read it. Several → list and ask via AskUserQuestion. Nothing +(exit 1) → retry with fewer keywords; the part number alone often works where +`<part> datasheet` does not, because words like "datasheet" and "manual" are +rarely in the metadata. `--any` only changes anything with two or more +keywords. Still nothing → say the document is missing rather than answering +from memory. + +Set `CALIBRE_LIBRARY` to search a library elsewhere. + +## Read + +`search.py` prints one `FORMAT path` line per stored file: + +- **PDF** — Read with `pages`; for >10 pages start `pages: "1-20"` (TOC/overview), + report the page count, then read sections on demand. +- **Any other format** (EPUB, MOBI, CHM, ZIP…) — Read has no decoder for these + and returns mojibake rather than an error. Say the document is not in a + readable format; do not paste what Read returned. +- **`MISSING`** — the metadata is real but the file is not on disk (library + mid-sync, or the file was deleted). Report the file as unavailable, not the + document as nonexistent. + +Summarize in one line (title, pages, coverage) and keep as reference context. + +## Common mistakes + +- Searching with `find`/`grep` over the library tree. It sees only truncated + filenames, missing the tags, series and descriptions where part numbers and + errata IDs actually live. Query the database. +- Skipping the gate on a machine with no library. +- Answering a register/spec question from memory when the datasheet is on disk. +- Loading a 1000-page PDF up front instead of TOC-first. +- Requiring all keywords to match — broaden, or use `--any`, on zero hits. +- Treating a `MISSING` file, or an exit 2, as proof the document is absent. diff --git a/.claude/skills/read-doc/search.py b/.claude/skills/read-doc/search.py new file mode 100755 index 000000000..c70d36805 --- /dev/null +++ b/.claude/skills/read-doc/search.py @@ -0,0 +1,112 @@ +#!/usr/bin/env python3 +"""Search the Calibre library by metadata and print matching document paths. + +Usage: search.py KEYWORD [KEYWORD...] all keywords must match (AND) + search.py --any KEYWORD [KEYWORD...] any keyword matches (OR) + +Matches title, authors, tags, series, publisher, description and stored +filename, and prints the exact path to read, best match first. + +Exit 0 matched, 1 nothing matched, 2 bad usage or no library. +""" +import glob +import os +import sqlite3 +import sys +import unicodedata +import urllib.parse + +LIB = os.path.realpath(os.path.expanduser(os.environ.get("CALIBRE_LIBRARY") or "~/Documents/calibre-library")) +DB = os.path.join(LIB, "metadata.db") +LIMIT = 40 + +QUERY = """ +SELECT b.id, b.title, b.path, + (SELECT group_concat(a.name, ', ') FROM authors a + JOIN books_authors_link l ON l.author = a.id WHERE l.book = b.id), + (SELECT group_concat(t.name, ', ') FROM tags t + JOIN books_tags_link l ON l.tag = t.id WHERE l.book = b.id), + (SELECT group_concat(s.name, ', ') FROM series s + JOIN books_series_link l ON l.series = s.id WHERE l.book = b.id), + (SELECT group_concat(p.name, ', ') FROM publishers p + JOIN books_publishers_link l ON l.publisher = p.id WHERE l.book = b.id), + (SELECT c.text FROM comments c WHERE c.book = b.id), + (SELECT group_concat(d.format || '/' || d.name, char(10)) FROM data d WHERE d.book = b.id) +FROM books b +""" + +_authors = None + + +def norm(s): + # NFKC + casefold so MICRO SIGN/GREEK MU, curly quotes and dashes compare equal. + return unicodedata.normalize("NFKC", s).casefold() + + +def resolve(bid, path, fmt, name): + """Absolute path of one format row, or None if the file is not on disk. + + Calibre renames `<author>/<title> (<id>)` when metadata is edited and leaves + the old directory behind, so on a miss retry by the stable book id. + """ + ext = "." + fmt.lower() + exact = os.path.join(LIB, path, name + ext) + if os.path.exists(exact): + return exact + global _authors + if _authors is None: + _authors = {} + for d in os.listdir(LIB): # case-only duplicates exist on a case-sensitive mount + _authors.setdefault(d.lower(), []).append(d) + for author in _authors.get(path.split("/")[0].lower(), ()): + for d in glob.glob(os.path.join(glob.escape(os.path.join(LIB, author)), "* (%d)" % bid)): + for f in sorted(glob.glob(os.path.join(glob.escape(d), "*" + ext))): + return f + return None + + +def main(argv): + match_any = "--any" in argv + keywords = [norm(k) for k in argv if k != "--any"] + if not keywords: + print(__doc__, file=sys.stderr) + return 2 + + if not os.path.exists(DB): + print(f"no Calibre database at {DB}", file=sys.stderr) + return 2 + + db = sqlite3.connect("file:" + urllib.parse.quote(DB) + "?mode=ro", uri=True) + hits = [] + for bid, title, path, authors, tags, series, publisher, comments, files in db.execute(QUERY): + entries = [e.split("/", 1) for e in (files or "").split("\n") if e] + hay = norm(" ".join(x for x in (title, authors, tags, series, publisher, comments) if x) + + " " + " ".join(n for _, n in entries)) + found = sum(k in hay for k in keywords) + if not found or (not match_any and found < len(keywords)): + continue + in_title = sum(k in norm(title) for k in keywords) + hits.append((-found, -in_title, title, authors, tags, bid, path, entries)) + + if not hits: + print("no match") + return 1 + + hits.sort(key=lambda h: h[:3]) # authors/tags may be None and are not comparable + print(f"{len(hits)} book(s)" + (f", showing the {LIMIT} best" if len(hits) > LIMIT else "")) + for _, _, title, authors, tags, bid, path, entries in hits[:LIMIT]: + print(f"\n{title}" + (f" [{authors}]" if authors else "") + (f" tags: {tags}" if tags else "")) + if not entries: + print(" (no file in this library)") + for fmt, name in entries: + p = resolve(bid, path, fmt, name) + print(f" {fmt} {p}" if p else f" {fmt} MISSING (library mid-sync or file deleted)") + return 0 + + +if __name__ == "__main__": + try: + sys.exit(main(sys.argv[1:])) + except BrokenPipeError: + os.dup2(os.open(os.devnull, os.O_WRONLY), sys.stdout.fileno()) + sys.exit(0) diff --git a/.claude/skills/target-debug/SKILL.md b/.claude/skills/target-debug/SKILL.md new file mode 100644 index 000000000..050a697b9 --- /dev/null +++ b/.claude/skills/target-debug/SKILL.md @@ -0,0 +1,386 @@ +--- +name: target-debug +description: Use when TinyUSB firmware — device or host stack — misbehaves on real hardware and capture from the other end can't explain it: a HIL test fails but usbmon shows only Submits with no Completes, the device silently NAKs, wedges, STALLs, babbles, or drops data, EP0 starves, tuh_ enumeration of an attached device fails, an ISR or DCD/HCD state bug is suspected — and you need target-side evidence: TU_LOG/RTT logs, GDB state dumps, a RAM ring-buffer event trace, or PC-sampling of where the core spins. +--- + +# target-debug — target-side capture & debugging on the HIL rig + +The **target** is whichever MCU runs TinyUSB — device stack (`dcd_*`), host +stack (`hcd_*`/`tuh_*`), or both. Its link peer is not always a Linux PC: a +TinyUSB host may face another TinyUSB board or a Linux gadget (e.g. a +Raspberry Pi). Pick capture channels by which end runs Linux, not by habit: + +| Skill | Answers | Exists when | +|--------------------|----------------------------------------------------|---------------------------------------------------| +| `usbmon` | what the Linux host exchanged (URBs) | a Linux PC is the link's host | +| `usb-kernel-debug` | why the Linux kernel acted (dmesg / dynamic debug) | Linux on either end: PC host or Linux gadget peer | +| **`target-debug`** | **what the target did** (logs, driver state, PC) | always — either role, needs a debug probe | +| `usb-sniffer` | what crossed the wire (PIDs, handshakes, resets) | hardware tap cabled in — role-agnostic | +| `etm-trace` | exactly which instructions executed (profile, coverage, history) | SEGGER J-Trace wired to this board's trace header — confirm with the user first | + +For enumeration/transfer bugs the default posture is **dual-side capture** — +both ends simultaneously: usbmon + a target +channel when a Linux PC is the host; TinyUSB-as-host has no usbmon on either +end — target channel + the wire (`usb-sniffer`), plus `usb-kernel-debug` on a +Linux gadget peer. + +## Rig discipline — lock first, always + +Hold the board lock for the WHOLE manual session; never stop the +actions-runner (see the `hil` skill for the full lock protocol): + +```bash +python3 test/hil/helper/hil_lock.py hold <board> --reason "target debug: <bug>" +# ... instrument / build / flash / capture / GDB ... +python3 test/hil/helper/hil_lock.py release <board> +``` + +Board → probe mapping: `test/hil/tinyusb.json` — `flasher.name` is the probe +family, `flasher.uid` the **probe serial** (many identical probes on the rig): + +- Select the probe by serial: J-Link `-SelectEmuBySN <uid>`, its GDB server + `-select usb=<uid>`, OpenOCD `-c 'adapter serial <uid>'`. +- `JLINK_DEVICE` / `OPENOCD_OPTION`: from + `hw/bsp/<family>/boards/<board>/board.cmake` (or `board.mk`); family via + `ls -d hw/bsp/*/boards/<board>`. +- Run on the host that owns the probe — config `test/hil/tinyusb.json` on ci, + `test/hil/hfp.json` on tusb, `local.json` on any other host (dev PC) (`hil` skill). +- Espressif boards (S3/P4): different toolchain, probe model, and PHY + constraints entirely — read `esp-target-debug` first. + +## Pick the least intrusive technique that can answer the question + +Observation can mask the bug — the ch32v307 Heisenbug changed behavior under +logging *and* under the debugger. If the bug disappears when instrumented, +that IS a finding (timing-sensitive): move down in intrusiveness, not up. + +| Technique | Intrusiveness | Reach for it when | +|------------------------------------|--------------------------------|---------------------------------------------------------| +| PC-sampling | none — no halt, no code change | core wedged/spinning somewhere unknown (rusb2 FRDY) | +| SWO exception trace / hw PC-sample | none — needs SWO pin wired | ISR ordering/timing with zero code change | +| DWT data trace | none — needs SWO pin wired | stream one address's accesses: value + accessor PC | +| Vector catch | none until a fault fires | crash-shaped wedges — autopsy AT the faulting pc | +| RAM ring-buffer | ~tens of cycles per event | ISR ordering/timing bugs (musb babble) | +| TU_LOG (RTT) | µs per line | logic bugs that survive logging (J-Link or OpenOCD rtt) | +| TU_LOG (UART) | ms per line — blocking write | same, when no debug-probe RTT path | +| dprintf / conditional breakpoint | halt+resume per hit (~ms) | low-rate probes post-wedge; never ISR-rate events | +| GDB halt / breakpoints | stops USB service entirely | post-mortem state autopsy once wedged | + +## PC-sampling, SWO trace & DWT data trace — watch without halting + +`DWT_PCSR` (0xE000101C) returns the current PC on every read, target running +(Cortex-M3+; optional on M0+, reads 0 if absent; 0xFFFFFFFF = core halted or +WFI-asleep — `mem32 E000EDF0, 1`, DHCSR bit 17 S_HALT, tells which). One +probe serves one client: quit JLinkExe before starting JLinkGDBServer on the +same probe. Nailed the rusb2 FRDY wedge: + +```bash +for i in $(seq 300); do echo 'mem32 E000101C, 1'; done \ + | JLinkExe -device $JLINK_DEVICE -SelectEmuBySN <uid> -if swd -speed 4000 -autoconnect 1 -nogui 1 \ + | awk '/E000101C = /{print $3}' | sort | uniq -c | sort -rn | head +arm-none-eabi-addr2line -e <firmware.elf> -f -a 0x<hot-pc> ... # PCs → functions +``` + +OpenOCD variant: repeat `mdw 0xE000101C` over telnet :4444. The histogram's +top entries are the spin site; a flat histogram = core is servicing normally. + +### SWO — hardware-timed trace on one pin (J-Link; verified on F407) + +If SWO (TRACESWO) is wired, DWT emits packets with ZERO code change: +**exception trace** (DWT_CTRL bit16 — every IRQ enter/exit, timestamped) and +**hardware PC sampling** (bit12), better histograms than DWT_PCSR polling. +SWOViewer tools decode only ITM *stimulus* (TinyUSB emits none) — capture +raw: + +```bash +# JLinkExe -CommandFile: +w4 E0001000, 0x00011401 # EXCTRCENA|PCSAMPLENA|SYNCTAP|CYCCNTENA +SWOStart 4000000 # explicit speed — autodetect fails headless +Sleep 3000 +SWORead # hex: 0x17+4B LE = PC sample, 0x0E+2B = IRQ enter/exit +``` + +Verified: 680 KB in 3 s (flash-range PC samples + SysTick enter/exit). +SWORead stuck at 0 = SWO pin not wired (many boards route only SWDIO/SWCLK). +Restore DWT_CTRL when done. + +### DWT data trace — stream one variable's accesses (value + PC), zero code + +The watchpoint comparators' non-halting sibling (ARMv7-M ARM Table C1-21; +absent on ARMv6-M): emit a packet on every access to a watched address +instead of halting. Verified on F407 (J-Link) and H743 (OpenOCD/ST-Link) — +both streamed `system_ticks`' live value plus the accessor PC +(`tusb_time_millis_api`): + +```bash +w4 E0001020, <&variable> # DWT_COMP0 (JLinkExe shown; OpenOCD: same via mww) +w4 E0001024, 0 # DWT_MASK0 = exact address +w4 E0001028, 0x3 # FUNCTION 0b0011: value + accessor-PC packets (0b0010: value only) +# stream: 0x47+4B = accessor PC, 0x87+4B = value, 0x70 = timestamp +``` + +Caveats: traces reads AND writes (no write-only encoding) — a variable the +main loop polls floods the pipe with read packets and squeezes out value +packets (seen on F407); disarm (`FUNCTION=0`) when done; costs one of the +DWT comparators. + +### Enabling SWO — the chain, and the vendor part that bites + +DEMCR.TRCENA → ITM (TCR/TER) → SWO/TPIU (protocol + prescaler) → pin mux. +Tools set the first three (`SWOStart` on SEGGER; `swo`/`tpiu` object +`enable` + `itm ports on` on OpenOCD) — pin mux and trace clocks are +per-family: + +- STM32F4: debug pins default to trace — nothing to configure. +- STM32H7 (verified, ST-Link): DBGMCU trace clocks + **PB3 muxed to AF0 by + hand** + native `stlink-dap.cfg` (the hla transport's tpiu path silently + does nothing) + the cfg-provided `stm32h7x.swo` object (`stm32h7x.tpiu` + is the parallel port — rejects uart). traceclk = c_ck 400 MHz, not HCLK: + too-slow guesses give ratio-garbled bytes, too-fast gives silence. + +```bash +openocd -f interface/stlink-dap.cfg -c 'adapter serial <uid>' -f target/stm32h7x.cfg -c init \ + -c "mww 0x5C001004 0x00700000" \ + -c 'set m [read_memory 0x58020400 32 1]; mww 0x58020400 [expr {([lindex $m 0] & ~0xC0) | 0x80}]' \ + -c 'set a [read_memory 0x58020420 32 1]; mww 0x58020420 [expr {[lindex $a 0] & ~0xF000}]' \ + -c "stm32h7x.swo configure -protocol uart -traceclk 400000000 -pin-freq 2000000 -output /tmp/swo.bin" \ + -c "stm32h7x.swo enable" -c "itm ports on" \ + -c "sleep 3000" -c "stm32h7x.swo disable" -c shutdown # then decode /tmp/swo.bin +``` + +## Vector catch + fault autopsy — catch the crash, not the wedge + +A wedge that is really a fault (HardFault loop, lockup) autopsies best AT +the faulting instruction. Two hardware-proven gotchas: **FPB/DWT comparators +survive reflash and dead sessions** — a stale one fires as a phantom SIGTRAP +at an unrelated line of NEW firmware — and J-Link's reset strategy manages +vector-catch bits: scrub first, arm AFTER reset: + +```gdb +# scrub: FP_COMP0..5 = 0xE0002008..201C, DWT_FUNCTIONn = 0xE0001028 + n*0x10 +set *(unsigned*)0xE0002008 = 0 +# ... (repeat per comparator; count = the GDB section's budget reads) +# arm (after monitor reset; tool-agnostic — works via JLinkExe w4 too): +set *(unsigned*)0xE000EDFC |= (1<<10)|(1<<9)|(1<<8)|(1<<7)|(1<<6)|(1<<5)|(1<<4) +# = VC_HARDERR|INTERR|BUSERR|STATERR|CHKERR|NOCPERR|MMERR; bit0 VC_CORERESET halts at reset +``` + +OpenOCD native: `cortex_m vector_catch hard_err bus_err state_err chk_err mm_err`. +It halts at exception ENTRY (pc = handler, LR = EXC_RETURN 0xFFFFFFFx); decode: + +```gdb +p/x *(unsigned*)0xE000ED28 # CFSR — low byte MemManage, byte1 BusFault, top half UsageFault +p/x *(unsigned*)0xE000ED2C # HFSR — bit30 FORCED = an escalated lower-priority fault +p/x *(unsigned*)0xE000ED38 # BFAR — faulting address (valid if CFSR bit15 BFARVALID) +x/8wx $msp # stacked frame: r0 r1 r2 r3 r12 lr pc xpsr — pc = culprit +# frame is on PSP when EXC_RETURN bit2 is set (LR = 0xFFFFFFFD — FreeRTOS +# tasks run on PSP): then x/8wx $psp instead. LR 0xFFFFFFF1/E9 = MSP. +``` + +`addr2line -e <elf> <stacked pc>` names the line (verified: CFSR 0x8200, +BFAR = the bad address, stacked pc = the faulting ldr). Loads fault +precisely; stores usually IMPRECISERR (BFAR invalid, pc late). ARMv6-M has no +CFSR/BFAR, only VC_HARDERR|VC_CORERESET — stacked frame alone. Still a halt +(host URB timeouts apply); clear DEMCR (`&= ~0x7F0`) before handing back; +RISC-V: breakpoint the trap handler; mcause/mepc/mtval are the CFSR/BFAR analogs. + +## RAM ring-buffer trace + +The zero-print instrument (cracked the musb babble): a small event ring in the +dcd/hcd, dumped over GDB after the failure. Single-writer (ISR) — no locking: + +```c +typedef struct { uint16_t ev; uint16_t a; uint32_t b; } dbg_ev_t; +#define DBG_N 512 // power of two +static volatile dbg_ev_t dbg_ring[DBG_N]; // volatile REQUIRED: -Os dead-store- +static volatile uint32_t dbg_wr; // eliminates a write-only static array +static inline void DBG_EV(uint16_t ev, uint16_t a, uint32_t b) { + uint32_t i = dbg_wr++; + dbg_ring[i & (DBG_N - 1)] = (dbg_ev_t){ ev, a, b }; +} +// call sites: DBG_EV(__LINE__, ep_addr, count); — __LINE__ as event id +``` + +After building, `nm` the ELF for `dbg_ring`/`dbg_wr` — if they're missing the +compiler deleted your instrument and the run will "reproduce" with an empty ring. + +Order is the index; if durations matter add a `uint32_t t = DWT->CYCCNT` field +(enable once: `CoreDebug->DEMCR |= CoreDebug_DEMCR_TRCENA_Msk; DWT->CTRL |= 1;` +RISC-V: read `mcycle`). Let the failure happen, halt, then: + +```gdb +p dbg_wr # total events; oldest slot = dbg_wr & (DBG_N-1) once wrapped +p dbg_ring +dump binary memory /tmp/ring.bin &dbg_ring[0] &dbg_ring[512] +``` + +## TU_LOG capture + +Build with `LOG=2` (`LOG=3` adds per-transfer noise and much more timing skew). +`LOGGER=rtt` routes it over the debug probe — no UART wiring. SEGGER's host +tools need a J-Link, but OpenOCD serves the same RTT buffer on ST-Link / +CMSIS-DAP / WCH-Link boards: + +```bash +# RTT: JLinkGDBServer from CLAUDE.md "GDB Debugging" + -RTTTelnetPort, then: +timeout 20s JLinkRTTClient > /tmp/rtt.log # non-interactive capture +# UART (board's debug serial, if wired): +stty -F /dev/ttyACM<N> 115200 raw && timeout 20s cat /dev/ttyACM<N> | tee /tmp/uart.log +``` + +```bash +# OpenOCD RTT (any probe OpenOCD drives) — in telnet :4444 (or -c equivalents): +rtt setup 0x20000000 0x8000 "SEGGER RTT" # RAM ORIGIN + LENGTH (from the .ld/map) +rtt start # after firmware booted; rerun after each reflash +rtt server start 19021 0 +# then: timeout 20s nc localhost 19021 > /tmp/rtt.log +``` + +OpenOCD polls — bursty logs can drop lines; prefer J-Link where both +exist. The drain-model warning below applies unchanged. + +An RTT-built firmware that has since wedged still holds a log tail in RAM — +but ONLY what fits the drain model: the default SEGGER mode (NO_BLOCK_SKIP) +**drops** writes once the ring fills with no reader, so an undrained target +holds the first KB after boot, not the wedge tail. There is no overwrite mode +in stock SEGGER RTT (only SKIP/TRIM/BLOCK): post-mortem RTT is evidence only +if a live drain was running — otherwise instrument with the RAM ring above. +Use `JLinkGDBServer -RTTTelnetPort 19021` + `JLinkRTTClient` for the drain +(proven; note the server briefly halts the core on connect). `JLinkRTTLogger` +fails to find the control block on some parts (LPC4088) even when it exists +and even given `-RTTAddress`; don't fight it — `nm` the ELF for `_SEGGER_RTT`, +read the aUp[0] descriptor (`mem32`), `savebin` the buffer — debug-AP RAM +reads don't halt the target. + +## GDB — state autopsy and watchpoints + +Connect/load recipes per probe family (J-Link, OpenOCD for ST-Link / +CMSIS-DAP / WCH-Link) are in CLAUDE.md "GDB Debugging"; script sessions with +JLinkGDBServer `-singlerun` — the server exits with the connection, and +back-to-back relaunches race the probe handle and hang. Release builds keep +DWARF (`MinSizeRel`), so `p`/struct access works on HIL firmware. + +**Autopsy of a wedged board: attach and halt ONLY** — skip CLAUDE.md's +`monitor reset halt` + `load` (those are for fresh starts; a reset destroys +the evidence). Symbolize with the ELF that is actually flashed — +`<build root>/cmake-build-<board>/<example>/<example>.elf` from the run that +wedged; do not rebuild while the wedge is still on the board. The debug-loop +specifics: + +```gdb +p/x _usbd_dev.ep_status # device stack: usbd [epnum][dir] (1=IN): busy/stalled/claimed +p _usbh_devices[0] # host stack: usbh per-device state (addr, enum/config) +p/x <port's private state> # per-port names — read the board's dcd_*.c first +x/32wx <USB peripheral base> # raw EP/FIFO regs; base = the macro the dcd uses +watch xfer_status[2][1].total_len # HW watchpoint (Cortex-M: ~4); dwc2 names shown +break dcd_int_handler # works, but see warning below +``` + +**Hardware budget — read it off the chip** (verified: F407/M4 = 6 bp + 4 wp, +rp2040/M0+ = 4 + 2; M7 typically 8/4): + +```gdb +p ((*(unsigned*)0xE0002000)>>4) & 0xF # FPB NUM_CODE = hw breakpoints (M7 adds bits[14:12]) +p (*(unsigned*)0xE0001000)>>28 # DWT_CTRL NUMCOMP = watchpoint comparators +``` + +- `hbreak`/`thbreak` force a hardware breakpoint (software breaks in flash + need flash-breakpoint support — J-Link has it; OpenOCD: `bp <addr> 2 hw`); + `tbreak` = one-shot. +- `watch -l <expr>` watches the address expr evaluates to once — almost + always what you want; `rwatch`/`awatch` trap reads/any access (hardware- + only — they error, never fall back). OpenOCD adds a data-VALUE match GDB + can't express: `wp <addr> 4 w <value> [mask]` — catch who writes 0 into a + busy flag, ignoring writes of 1. +- **Demand the word "Hardware" in the confirmation.** `watch` silently falls + back to a software watchpoint when no DWT comparator fits — GDB then + single-steps the whole program, hundreds of times slower: certain USB + death. Plain `Watchpoint 2:` = delete it and narrow the expression + (`watch -l`, cast to a 4-byte int). +- Conditional breaks (`break ... if ep_addr==0x81`) and `dprintf + <loc>,"fmt",args` (printf without recompiling; keep `dprintf-style gdb`) + are host-evaluated — no Cortex-M agent expressions in our stubs: every hit + halts+resumes (~ms) even when the condition is false. Post-wedge/cold + paths only; ISR-rate events belong in the RAM ring. +- `commands <bpnum> ... end` (start `silent`, end `continue`) auto-collects + evidence per hit — same halt-per-hit cost. +- Stepping with the USB ISR firing between steps is chaos: OpenOCD + `cortex_m maskisr steponly`. The bus runs either way — the host may still + reset a halted-looking device. +- Poking state while halted (`set var _usbd_dev.ep_status[2][1].busy = 0`) + tests a hypothesis but invalidates the post-mortem — dump first, poke after. +- FreeRTOS examples: `-rtos GDBServer/RTOSPlugin_FreeRTOS` (OpenOCD: `-rtos + FreeRTOS`) → `info threads` lists every task with state/prio/frame + (verified: 6 tasks). It populates only after a + run→stop cycle — plain attach shows one 0xDEAD placeholder. Semihosting is + never the answer (traps + halts per call — RTT instead). **Monitor-mode + debugging** (J-Link, M3+) keeps chosen IRQs serviced at a breakpoint — + needs SEGGER's JLINK_MONITOR files + `SetMonModeDebug=1`; not set up here: + <https://kb.segger.com/Monitor_Mode_Debugging> (untested). + +While halted the device answers **nothing**: host control transfers time out +in ~5 s and the OS may reset/re-enumerate — after `continue`, the bus traffic +shows recovery, not the original bug. Prefer one halt for a post-mortem dump +over stepping through live USB traffic. + +## Dual-side capture — the default for enumeration/transfer bugs + +Start both channels, then trigger the failing test (Linux-PC-host shown; +TinyUSB-as-host: swap usbmon for `usb-sniffer`, + `usb-kernel-debug` on a +Linux gadget peer): + +```bash +.claude/skills/usbmon/scripts/usbcap.sh cafe: 30 /tmp/host.pcapng & # host URBs (usbmon skill) +timeout 30s JLinkRTTClient > /tmp/target.rtt & # target (or ring dump after) +wait +``` + +RTT lines and ring events carry no wall-clock: correlate on unambiguous +anchors — bus reset, SET_ADDRESS, the first transfer on the failing EP — then +lay device events between anchors in host-URB order. Logging the SOF/frame +number on the target gives a shared clock when you need finer alignment. +When host and target evidence disagree, or the host sees nothing at all, add +the wire itself: `usb-sniffer` skill (hardware tap, PID-level). + +## Manuals + +- J-Link (UM08001): <https://kb.segger.com/UM08001_J-Link_/_J-Trace_User_Guide> — flash breakpoints, RTT, SWO, monitor mode, Commander. +- OpenOCD: <https://openocd.org/doc/html/index.html> — `rtt`, `bp`/`wp`, `cortex_m vector_catch`/`maskisr`, `itm`/`tpiu`. +- "Debugging with GDB" (§5.1 = break/watch/dprintf): Tenth Edition (GDB 18) + via the `read-doc` skill, or + `curl -sL -o /tmp/gdb.pdf https://sourceware.org/gdb/current/onlinedocs/gdb.pdf` + (the HTML mirror blocks fetchers). Installed `arm-none-eabi-gdb` + `help <cmd>` is authoritative here. + +## Warnings + +- **Halting/resetting via the probe does NOT disconnect the device**: a DWC2 + soft-connect pullup stays up through core halt *and* reset, so the host's + stuck URBs stay stuck and a wedged DUT stays wedged — recover the Linux + host side with the `usb-kernel-recover` skill. +- **A bug that vanishes under LOG=2 is a timing bug**, not fixed: switch to + the ring buffer; if it vanishes under GDB too, PC-sampling only. +- **UART TU_LOG blocks in the write path** (worst perturbation, including + inside the ISR); RTT is much cheaper but not free; `LOG=3` multiplies both. +- Flash/GDB only with the board lock held; a `hold` refused with reason + `hil_test.py` means CI is mid-test on that board — wait, don't force. +- **Instrumentation is temporary**: before `release`, reflash pristine + firmware (the next CI run must not inherit a debug build) and revert the + instrumentation diff — or hand it over explicitly with the diagnosis. +- **A register snapshot without a validity anchor lies**: J-Link tool sessions + can reset or briefly halt the DUT as a side effect, and a snapshot of a + freshly-reset chip (e.g. NVIC ISER = 0) reads like a smoking gun. Read DHCSR + (0xE000EDF0: bit 17 S_HALT, bit 25 S_RESET_ST) with every snapshot, and + cross-check against something the device demonstrably still does. +- **"Flash OK" can lie** (silent no-op — old firmware keeps running). When + behavior contradicts the flashed code: `objcopy -O binary fw.elf + /tmp/fw.bin`, then `verifybin /tmp/fw.bin,<flash-base>` (J-Link, verified) + or `verify_image` (OpenOCD); on mismatch reflash before debugging further. +- **A marginal link can fake a deterministic firmware bug** — down to failing + the same test at the same iteration twice. "USB disconnect" in dmesg on a + freshly re-cabled port (high devnum = churn) means the plug, not the code: + first sustained bulk traffic is when a bad contact drops. Before declaring a + regression, re-run the OLD build on the SAME link state — and if a bisect + exonerates every hunk, believe it: re-test the exact failing binary. +- **Release your manual lock before `hil_test.py`** — it self-locks each board + and fails immediately on your own hold (`hil` skill). diff --git a/.claude/skills/usb-kernel-debug/SKILL.md b/.claude/skills/usb-kernel-debug/SKILL.md new file mode 100644 index 000000000..fd291c0ac --- /dev/null +++ b/.claude/skills/usb-kernel-debug/SKILL.md @@ -0,0 +1,47 @@ +--- +name: usb-kernel-debug +description: Use when USB enumeration fails or misbehaves and packet/URB capture can't explain WHY the Linux kernel acted — port reset storms, repeated re-enumeration, address errors, xHCI ring/command errors, "device descriptor read error", babble — on whichever end of the link runs Linux: the PC host when testing a TinyUSB device, or a Linux gadget peer (e.g. Raspberry Pi) when testing the TinyUSB host stack. +--- + +# usb-kernel-debug — Linux kernel dynamic debug for USB + +Kernel **dynamic debug** shows the Linux side's *reasoning* that packet +capture can't: port resets and their causes, enumeration retries, address +(re)assignment, EP halts, xHCI ring/command errors. It applies wherever Linux +sits in the link — the rig PC when it is the host, or a Linux gadget peer +(dwc2/UDC + gadget modules) when TinyUSB is the host. It cannot see inside +the TinyUSB MCU — that is the `target-debug` skill. + +Run this skill's `scripts/usb_dyndbg.sh` with `sudo`. It flips the dynamic-debug +print flag for an allowlisted set of USB modules only: + +```bash +# all examples below abbreviate: sudo .claude/skills/usb-kernel-debug/scripts/usb_dyndbg.sh +sudo usb_dyndbg.sh on usbcore xhci_hcd # enable +p; pick modules from `lsusb -t` Driver= +sudo usb_dyndbg.sh status [module] # list enabled print sites +sudo usb_dyndbg.sh off usbcore xhci_hcd # ALWAYS turn off when done — very noisy +``` + +Allowlisted modules: `usbcore xhci_hcd xhci_pci xhci_pci_renesas ehci_hcd +ehci_pci ohci_hcd ohci_pci uhci_hcd dwc2 dwc3 cdc_acm usb_storage uas +libcomposite udc_core` (`dwc2`/`dwc3` + the last two cover a Linux gadget +peer's device side). + +## Workflow + +1. `sudo usb_dyndbg.sh on usbcore <hcd-module>` — `usbcore` for enumeration/hub + logic, plus the controller module (`lsusb -t` shows the driver per bus). + On a gadget peer: `dwc2` (or `dwc3`) + `udc_core` + `libcomposite` instead — + run on the peer itself (its SSH/serial console); the script is self-contained, + copy it over or use the raw `dynamic_debug/control` writes from the `usbmon` + skill. +2. Reproduce (replug / re-enumerate / rerun the failing test) while following + `sudo dmesg -w` (or grab `sudo dmesg | tail` afterwards). +3. `sudo usb_dyndbg.sh off ...` — leaving it on floods the log and skews timing. + +On a Linux-PC-host link, pair with the `usbmon` skill: usbmon for what crossed +the bus, dynamic debug for why the kernel reacted. A gadget peer's UDC has no +usbmon — pair with `usb-sniffer` on the wire instead. For a wedged device/bus +on the rig PC use the `usb-kernel-recover` skill. + +Requires `CONFIG_DYNAMIC_DEBUG` and mounted debugfs (standard on distro kernels). diff --git a/.claude/skills/usb-kernel-debug/scripts/usb_dyndbg.sh b/.claude/skills/usb-kernel-debug/scripts/usb_dyndbg.sh new file mode 100755 index 000000000..3923cdc6d --- /dev/null +++ b/.claude/skills/usb-kernel-debug/scripts/usb_dyndbg.sh @@ -0,0 +1,47 @@ +#!/usr/bin/env bash +# usb_dyndbg.sh — toggle kernel dynamic-debug on USB drivers (host or gadget +# side); run with sudo. Flips +p/-p only on an allowlisted set of USB modules, +# so it can't reach arbitrary kernel debug or unrelated subsystems. +# +# Usage: +# sudo usb_dyndbg.sh on <module>... # enable +p (e.g. usbcore xhci_hcd) +# sudo usb_dyndbg.sh off <module>... # disable -p +# sudo usb_dyndbg.sh status [module] # show enabled sites (or one module's sites) +set -euo pipefail + +CTL=/sys/kernel/debug/dynamic_debug/control +# Allowlist: USB core + host-controller + common class drivers, plus the +# gadget/UDC side of a Linux peer (dwc2/dwc3, udc_core, libcomposite). +ALLOW='usbcore xhci_hcd xhci_pci xhci_pci_renesas ehci_hcd ehci_pci ohci_hcd ohci_pci uhci_hcd dwc2 dwc3 cdc_acm usb_storage uas libcomposite udc_core' + +die() { echo "usb_dyndbg: $*" >&2; exit 1; } +usage() { + echo "usage: usb_dyndbg.sh {on|off} <module>... modules: $ALLOW" >&2 + echo " usb_dyndbg.sh status [module]" >&2 + exit 2 +} +allowed() { local m; for m in $ALLOW; do [ "$m" = "$1" ] && return 0; done; return 1; } + +[ -e "$CTL" ] || die "dynamic_debug unavailable (need CONFIG_DYNAMIC_DEBUG + debugfs mounted)" + +action=${1:-}; shift || true +case "$action" in + on|off) + [ "$#" -ge 1 ] || usage + flag='+p'; [ "$action" = off ] && flag='-p' + for m in "$@"; do allowed "$m" || die "module not allowlisted: $m"; done + for m in "$@"; do echo "module $m $flag" > "$CTL"; echo "dynamic debug $action: $m"; done + ;; + status) + m=${1:-} + if [ -n "$m" ]; then + allowed "$m" || die "module not allowlisted: $m" + grep -E "\[$m\]" "$CTL" || echo "(no sites for $m)" + else + grep -E '=p( |$)' "$CTL" || echo "(no print sites enabled)" + fi + ;; + *) + usage + ;; +esac diff --git a/.claude/skills/usb-kernel-recover/SKILL.md b/.claude/skills/usb-kernel-recover/SKILL.md new file mode 100644 index 000000000..009090769 --- /dev/null +++ b/.claude/skills/usb-kernel-recover/SKILL.md @@ -0,0 +1,206 @@ +--- +name: usb-kernel-recover +description: Use when a USB device or fixture on a HIL rig's Linux host (ci.lan, hifiphile/tusb, a bench PC) is wedged, not enumerating, or when processes touching USB (testusb, JLinkExe, uhubctl, openocd, libusb tools) hang in D state. Linux-host side only — a bus owned by a TinyUSB host is out of reach. +--- + +# USB Recovery on the HIL Rig (Linux kernel side) + +**The rule:** a wedged usbfs ioctl holds that device's `device_lock` +(`usbdev_do_ioctl` takes `usb_lock_device`, the uninterruptible variant — +v6.12.96 devio.c:2609) and the driver under it waits in a plain +`wait_for_completion()` with no timeout (usbtest.c:1404; `usb_sg_wait`, +message.c:765). Nothing that also takes that lock can help. Only two levers +don't: **failing the URB at the device** (rung 1) and **the port-side data-line +drop** (rung 2). + +## 1. Triage: find the holder + +```bash +ps -eo pid,stat,etimes,wchan:22,args | awk '$2 ~ /D/' +sudo cat /proc/<pid>/stack # never opens the node, so it cannot block +``` + +- **`S` = victim.** Lock-taking sysfs *reads* use `usb_lock_device_interruptible` + (sysfs.c:124-139, 11 sites), so readers are killable and `timeout` bounds them. + Ignore them; they unwind by themselves. +- **`D` = the holder, or a writer that took the uninterruptible path.** + +| Stack shows | Meaning | Go to | +|---|---|---| +| `usbdev_ioctl` + a driver module (`[usbtest]`) | **owner, holds the lock** | rung 3 — terminal | +| `usbdev_ioctl`, no driver frames | owner waiting on a URB | rung 1 (DUT) / rung 2 (probe) | +| `usbdev_open`, sysfs reads | victim | ignore | +| `tee .../usbtest/new_id`, `bind`, `unbind` | **victim that SPREADS it** | stop issuing them | +| `hub_event` in a kworker | teardown stuck behind an owner | rung 3 | + +Driver-bind writes are not passive: `__device_driver_lock` (drivers/base/dd.c) +takes `device_lock()` uninterruptibly **and `device_lock(parent)`**, because +`usb_bus_type` sets `.need_parent_lock = true` (driver.c:2048) — each one holds +the HUB's lock, which is how one wedged port takes a whole bus down. + +Map the holder to a busport with **lock-free attrs only** (`devnum`, `idVendor`, +`idProduct` are `usb_descriptor_attr*`, plain `sysfs_emit`, sysfs.c:688-705): + +```bash +for d in /sys/bus/usb/devices/<bus>-*/; do + [ "$(cat $d/devnum)" = "<devnum>" ] && echo "$d $(cat $d/idVendor):$(cat $d/idProduct)" +done +grep -l <SERIAL> /sys/bus/usb/devices/*/serial # only on a HEALTHY device +``` + +## 2. Shield first (prerequisite for anything using libusb) + +A wedged device blocks every enumerator that reads its locking attributes — +JLinkExe, uhubctl, openocd's HID fallback. `chmod 000` makes the VFS reject the +read before `->show()` runs, so they skip it and keep enumerating: + +```bash +for f in bNumInterfaces bmAttributes bMaxPower configuration bConfigurationValue \ + product manufacturer serial avoid_reset_quirk; do + sudo chmod 000 /sys/bus/usb/devices/<busport>/$f +done +``` + +- Shield the **leaf, its parent hub, and the root hub** (`usb<N>`) — a stuck + uhubctl locks the root hub too. +- **Run the recovery tool as NON-root**: root has `CAP_DAC_OVERRIDE`, ignores the + `000`, and blocks anyway. +- **Only those nine.** `descriptors`, `busnum`, `devnum`, `speed`, `idVendor`, + `idProduct` are lock-free and libusb needs them; a blanket `chmod` breaks + enumeration instead of fixing it. +- `chmod` never blocks (inode setattr, no `show()`), so it works on a fully + wedged device. +- **Not needed for openocd pinned with `vid_pid`** — it matches the cached + descriptor and skips a foreign device before `libusb_open` + (cmsis_dap_usb_bulk.c:107, bulk backend; the HID fallback ignores the pin). +- Leaf shields vanish on re-enumeration; **the root hub's must be restored**: + `sudo chmod "$(stat -c %a /sys/bus/usb/devices/usb<healthy>/$f)" …/usb<N>/$f` + +## 3. The rungs — go straight to the one triage names + +**Rung 1 — wedged DUT: reset it through its own probe.** + +```bash +printf "r\ng\nq\n" > /tmp/rec.jlink +JLinkExe -device <DEV> -if SWD -speed 4000 -SelectEmuBySN <probe-sn> \ + -autoconnect 1 -nogui 1 -CommandFile /tmp/rec.jlink +``` + +Reset **before** park-flash: non-destructive (the firmware under test survives +for autopsy), no flash wear, and no bad park image — a `wfe`/`wfi` park has +bricked SWD on mimxrt1064_evk and max32666fthr through a power cycle. +`ResetTarget` measures 128-129 ms; cleared 57 → 0, 26 → 0 and 5 → 0 D-state +processes, single shot each. Mechanism: chip reset drops the pull-up → +`usb_hcd_flush_endpoint` unlinks the URB `-ESHUTDOWN` (hcd.c:1783) → the +completion fires → the ioctl returns → the lock releases. + +Works on i.MX RT (`USBCMD.RS` = 0 detaches, RT1050 RM Rev 3 p.2453) **and on +DWC2** — measured 2026-08-16 on stm32f407disco: `r; g` gave +`usb 13-2.2: USB disconnect, device number 107`, re-enumerating 325 ms later. +(A bare **halt** does not: the core keeps running with the pull-up asserted.) + +**Park-flash** (`--recover-board`/`--recover-fw`, what `usbtest.py` automates) is +the fallback where the reset cannot reach the peripheral. Delivery must be +convoy-safe: **openocd pinned with `vid_pid`**, or esptool (`-p <ttyACM>`). +JLinkExe selects by serial, which needs `libusb_open`, so it needs the shield. + +**Rung 2 — wedged PROBE: `root-cycle`.** A probe has no probe to reset it, so the +port-side drop is the only lock-free lever left. It commands the ROOT hub and +never touches the wedged device's lock. + +```bash +sudo usb_recover.sh root-cycle <busport> [expected-serial] +``` + +Bounces **every fixture under that root port** (up to 25 here). Renesas `ppps` +disables D+/D− only — VBUS stays up, so it is a forced re-enumeration, not a +power cycle. Success is the sysfs inode changing, not uhubctl's exit code. + +**Rung 3 — terminal case: a driver ioctl that OWNS the lock.** No software cure: +the task is uninterruptible and SIGKILL is queued, not delivered. Reboot with +**sysrq**, never `reboot(2)` — a graceful reboot runs `device_shutdown()`, which +takes every device lock and stalls on the wedged one. + +```bash +echo b | sudo tee /proc/sysrq-trigger # after: sync; sudo umount -a +``` + +**Rung 4 — hypervisor.** ci.lan only, and never needed in eight recorded wedges: +`qm stop <vmid> && qm start <vmid>` from the PVE host. A VM *reboot* is not +reliable — hubs can latch across the PCIe reset. + +## 3b. If the CONTROLLER is dead, not a device + +Signature: `xhci-pci-renesas <addr>: Timeout while waiting for setup device +command`, devices on that controller failing to enumerate, or its buses gone — +as opposed to ONE device wedged. The rungs above cannot help; the controller +itself needs re-initialising. + +```bash +sudo usb_recover.sh pci-rebind <pciaddr> # unbind + bind the whole xHCI +sudo usb_recover.sh pci-bind <pciaddr> # only if it ends up driverless +``` + +Measured on ci.lan 2026-08-17 02:34:41 after a `hub-cycle` failed to take: unbind +deregistered buses 17 and 18, the re-bind registered new buses **1 and 2** one +second later, and every fixture re-enumerated. **It renumbers every bus that +controller owns**, so hold all affected boards' locks first (`hil_lock.py hold +--all`) and re-derive busports afterwards. + +Do NOT reach for it while a device-lock convoy is live — see Common mistakes. + +## 4. If nothing is in D state + +The device is dead or silent, not wedged. `sudo usb_recover.sh authorized +<busport>` unconfigures and reconfigures it (`usb_set_configuration(dev, -1)` +then re-choose, hub.c) — it fixes stale driver/interface state, does **not** +replug: the `usb_device` survives, so most probes keep their sysfs node. If that +does not take, the device is wedged rather than silent — go to rung 1 or 2. +`resolve <dev-node>` maps `/dev/ttyACM3` → busport. + +**It takes `usb_lock_device` uninterruptibly** (hub.c `usb_deauthorize_device`), +so it is safe only while nothing is in D state. + +## 5. Before declaring the rig healthy + +```bash +ps -eo stat,args | awk '$1 ~ /D/' | wc -l # must be 0 +timeout 15 lsusb # rc 0 and a sane device count +sudo uhubctl -l <bus> -p <port> # "0000 off" = never came back +sudo uhubctl -l <bus> -p <port> -a on +``` + +Observed: 5 boards missing with a completely clean D-state list, because +`usb17-port2` sat at `disable=1`. + +## Common mistakes + +- **`uhubctl -a cycle` on a root port without `-S`.** It writes sysfs + `disable`, and `disable_store` takes `usb_lock_device(hdev)` uninterruptibly + then calls `usb_disconnect(child)` inside it (port.c) — against a wedged child + that blocks while holding the root hub's lock, poisoning the bus. + `usb_recover.sh` passes `-S`. +- **`echo 1 > .../remove`** to make a wedged device "go away": `remove_store` is + the one attribute in sysfs.c taking the uninterruptible `usb_lock_device` + (sysfs.c:765). It joins the convoy instead of clearing it. +- **`authorized` on anything wedged** — same uninterruptible lock. Driver + unbind/bind (`/sys/bus/usb/drivers/usb/{unbind,bind}`) does the same + unconfigure/reconfigure via `usb_generic_driver_disconnect` (generic.c) but ALSO + takes the parent hub's lock (`need_parent_lock`), so it is strictly worse; it was + removed from `usb_recover.sh` for that reason. +- **`pci-rebind` for a wedged DEVICE.** It is the cure for a dead CONTROLLER (see + below), not for a device-lock convoy: with a live D-state URB the re-bind can + hang and leave the controller with **no driver** and every fixture offline + (observed once). Recover that with `pci-bind <addr>`. +- **Writing `/sys/bus/pci/devices/<addr>/reset`** — no rig controller has FLR, so + it becomes a bus reset behind a live driver: card halted, host power cycle. +- **Resetting a victim's board.** Two boards were reset innocently before anyone + found the holder. Map by `devnum`, not by which board "should" be running. +- **Assuming one controller.** Observed: 26 D-state processes across three xHCI + controllers, all cleared by one probe reset on one device. + +## Rig layout (ci.lan, bus numbers renumber every boot) + +`readlink -f /sys/bus/usb/devices/usb<N>` → its PCI address. AMD `0000:02:00.0` +has no port-power switching; Renesas `0000:01:00.0` (probe tree) and +`0000:03:00.0`/`0000:05:00.0` (DUT hubs) have real per-port `ppps`. diff --git a/.claude/skills/usb-kernel-recover/scripts/usb_recover.sh b/.claude/skills/usb-kernel-recover/scripts/usb_recover.sh new file mode 100755 index 000000000..876e0938b --- /dev/null +++ b/.claude/skills/usb-kernel-recover/scripts/usb_recover.sh @@ -0,0 +1,205 @@ +#!/usr/bin/env bash +# usb_recover.sh — USB recovery helper for the HIL rig; run with sudo. Writes only +# to the specific sysfs control files below; arg regexes block path traversal. +# +# Usage: +# sudo usb_recover.sh authorized <busport> # e.g. 3-2 -> deauthorize+reauthorize (re-enumerate, NO VBUS cut) +# sudo usb_recover.sh root-cycle <busport> [serial] # e.g. 13-1.6 -> uhubctl port-off/on at the ROOT port feeding +# # it; [serial] is verified against the device and refused on mismatch, +# # skipping the leaf hubs (which fake ganged switching and do not +# # actually cut power). Bounces every sibling under that root port. +# # The D-state escape: no device lock, so it cannot convoy. +# sudo usb_recover.sh pci-rebind <pciaddr> # e.g. 0000:05:00.0 -> unbind+bind the whole xHCI +# # controller. For a DEAD CONTROLLER, not a wedged +# # device: it renumbers every bus it owns. +# sudo usb_recover.sh pci-bind <pciaddr> [drv] # re-attach a driver to a DRIVERLESS controller +# sudo usb_recover.sh resolve <devnode> # e.g. /dev/ttyACM3 -> print its <busport> (no privilege needed) +set -euo pipefail + +USBPATH_RE='^[0-9]+-[0-9]+(\.[0-9]+)*$' +PCI_RE='^[0-9a-fA-F]{4}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}\.[0-9]$' +DRIVER_RE='^[A-Za-z0-9_-]+$' + +die() { echo "usb_recover: $*" >&2; exit 1; } + +lock_read() { + # Read an attribute served under the device lock (serial, product) with a 2s bound. + # Prints the value, '' when the attribute is absent, or '?' when it did not answer. + # + # Bounding these is load-bearing, not defensive: they are the FIRST thing root-cycle + # does, so on a real wedge an unbounded read blocks before reaching uhubctl at all + # (observed live: one attempt sat 3h; three concurrent invocations all frozen there). + # The operator then reads that as "recovery didn't work" and escalates to a bare + # `uhubctl -a cycle`, which tears the subtree down and blocks holding the ROOT HUB + # lock -- taking the whole bus with it. That is how one wedge becomes an incident. + # + # `timeout` is enough, though this said for a while that it was not (claiming the read + # sits in D state, where SIGKILL is not delivered, so timeout waitpid()s forever). It + # does not: v6.12.101 drivers/usb/core/sysfs.c takes the lock for every READ through + # usb_lock_device_interruptible -> device_lock_interruptible -> mutex_lock_interruptible, + # so the waiter sleeps INTERRUPTIBLY and SIGTERM ends it. Uninterruptible is the usbfs + # ioctl HOLDER, not us. The abandon-a-background-reader dance that claim justified is + # gone, and with it a fail-open where an absent attribute answered '?' -- the wedge + # signature, which root-cycle reads as "cannot confirm serial, proceed". + local v rc=0 + # `|| rc=$?`, never a bare assignment: under this script's `set -e` a command + # substitution that FAILS (an absent attribute -- most hubs and probes have no + # iSerialNumber, and `product` is often missing) exits the whole recovery script. + v=$(timeout 2 cat "$1" 2>/dev/null) || rc=$? + [ "$rc" -eq 124 ] && { echo '?'; return; } # timed out: nobody answered + printf '%s\n' "$v" +} + +# Generation marker for "did this device actually re-enumerate". A real disconnect destroys the +# usb_device and its sysfs kobject; reconnecting creates a new one, and kernfs hands out inode +# numbers monotonically, so the directory inode changes. Verified on the rig: ports re-enumerated +# minutes ago carry inodes in the millions while ports untouched since boot are still in the tens +# of thousands, ranking identically to their mtimes. +# +# This beats comparing devnum, which Linux reuses once the per-bus map wraps (observed live: a +# single cycle moved one device 123 -> 113). It also beats watching for the node to vanish, since +# `uhubctl -a cycle` holds the whole power-off window inside itself and a poll afterwards can +# never witness the gap. The inode survives the gap, so no observation window is needed. +# +# Crucially, if the disconnect is blocked on the wedged device's lock the kobject is never +# recreated -- same inode -- which is exactly the case that must be reported as a failure. Verified +# against kernfs: __kernfs_new_node() allocates via idr_alloc_cyclic() but kernfs_id_ino() exposes +# the full 64-bit (id_highbits<<32 | lowbits) as st_ino on 64-bit ino_t, so a repeat needs ~2^64 +# node creations. authorized-toggle, set_configuration and suspend/resume all leave the parent +# device kobject alone, so none of them can move the marker and fake a success. +# +# The trailing slash is load-bearing: /sys/bus/usb/devices/<busport> is a SYMLINK with its own +# separate inode, so without it stat reports the link rather than the device it points at, and the +# value would never change. Do not "tidy" it away. +# Refuse to touch a PCI function that is not a USB controller (class 0x0c03xx), so a stray or +# mistyped BDF can't unbind/reset an unrelated device (storage, NIC) on a shared HIL host. +require_usb_controller() { + local addr=$1 cls + cls=$(cat "/sys/bus/pci/devices/$addr/class" 2>/dev/null) || die "no such pci device: $addr" + [[ "$cls" =~ ^0x0c03 ]] || die "$addr is not a USB controller (class $cls); refusing" +} + +sysfs_gen() { stat -c %i "/sys/bus/usb/devices/$1/" 2>/dev/null || echo none; } +usage() { grep -E '^# sudo usb_recover' "$0" >&2; exit 2; } + +# Resolve a /dev node (ttyACMx, ttyUSBx, sgN, ...) up to its USB device busport. +resolve() { + local node=$1 syspath dev + [ -e "$node" ] || die "no such device node: $node" + syspath=$(udevadm info -q path -n "$node" 2>/dev/null) || die "udevadm failed for $node" + dev="/sys$syspath" + while [ "$dev" != "/sys" ] && [ -n "$dev" ]; do + if [ -e "$dev/busnum" ] && [ -e "$dev/devnum" ] && [ -e "$dev/authorized" ]; then + basename "$dev"; return 0 + fi + dev=$(dirname "$dev") + done + die "could not find parent USB device for $node" +} + +action=${1:-}; target=${2:-} +[ -n "$action" ] && [ -n "$target" ] || usage + +case "$action" in + resolve) + resolve "$target" + ;; + authorized) + [[ "$target" =~ $USBPATH_RE ]] || die "bad usb path: $target" + d="/sys/bus/usb/devices/$target" + [ -e "$d/authorized" ] || die "no such usb device: $target" + echo 0 > "$d/authorized"; sleep 1; echo 1 > "$d/authorized" + echo "re-authorized $target" + ;; + pci-rebind) + [[ "$target" =~ $PCI_RE ]] || die "bad pci addr: $target" + require_usb_controller "$target" + [ -e "/sys/bus/pci/devices/$target/driver" ] || die "no driver bound to $target" + drv=$(basename "$(readlink -f "/sys/bus/pci/devices/$target/driver")") + echo "$target" > "/sys/bus/pci/drivers/$drv/unbind"; sleep 1 + echo "$target" > "/sys/bus/pci/drivers/$drv/bind" + echo "rebound pci $target ($drv)" + ;; + pci-bind) + # Re-attach a driver to a controller left DRIVERLESS (e.g. a pci-rebind whose re-bind hung). + [[ "$target" =~ $PCI_RE ]] || die "bad pci addr: $target" + require_usb_controller "$target" + [ -e "/sys/bus/pci/devices/$target" ] || die "no such pci device: $target" + [ -e "/sys/bus/pci/devices/$target/driver" ] && die "$target already has a driver bound" + drv=${3:-} + if [ -n "$drv" ]; then + [[ "$drv" =~ $DRIVER_RE ]] || die "bad driver name: $drv" + [ -e "/sys/bus/pci/drivers/$drv/bind" ] || die "no such pci driver: $drv" + echo "$target" > "/sys/bus/pci/drivers/$drv/bind" + echo "bound pci $target ($drv)" + else + # Auto-try the xHCI drivers (Renesas uPD720201 uses xhci-pci-renesas; others xhci_hcd). + for cand in xhci-pci-renesas xhci_hcd; do + [ -e "/sys/bus/pci/drivers/$cand/bind" ] || continue + if echo "$target" > "/sys/bus/pci/drivers/$cand/bind" 2>/dev/null; then + echo "bound pci $target ($cand)"; exit 0 + fi + done + die "could not bind $target with a known xHCI driver; pass the driver explicitly" + fi + ;; + root-cycle) + # Port-off/on at the ROOT port. NOTE: the Renesas ppps only disables D+/D- (VBUS stays up), + # so this is a forced re-enumeration, not a power cycle. It goes straight at the root port -- + # no leaf walk (the 1a40:0201 hubs claim ganged switching but never cut power) -- and never + # writes the wedged device's sysfs or takes its lock, so it cannot join a D-state convoy. + # uhubctl exits 0 even when it does nothing ("No compatible devices detected" still returns + # 0), so its status proves nothing -- the sysfs_gen check below is the only real verdict. + [[ "$target" =~ $USBPATH_RE ]] || die "bad usb path: $target" + UHUBCTL=$(command -v uhubctl || echo /sbin/uhubctl) + [ -x "$UHUBCTL" ] || die "uhubctl not installed" + # Existence alone only proves *something* occupies that path -- bus numbers renumber every + # boot, so a stale busport can name a different device entirely and we would cut power to its + # whole subtree (up to 25 fixtures on this rig). Callers that know what they expect pass the + # serial as a third argument and we refuse on mismatch; otherwise print the identity so a + # wrong target is at least visible. + [ -e "/sys/bus/usb/devices/$target" ] || die "no such usb device: $target" + idf="/sys/bus/usb/devices/$target" + serial=$(lock_read "$idf/serial") + expect=${3:-} + if [ -n "$expect" ] && [ "$serial" = '?' ]; then + # Warn and PROCEED: an unreadable serial is the wedge signature itself, so refusing + # here would block the cure on exactly the condition it exists for. The identity + # guard is lost for this call -- say so, because the cost of a wrong target is the + # whole subtree. + echo "root-cycle: WARNING $target's serial did not answer (it is wedged), so '$expect'" \ + "could NOT be confirmed; proceeding, but verify the busport if siblings drop" >&2 + elif [ -n "$expect" ] && [ "$expect" != "$serial" ]; then + die "root-cycle: $target has serial '$serial', expected '$expect' — stale busport, refusing" + fi + # idVendor/idProduct are usb_descriptor_attr_le16: served WITHOUT the device lock, so + # a plain cat is safe on a wedged device. serial/product are usb_string_attr and are not. + echo "root-cycle: target $target is $(cat "$idf/idVendor" 2>/dev/null || echo -):$(cat "$idf/idProduct" 2>/dev/null || echo -)" \ + "serial=$serial product=$(lock_read "$idf/product")" + bus=${target%%-*}; rest=${target#*-}; rootport=${rest%%.*} + gen=$(sysfs_gen "$target") + echo "root-cycle: disabling D+/D- on bus $bus root port $rootport (no VBUS cut; feeds $target, bounces its siblings)" + # -S is load-bearing. By default uhubctl writes /sys/.../usb<bus>-port<n>/disable (observed: + # two O_WRONLY opens per cycle), and the kernel's disable_store() takes the ROOT HUB's lock and + # synchronously usb_disconnect()s the child BEFORE cutting power -- confirmed in v6.12.96 + # drivers/usb/core/port.c: usb_lock_device(hdev), the UNINTERRUPTIBLE variant, then + # usb_disconnect(&port_dev->child) inside it. Against a wedged device that disconnect blocks on + # the very lock we are trying to free, so power never drops and uhubctl D-states holding the + # root hub's lock, poisoning the whole bus. -S forces the libusb path, which sends the + # power-off control transfer straight to the root hub with no child-disconnect in front. + "$UHUBCTL" -S -l "$bus" -p "$rootport" -a cycle -d 5 \ + || die "uhubctl failed to cycle bus $bus port $rootport" + for _ in $(seq 1 10); do + sleep 1 + now=$(sysfs_gen "$target") + if [ "$now" != none ] && [ "$now" != "$gen" ]; then + echo "root-cycled $bus port $rootport: $target re-enumerated"\ + "(devnum $(cat "/sys/bus/usb/devices/$target/devnum" 2>/dev/null || echo ?), gen $gen -> $now)"; exit 0 + fi + done + die "root-cycle: $target did not re-enumerate after cycling bus $bus port $rootport (sysfs generation still $gen: no disconnect happened)" + ;; + *) + usage + ;; +esac diff --git a/.claude/skills/usb-sniffer/SKILL.md b/.claude/skills/usb-sniffer/SKILL.md new file mode 100644 index 000000000..8d070709f --- /dev/null +++ b/.claude/skills/usb-sniffer/SKILL.md @@ -0,0 +1,151 @@ +--- +name: usb-sniffer +description: Use when you need wire-level USB evidence that host-side capture can't provide — a device that never enumerates (usbmon shows nothing or only Submits), suspected NAK storms/STALL/babble/bad handshakes, bus-reset or enumeration timing, split-transaction issues, a usbmon-vs-device-log disagreement the wire must arbitrate, or any link where TinyUSB is the host (no Linux PC host to run usbmon on). Captures LS/FS/HS packets (PIDs, tokens, handshakes, SE0/line states) with the ataradov usb-sniffer hardware into Wireshark pcapng. +--- + +# usb-sniffer — wire-level capture with the ataradov hardware analyzer + +Extends the debugging trio with the layer below URBs: + +| Skill | Answers | +|--------------------|--------------------------------------------------------------------| +| `usbmon` | what a Linux PC host exchanged (URBs) | +| `usb-kernel-debug` | why the Linux kernel acted (dmesg / dynamic debug) | +| `target-debug` | what the TinyUSB target did (device or host role) | +| **`usb-sniffer`** | **what actually crossed D+/D-** (PIDs, handshakes, resets, timing) | + +Reach for it when usbmon can't see (device never binds, pre-enumeration +failures), can't be trusted (URB completed but did the wire really ACK?), or +doesn't exist — a link where TinyUSB is the host has no usbmon on either end +(an MCU host runs no kernel; a Linux gadget peer's UDC bypasses usbmon). +Where a Linux PC is the host, usbmon is cheaper — no hardware, no locks. + +## Rig inventory — find the sniffer and what it taps + +```bash +lsusb -d 6666:6620 # sniffer present? (github.com/ataradov/usb-sniffer) +``` + +The sniffer is a passive tap: host-side and device-side connectors pass +through, the capture port is a separate USB device. What it taps is a cabling +fact you must confirm every session, not assume: start a capture (below), +provoke known control traffic to a candidate (`lsusb -v -s <bus>:<dev> +>/dev/null`), and see whether those requests appear on the wire. The DUT's +link speed (`cat /sys/bus/usb/devices/<port>/speed`) picks `--speed`. + +The tapped board is rig hardware: hold its board lock for any session that +resets or reflashes it (`hil` skill). The sniffer itself is not lockable and +capture alone perturbs nothing. + +## Capture + +The tool is `usb_sniffer` (installed in `~/.local/bin`, extcap-symlinked so +Wireshark's GUI also shows a "USB Sniffer" interface). Headless recipe: + +```bash +timeout 15s usb_sniffer --capture --fifo /tmp/cap.pcapng --speed hs # or fs / ls +``` + +- `--speed` MUST match the DUT's link speed (default is fs!). Wrong speed = + no USB packets, only Syslog pseudo-packets ("Line state: SE0", "VBUS ON"). + If you see only those, fix `--speed` before doubting the hardware. +- ALWAYS bound the capture: `timeout` and/or `--limit N` (packets). HS runs + 15–20 MB/s even with `--fold` when any device on the bus is busy (`--fold` + only collapses truly empty frames). Unbounded HS captures reach GB fast. +- The output is valid pcapng the moment the process dies; a plain file path + works (no FIFO needed). `--trigger low|high|falling|rising` arms capture + on the external trigger pin instead of starting immediately. +- Tool diagnostics: `USB_SNIFFER_LOG=/tmp/sniffer.log usb_sniffer ...` + +Start the capture FIRST, then trigger the event you care about. The proven +one-pass enumeration recipe (`--limit` makes the tool exit by itself; on a +busy HS bus ~470k packets/s ≈ 20 MB/s, so 3M packets ≈ 6–7 s ≈ 120 MB — do +NOT capture for 20+ s "to be safe", the raw balloons and every later tshark +pass pays for it; but do NOT go below ~3M either: J-Link connect latency +varies run-to-run (0.5–4 s) and a 3 s window has provably missed the ladder): + +```bash +usb_sniffer --capture --fifo raw.pcapng --speed hs --fold --limit 3000000 & +sleep 1 +# trigger: full ladder incl. SET_ADDRESS (needs board lock; J-Link resets the MCU): +printf 'r\ng\nqc\n' | JLinkExe -device $JLINK_DEVICE -SelectEmuBySN <probe-uid> \ + -if swd -speed 4000 -autoconnect 1 -nogui 1 +wait # tool prints "Capture limit reached" and exits +``` + +No-probe trigger alternative — kernel-side re-enumeration (may reuse the +xHCI address and skip parts of the ladder; fine for descriptor reads, weak +for reset timing): +`echo 0 | sudo tee /sys/bus/usb/devices/<port>/authorized; sleep 1; echo 1 | sudo tee ...` + +## Reading the capture + +```bash +tshark -r cap.pcapng -Y 'usb.bmRequestType' # the control ladder +tshark -r cap.pcapng -Y 'usb.bDescriptorType == 1' \ + -T fields -e usb.idVendor -e usb.idProduct # VID:PID off the wire +tshark -r cap.pcapng -Y 'usbll.pid' # raw token/handshake level +editcap -r cap.pcapng slice.pcapng <first>-<last> # trim huge captures +``` + +On a capture >100 MB, make exactly ONE filtered pass (the ladder filter +above) to find the frame numbers of your event window, `editcap -r` to that +window, and do all further analysis on the slice — repeated broad tshark +passes over a 300 MB raw are what turn a 5-minute job into 15. + +Find the DUT's wire address from the capture, not from lsusb: the +SET ADDRESS request payload carries it (`00 05 <addr> 00 ...`), and all +subsequent traffic goes to `<addr>.<ep>` (`usbll.addr`). **On xHCI hosts the +lsusb device number is NOT the wire address** — they diverge routinely. +Filter analysis to the DUT: `-Y 'usbll.addr contains "4."'`. + +## What the wire really shows (read before concluding anything) + +- **Downstream is broadcast.** Tokens, SETUP and OUT data addressed to EVERY + device on the tapped bus segment appear in the capture; upstream (DATA in + response to IN) appears only from devices on the tapped branch. Lone + IN→ACK pairs without DATA to some other address are normal, not corruption. +- **The sniffer can capture its own upload.** If its capture port shares the + host controller bus with the tap, its bulk-IN polling floods the capture + (easily >90% of packets) — filter it out by address; for surgically clean + captures move the capture cable to a different host controller. +- **Port-reset visibility depends on the tap point.** Tapping the DUT's own + cable: a reset reaches the sniffer PHY and you get explicit + `--- Bus Reset ---` / `Detected speed:` Syslog records. Tapping a hub + upstream: the hub isolates downstream port resets — no marker appears. Anchor reset timing on the hub choreography instead: + SetPortFeature(PORT_RESET) to the hub's address = reset start, + ClearPortFeature(C_PORT_RESET) = reset end (start the capture before + triggering, or the initiating SetPortFeature is missing from the file). + The DUT's silence gap corroborates, but do not read every gap as a + reset — idle captures contain benign multi-ms gaps. +- **FS device behind an HS hub**: the upstream tap shows SPLIT transactions, + not native FS packets. Tap the DUT's own cable and capture at `fs` for + clean full-speed traffic. + +## Setup (one-time) + +```bash +# udev: tools/88-tinyusb.rules covers 6666:6620 + blank FX2LP 04b4:8613 +sudo cp tools/88-tinyusb.rules /etc/udev/rules.d/ +sudo udevadm control --reload-rules && sudo udevadm trigger -s usb + +# binary (ataradov repo bin/usb_sniffer_linux) + Wireshark extcap symlink (needs Wireshark >= 4.x) +cp usb_sniffer_linux ~/.local/bin/usb_sniffer && chmod +x ~/.local/bin/usb_sniffer +mkdir -p ~/.local/lib/wireshark/extcap +ln -sf ~/.local/bin/usb_sniffer ~/.local/lib/wireshark/extcap/usb_sniffer +``` + +Never run `--mcu-eeprom` / `--fpga-flash` / `--fpga-erase` against a working sniffer — those program NEW hardware. + +## Warnings + +- **Bound every capture** (`timeout` / `--limit`) and delete or `editcap`-trim + multi-hundred-MB raws before handing off; a forgotten capture process fills + the disk at HS rates. +- The tap is passive — capturing, or unplugging the capture port, does not + disturb the DUT's link. Unplugging the pass-through DOES. +- Answers must come from packet payloads (SETUP/DATA hex), not from host-side + logs — that is the whole point of being on the wire; if an answer isn't in + the capture, say so rather than approximating from sysfs/dmesg. +- Release the board lock and leave no capture processes running at session + end (`pgrep -a usb_sniffer`). diff --git a/.claude/skills/usbmon/SKILL.md b/.claude/skills/usbmon/SKILL.md index 4164dcf2e..490c79e7f 100644 --- a/.claude/skills/usbmon/SKILL.md +++ b/.claude/skills/usbmon/SKILL.md @@ -1,15 +1,13 @@ --- name: usbmon -description: Use when capturing, analyzing, or debugging USB bus traffic for TinyUSB device development on Linux — enumeration failures, STALLed control transfers, missing/short bulk or interrupt transfers, isochronous/audio dropouts, or descriptor problems. Captures host-side URBs with usbmon + tshark into a Wireshark pcapng and decodes them. Use whenever you need to see what the host actually exchanged with a device on real hardware, even if the user just says "sniff USB", "capture the enumeration", or "why won't my device enumerate". +description: Use when capturing, analyzing, or debugging USB bus traffic on a link where a Linux PC is the host (TinyUSB in device role) — enumeration failures, STALLed control transfers, missing/short bulk or interrupt transfers, isochronous/audio dropouts, or descriptor problems. Captures host-side URBs with usbmon + tshark into a Wireshark pcapng and decodes them. Not applicable when TinyUSB is the host — no URBs traverse the PC (use usb-sniffer / target-debug). Use whenever you need to see what the Linux host actually exchanged with a device on real hardware, even if the user just says "sniff USB", "capture the enumeration", or "why won't my device enumerate". --- # usbmon — capture & debug USB traffic -`usbmon` records host-side **URBs** — control / bulk / interrupt / isochronous transfers, descriptors, class requests, STALLs, short packets — i.e. exactly what the host exchanged with a device. Use it to debug a TinyUSB device on real hardware. (It's host/URB-level, not wire-level; for SOF/ACK/electrical use a hardware analyzer.) +`usbmon` records host-side **URBs** — control / bulk / interrupt / isochronous transfers, descriptors, class requests, STALLs, short packets — i.e. exactly what the host exchanged with a device. Use it to debug a TinyUSB device on real hardware. (It's host/URB-level, not wire-level; for SOF/ACK/electrical use a hardware analyzer.) It exists only on the Linux host side of a link: when TinyUSB runs the *host* stack (peer = another TinyUSB board or a Linux gadget, e.g. a Raspberry Pi), neither end has usbmon — capture the wire (`usb-sniffer` skill) or instrument the target (`target-debug` skill). -**Setup (assumed in place):** `usbmon` loaded and a udev rule `SUBSYSTEM=="usbmon", GROUP="wireshark", MODE="0640"` with your user in the `wireshark` group — so `tshark` captures with no `sudo`. - -If you were just added to `wireshark` (e.g. `usermod -aG`), the running shell/agent still has the old group set (group adds only apply to a fresh login). Don't restart — wrap each capture in `sg wireshark -c '…'`, which re-reads `/etc/group` immediately: `sg wireshark -c 'tshark -i usbmon3 -s 128 -a duration:30 -w /tmp/cap.pcapng'`. (Reading a finished `.pcapng` with `tshark -r` needs no special group.) For long/high-throughput captures add `-s 128` (snaplen) to keep only URB headers/status, not payloads. +**Setup (assumed in place):** `usbmon` loaded and a udev rule `SUBSYSTEM=="usbmon", GROUP="wireshark", MODE="0640"` with your user in the `wireshark` group — so `tshark` captures with no `sudo`. Freshly added to the group? The running shell doesn't have it yet (group adds need a new login) — wrap captures in `sg wireshark -c 'tshark -i usbmon3 -s 128 -a duration:30 -w /tmp/cap.pcapng'`; reading a finished `.pcapng` (`tshark -r`) needs no group. `-s 128` (snaplen) keeps only URB headers/status, not payloads — use it for long/high-throughput captures. ## Capture @@ -31,18 +29,18 @@ tshark -r cap.pcapng -Y 'usb.device_address==26' # filter to one device ## Filter (`-Y '<expr>'`) -| Goal | Expression | -|---|---| -| One device / endpoint | `usb.device_address==26` / `usb.endpoint_address==0x81` | -| IN (to host) / OUT (from host) | `usb.endpoint_address.direction==1` / `==0` | -| Submit / Complete event | `usb.urb_type=='S'` / `=='C'` (char literal: single quotes) | -| Control / bulk / interrupt / iso | `usb.transfer_type==2` / `3` / `1` / `0` | -| Only transfers carrying data | `usb.data_len>0` | -| GET_DESCRIPTOR / SET_ADDRESS / SET_CONFIGURATION | `usb.setup.bRequest==6` / `5` / `9` | -| SET_INTERFACE / CLEAR_FEATURE (clear-halt) | `usb.setup.bRequest==11` / `1` | -| Descriptor type DEVICE/CONFIG/STRING/HID-report | `usb.bDescriptorType==1` / `2` / `3` / `0x22` | -| Class / vendor requests | `usb.bmRequestType.type!=0` | -| STALLs / errors | `usb.urb_status!=0 && usb.urb_status!=-115` | +| Goal | Expression | +|--------------------------------------------------|-------------------------------------------------------------| +| One device / endpoint | `usb.device_address==26` / `usb.endpoint_address==0x81` | +| IN (to host) / OUT (from host) | `usb.endpoint_address.direction==1` / `==0` | +| Submit / Complete event | `usb.urb_type=='S'` / `=='C'` (char literal: single quotes) | +| Control / bulk / interrupt / iso | `usb.transfer_type==2` / `3` / `1` / `0` | +| Only transfers carrying data | `usb.data_len>0` | +| GET_DESCRIPTOR / SET_ADDRESS / SET_CONFIGURATION | `usb.setup.bRequest==6` / `5` / `9` | +| SET_INTERFACE / CLEAR_FEATURE (clear-halt) | `usb.setup.bRequest==11` / `1` | +| Descriptor type DEVICE/CONFIG/STRING/HID-report | `usb.bDescriptorType==1` / `2` / `3` / `0x22` | +| Class / vendor requests | `usb.bmRequestType.type!=0` | +| STALLs / errors | `usb.urb_status!=0 && usb.urb_status!=-115` | Combine with `&&` — e.g. one endpoint's data: `usb.endpoint_address==0x02 && usb.data_len>0`. diff --git a/.claude/skills/usbtest/SKILL.md b/.claude/skills/usbtest/SKILL.md new file mode 100644 index 000000000..32c2be913 --- /dev/null +++ b/.claude/skills/usbtest/SKILL.md @@ -0,0 +1,153 @@ +--- +name: usbtest +description: Use when running, debugging, or porting the Linux usbtest/testusb battery (examples/device/usbtest, cafe:4010) — device "did not bind", SET_CONFIGURATION fails, a case fails with errno 110/32/5/71, toggle-clear/halt/unlink/iso failures, iso packets dropped, or a new MCU/DCD needs the full 30/30 sign-off. Needs a Linux PC as the link's host driving TinyUSB in device role — it exercises the DCD, not the TinyUSB host stack. +--- + +# usbtest — porting & debugging the Linux kernel USB battery + +## Overview + +`examples/device/usbtest` is the device-side peer of the Linux kernel's `usbtest.ko`/`testusb` +(gadget-zero source/sink protocol): 30 cases over bulk, EP0, interrupt, and isochronous, including +halt, data-toggle, and unlink storms. It is the most adversarial exerciser a DCD gets — every port +so far surfaced at least one real driver bug. Host runner: `test/hil/usbtest.py`; HIL integration +runs it per board and reports `✅ 30/30` cells. + +**Core principle: the battery is a DCD test, not a firmware test.** When a case fails, suspect the +DCD path it exercises (table below), reproduce that one case, and root-cause on hardware before +changing anything (`superpowers:systematic-debugging`). One variable at a time; a fix is proven by +the failing case passing *and* the full battery still at 30/30 across reflash cycles. + +## Run + +```bash +# build (cmake); descriptor sizes auto-adapt per MCU via src/usb_descriptors.h + src/tusb_config.h +cd examples/device/usbtest && cmake -B build -DBOARD=<board> -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel && cmake --build build +# flash, wait ~3-5 s for enumeration to settle, then: +python3 test/hil/usbtest.py --serial <uid> --keep-binding # full battery for the advertised tier +python3 test/hil/usbtest.py --serial <uid> --keep-binding --tests 29 # one case +``` + +- **Always `--keep-binding`**: the cleanup unbind path has wedged host xHCIs (`usb_hcd_alloc_bandwidth`). +- CI (`hil_test.py`) additionally passes `--budget`, `--outer-timeout` and + `--recover-board`/`--recover-fw`: on a HUNG case the battery aborts, RESETS the DUT + through its roster probe (non-destructive, ~130 ms) and reflashes only if that does not + clear the wedge (see usb-kernel-recover). Manual runs without those flags leave a HUNG + device wedged and skip cleanup — expected; reset or reflash it yourself. +- Always settle a few seconds after flashing — enumeration can bounce once; testusb into the gap sees + the device drop mid-case. +- On a CI rig: stop the actions runner before touching hardware; restart after. Never run two + batteries concurrently (hil_test.py serializes them; concurrent batteries have hard-frozen a rig + via a fatal PCIe error on a VFIO-passed xHCI). + +## Porting ladder — new MCU/DCD to 30/30 + +1. **Tier 1 (bulk)**: set `USBTEST_TIER 1`, get enumeration + cases 0,9,10 (EP0) + 1–8,17–20,27,28 + solid. EP0 correctness first — everything else reports through it. +2. **Tier 2 (ctrl_out 14/21)**, **tier 3 (interrupt 25/26)**, **tier 4 (iso 15/16/22/23)** — raise + the tier only when the layer below is clean; run the *full* battery after each layer. +3. **Fit the endpoints**: tier 4 needs 6 endpoints + EP0. Small parts need per-MCU mps/epbuf + overrides in `src/usb_descriptors.h` (`USBTEST_INT/ISO_EP_MPS_FS`) and `src/tusb_config.h` + (`CFG_TUD_VENDOR_TX_EPSIZE`) — follow the existing CH32/LPC11 patterns. Parts that can't fit go + in `skip.txt`. +4. **Sign-off = reliability, not one pass**: 3–10 full flash→battery cycles. One 30/30 proves + nothing on a flaky bring-up; deterministic partial counts (e.g. exactly 1-in-8 lost) are a + signature, not noise — chase them. +5. Register the board in `test/hil/tinyusb.json` so the HIL suite runs it. + +## Case → DCD subsystem map + +| Failing case(s) | Exercises | First suspect | +|---|---|---| +| 9, 10 | EP0 control storms | EP0 state machine, ZLP/status stage, control starvation under load | +| 1–8, 17–20, 27, 28 | bulk source/sink, sg, perf | FIFO handling, multi-packet, ZLP tolerance | +| 11, 12, 24 | URB unlink mid-transfer | abort/close paths leaving state half-armed | +| 13 | set/clear halt | stall must kill the transfer; halt on armed IN must flush the TX FIFO | +| **29** | clear-halt on an **armed, un-halted** ep | **the classic**: `dcd_edpt_clear_stall` resets toggle but disarms the queued receive → NAKs forever, errno 110. Fix: reset toggle to DATA0 *and* re-arm/preserve the pending transfer. Found independently on rp2040, fsdev, ch32_usbhs, rusb2 | +| 14, 21 | vendor EP0 write/readback | multi-packet control-OUT chunking, DCP flow control | +| 25, 26 | interrupt src/sink | usually free once bulk works | +| 15, 16, 22, 23 | isochronous | see iso rules below | + +## Iso rules (most-violated contract) + +- **DATA0-only in BOTH directions** at FS — never run bulk-style toggle logic on an iso endpoint + (manual-toggle parts: skip the ISR toggle flip for iso IN *and* the toggle-mismatch drop for iso + OUT). Symptom of violating it: exactly every-other packet lost. +- **No handshake** — iso never NAKs/STALLs; parts with response fields use their "no response" + encoding (e.g. NYET on WCH). +- `dcd_edpt_iso_alloc`/`iso_activate` **must not be stubs returning false** — usbd fails the + interface open and the kernel logs "did not bind"/SET_CONFIG times out. If a DCD refuses iso + "because the hardware can't", **verify against the datasheet — the manual outranks the code + comment** (two "no iso support" claims in this tree were false, incl. a per-endpoint exception + the RM documents for one endpoint number only). +- A multi-packet iso IN submit is legal: the DCD streams it one packet per frame, refilling in the + ISR. Slow cores may need double-buffered iso to make the frame deadline. + +## Debug ladder (escalate in order) + +| errno | Meaning | +|-------|--------------------------------------------------------------| +| 110 | timeout — endpoint NAKing forever / device wedged | +| 32 | EPIPE — unexpected STALL | +| 5 | EIO — iso packet errors (check `dmesg`: "N errors out of M") | +| 71 | EPROTO — device answered wrong / too slow (after HC retries) | + +**Step 0 — read what the case actually does.** The kernel module is ground truth; +the table above is a summary. Do this before theorising, and always before deciding +whether a hung case is recoverable. Fetch the rig's exact version (`uname -r`): + +```bash +curl -sO "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/plain/drivers/usb/misc/usbtest.c?h=v6.12.96" +# case N lives under `case N:` in usbtest_do_ioctl(); tools/usb/testusb.c maps the flags: +# -c = param.iterations, -s = param.length, -g = param.sglen (NOT what they read like) +``` + +- **Real traffic and pass criteria.** Case 24 at `-c 256 -s 1024 -g 8` is 256 rounds + of 8 bulk-OUT URBs, unlinking `urbs[num-4]`/`urbs[num-2]` and requiring + `-ECONNRESET` on those two plus normal completion on the other 6 — not the + "256 URBs" the flags suggest. +- **Whether the wait is bounded** — decisive for recovery. `simple_io` uses + `wait_for_completion_timeout` (:481); the unlink paths use a bare + `wait_for_completion` (:1502, :1615). A device stalling there wedges the ioctl in + **D state permanently** — it holds the device lock, so nothing recovers it + (usb-kernel-recover, "The terminal case"). Knowing this first stops you burning + the rig on attempts that cannot work. +- **Which DCD path is implicated**, precisely rather than by category. + +1. `usbtest.py` per-case output + its captured `dmesg` (`TEST n` markers bracket each case). +2. **usbmon** (`usbmon` skill): URB-level ground truth. **It cannot show data toggles or NAKs** — + a toggle desync and a dead endpoint look identical (Submits without Completes); distinguish + device-side with GDB. +3. **On-device gdb/openocd**: read the EP control registers and DCD structs at the hang. +4. Heisenbugs (vanish under logging): RAM ring-buffer trace dumped over openocd; for silent lockups + JLink PC-sampling (`halt`+`regs` repeatedly — a pinned PC names the spin). +5. **Cross-check the reference manual** (`read-doc` skill) before changing any register-level code — + per CLAUDE.md, and because comments/assumptions in DCDs have been wrong about hardware caps. +6. Check the vendor's **silicon errata** early for timing/DMA hangs (an unimplemented erratum + workaround caused a case-10 hang on one port). + +## Traps that pass gcc/desk review but fail elsewhere + +- `TUD_OPT_HIGH_SPEED` is a **compile-time capability, not the live speed**: the FS config + descriptor (and OTHER_SPEED) must use FS-legal sizes (int ≤ 64, iso IN+OUT ≤ 1023 B/frame) even + on HS builds — use separate `_FS`/`_HS` descriptor macros. +- Unused `static inline` helpers: clang `-Wunused-function` and IAR `Pe177` error where gcc stays + quiet → `TU_ATTR_UNUSED`. +- A symbol referenced only inside naked asm is invisible to LTO and gets dropped in `-flto` make + builds → keep a `TU_ATTR_USED` C reference to it. +- Nested USB IRQs on cores with hardware context stacks (QingKe HWSTK): plain + `__attribute__((interrupt))` corrupts the return — use naked handlers relying on the HW stack. +- Dedicated USB RAM budgets (PMA/USB-RAM) differ per part *and* per build system section placement: + check the link map, not just that it builds. + +## Red flags — stop and re-examine + +- "One pass = done" → run reflash cycles. +- "The DCD comment says the hardware can't" → open the datasheet. +- "usbmon shows no toggle problem" → usbmon can't see toggles. +- "It works on gcc" → clang/IAR/LTO/make still pending. +- "Fixed iso IN" → apply the same exemption to iso OUT (toggle logic is symmetric). +- A clean single-board run does not validate concurrent/fleet behavior — batteries serialize. +- Reasoning about a case from its name or table row → open `usbtest.c` (step 0). The + flags don't mean what they look like, and recoverability is a property of that + case's wait, not of the rig. diff --git a/.claude/workflows/driver-review.js b/.claude/workflows/driver-review.js index 3638aa179..255b8ac74 100644 --- a/.claude/workflows/driver-review.js +++ b/.claude/workflows/driver-review.js @@ -16,7 +16,7 @@ if (!args || !Array.isArray(args.dirs) || args.dirs.length === 0) { const DIMS = args.question ? [args.question] : (args.dimensions || [ 'correctness: transfer state machines, endpoint bookkeeping, completion and error paths', 'ISR safety: work deferred to task context, shared-state races, register access ordering', - 'register use vs datasheet and MCU errata: cross-check the reference manual AND errata sheets in $HOME/Documents/calibre-library; a missing erratum workaround is a finding', + 'register use vs datasheet and MCU errata: cross-check the reference manual AND errata sheets via the read-doc skill (python3 .claude/skills/read-doc/search.py <keywords>); a missing erratum workaround is a finding', 'style: repo conventions (TU_ASSERT, no dynamic allocation, include order, naming)', ]) if (!DIMS.length) { diff --git a/.claude/workflows/hil-validate.js b/.claude/workflows/hil-validate.js index aa0556abc..136f9075e 100644 --- a/.claude/workflows/hil-validate.js +++ b/.claude/workflows/hil-validate.js @@ -26,8 +26,8 @@ const runBoard = (b) => agent( ? 'THE USER HAS EXPLICITLY AUTHORIZED FORCING: run hil_test.py with HIL_NO_BOARD_LOCK=1 in the environment (bypasses the board lock check; do NOT release or kill the existing holder). ' : 'If the run fails because the board lock is held (a dev session or concurrent CI job), report pass=false and set detail to start EXACTLY with "board locked:" followed by the holder JSON verbatim — never force the lock. ') + 'Reserve the phrase "board locked" strictly for lock contention; describe a frozen or non-enumerating board as "unresponsive" instead. ' + - `Firmware is in examples/cmake-build-${b}. Use the config for this host (hostname first), single-board flag -b ${b}, Bash timeout >= 20 min, never cancel early. ` + - 'On non-lock failures retry once with -v -r 1 (one verbose attempt for diagnosis — the first run already did the flake-retries). wedged=true if the board/fixture is unresponsive after the run (capture dmesg | tail -50 into detail).', + `Firmware is in examples/cmake-build-${b}. Use the config for this host (hostname first), single-board flag -b ${b}. Run hil_test.py as a BACKGROUND Bash task and wait for it (a stuck fleet runs to its pool guard, 60 min by default — beyond any foreground timeout); never cancel it early. ` + + 'On non-lock failures retry once with -v -r 1 (one verbose attempt for diagnosis; note a usbtest battery that produced per-case verdicts is NOT auto-retried, so its result already stands). wedged=true if the board/fixture is unresponsive after the run (capture dmesg | tail -50 into detail).', { label: `hil:${b}`, phase: 'HIL', agentType: 'hil-operator', schema: HIL }, ) @@ -52,7 +52,7 @@ if (!args.force) { } const wedged = results.filter(r => r.wedged).map(r => r.board) -if (wedged.length) log(`WEDGED boards needing usb-recover: ${wedged.join(', ')}`) +if (wedged.length) log(`WEDGED boards needing usb-kernel-recover: ${wedged.join(', ')}`) // Workers cannot prompt the user — surface still-locked boards for the main // session to ask: force (re-invoke with force: true), wait, or accept. const locked = args.force ? [] : results.filter(r => !r.pass && r.detail.startsWith('board locked')).map(r => r.board) diff --git a/.claude/workflows/pr-babysit.js b/.claude/workflows/pr-babysit.js index 8bb414114..406213a5f 100644 --- a/.claude/workflows/pr-babysit.js +++ b/.claude/workflows/pr-babysit.js @@ -109,7 +109,7 @@ const history = [] const repliedIds = new Set() // issue comments can't be thread-resolved, so they re-harvest every cycle — never reply twice for (let cycle = 1; cycle <= maxCycles; cycle++) { const t = await agent( - `Triage PR #${args.pr}. If checks are still running, wait for them first (gh pr checks ${args.pr} --watch, Bash timeout >= 30 min). ` + + `Triage PR #${args.pr}. If checks are still running, wait for them first (gh pr checks ${args.pr} --watch as a BACKGROUND Bash task; the foreground timeout is capped at 10 min). ` + 'Then follow your triage procedure: classify CI failures, re-run infra ones, harvest and adversarially validate bot review findings, draft replies for invalid/stale ones.', { label: `triage#${cycle}`, phase: 'Triage', agentType: 'pr-monitor', schema: TRIAGE }, ) diff --git a/.github/workflows/ci_set_matrix.py b/.github/scripts/ci_set_matrix.py index dc0d3871f..50ada5964 100755 --- a/.github/workflows/ci_set_matrix.py +++ b/.github/scripts/ci_set_matrix.py @@ -16,6 +16,7 @@ toolchain_list = [ # family: [supported toolchain] family_list = { + "apm32f0xx": ["arm-gcc"], "at32f402_405": ["arm-gcc"], "at32f403a_407": ["arm-gcc"], "at32f413": ["arm-gcc"], diff --git a/test/hil/hil_ci_set_matrix.py b/.github/scripts/hil_ci_set_matrix.py index 13f7f1882..65f50788e 100644 --- a/test/hil/hil_ci_set_matrix.py +++ b/.github/scripts/hil_ci_set_matrix.py @@ -7,9 +7,11 @@ def _resolve_config_path(config_file): if os.path.exists(config_file): return config_file - script_relative = os.path.join(os.path.dirname(__file__), config_file) - if os.path.exists(script_relative): - return script_relative + # bare roster names resolve against the repo's test/hil (this script lives in + # .github/scripts); build.yml passes explicit paths, this is for hand-runs + repo_relative = os.path.join(os.path.dirname(__file__), '..', '..', 'test', 'hil', config_file) + if os.path.exists(repo_relative): + return repo_relative raise FileNotFoundError(f'Config file not found: {config_file}') @@ -17,8 +19,14 @@ def _resolve_config_path(config_file): def main(): parser = argparse.ArgumentParser() parser.add_argument('config_files', nargs='+', help='Configuration JSON file(s)') + parser.add_argument('--select', help='hil_select.py JSON; scopes boards when full=false') args = parser.parse_args() + selected = None + sel = json.loads(args.select) if args.select else None + if sel and not sel.get('full'): + selected = set(sel.get('boards', {})) + # Toolchain buckets must match the toolchains instantiated by the hil-build # job in .github/workflows/build.yml. Keep all keys present (even if empty) # so `fromJSON(hil_json)[toolchain]` always resolves to a list. @@ -40,6 +48,8 @@ def main(): config = json.load(f) for board in config['boards']: + if selected is not None and board['name'] not in selected: + continue name = board['name'] flasher = board['flasher'] # esptool boards must build under esp-idf; others default to arm-gcc @@ -49,6 +59,13 @@ def main(): toolchain = 'esp-idf' else: toolchain = board.get('toolchain', 'arm-gcc') + if toolchain not in matrix: + # a board in no bucket would never be built, and the bare KeyError + # below would only say so as a traceback from the set-matrix job + raise SystemExit( + f'{name}: toolchain {toolchain!r} is not a build bucket ' + f'({", ".join(matrix)}); add it here and to the hil-build / ' + f'hil-build-esp jobs in .github/workflows/build.yml') build_board = f'-b {name}' if 'build' in board and 'args' in board['build']: diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index c8c597e50..8f6014f48 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -41,27 +41,100 @@ jobs: - '.github/actions/**' - '.github/workflows/build.yml' - '.github/workflows/build_util.yml' - - '.github/workflows/ci_set_matrix.py' + - '.github/scripts/**' set-matrix: runs-on: ubuntu-latest outputs: json: ${{ steps.set-matrix-json.outputs.matrix }} hil_json: ${{ steps.set-matrix-json.outputs.hil_matrix }} + # one pair per rig job: hil-tinyusb (tinyusb.json minus esptool boards), + # hil-tinyusb-esp (esptool boards only), hil-tinyusb (hfp.json) + hil_args_tinyusb: ${{ steps.hil-select.outputs.args_tinyusb }} + hil_run_tinyusb: ${{ steps.hil-select.outputs.run_tinyusb }} + hil_args_tinyusb_esp: ${{ steps.hil-select.outputs.args_tinyusb_esp }} + hil_run_tinyusb_esp: ${{ steps.hil-select.outputs.run_tinyusb_esp }} + hil_args_hfp: ${{ steps.hil-select.outputs.args_hfp }} + hil_run_hfp: ${{ steps.hil-select.outputs.run_hfp }} steps: - name: Checkout TinyUSB uses: actions/checkout@v6 + with: + fetch-depth: 0 + + - name: HIL selection (PR only) + id: hil-select + if: github.event_name == 'pull_request' + env: + BASE_REF: ${{ github.base_ref }} + run: | + # Best-effort by design: set-matrix gates cmake, hil-build and every rig job, + # so a missing origin/<base>, a shallow-clone hiccup or a selector traceback + # must fall back to the FULL matrix (no --select, run=true, no args) instead + # of failing the job. Same fail-open shape as pr_comment.yml's `|| true`. + # + # The selector's own unit suite gates it (stdlib-only, seconds): a selector + # whose tests fail can still exit 0 with valid-but-WRONG JSON -- fail-open alone + # never catches that class, and the pre-commit hil-test hook is a separate, + # advisory workflow that nothing here can `needs:`. Test-failing selector => + # full matrix, same as a crashing one. + SELECT_JSON='' + if ! python3 test/hil/test/test_hil_select.py; then + echo "::warning::hil_select unit suite failed - falling back to the full HIL matrix" + elif ! SELECT_JSON=$(python3 test/hil/helper/hil_select.py --base "origin/$BASE_REF" test/hil/tinyusb.json test/hil/hfp.json); then + echo "::warning::hil_select failed - falling back to the full HIL matrix" + SELECT_JSON='' + fi + + # One args/run pair per rig job, split by flasher: a job whose own subset is + # empty skips explicitly instead of running a board filter that matches zero + # boards ("No tests were run." exits 0 and would read as a green HIL run). + OUT='' + if [ -n "$SELECT_JSON" ]; then + OUT=$(SELECT_JSON="$SELECT_JSON" python3 -c ' + import json, os + s = json.loads(os.environ["SELECT_JSON"]) + tin = s.get("args_flasher", {}).get("tinyusb.json", {}) + legs = (("tinyusb", " ".join(a for f, a in sorted(tin.items()) if f != "esptool" and a)), + ("tinyusb_esp", tin.get("esptool", "")), + ("hfp", s.get("args", {}).get("hfp.json", ""))) + for key, a in legs: + print("args_" + key + "=" + a) + print("run_" + key + "=" + ("true" if (s.get("full") or a) else "false")) + ') || OUT='' + if [ -z "$OUT" ]; then + echo "::warning::hil_select output unusable - falling back to the full HIL matrix" + SELECT_JSON='' + fi + fi + if [ -z "$OUT" ]; then + OUT=$(for k in tinyusb tinyusb_esp hfp; do printf 'args_%s=\nrun_%s=true\n' "$k" "$k"; done) + fi + echo "$OUT" + { echo "select=$SELECT_JSON"; echo "$OUT"; } >> $GITHUB_OUTPUT - name: Generate matrix json id: set-matrix-json + env: + SELECT: ${{ steps.hil-select.outputs.select }} run: | # build matrix - MATRIX_JSON=$(python .github/workflows/ci_set_matrix.py) + MATRIX_JSON=$(python .github/scripts/ci_set_matrix.py) echo "matrix=$MATRIX_JSON" echo "matrix=$MATRIX_JSON" >> $GITHUB_OUTPUT - # HIL matrix (merged from tinyusb + hifiphile configs) - HIL_MATRIX_JSON=$(python test/hil/hil_ci_set_matrix.py test/hil/tinyusb.json test/hil/hfp.json) + # HIL matrix (merged from tinyusb + hifiphile configs), scoped on PRs. + # Scoping is best-effort too: fall back to the unscoped (full) matrix. + HIL_MATRIX_JSON='' + if [ -n "$SELECT" ]; then + HIL_MATRIX_JSON=$(python .github/scripts/hil_ci_set_matrix.py --select "$SELECT" test/hil/tinyusb.json test/hil/hfp.json) || HIL_MATRIX_JSON='' + if [ -z "$HIL_MATRIX_JSON" ]; then + echo "::warning::scoped HIL matrix failed - falling back to the full HIL matrix" + fi + fi + if [ -z "$HIL_MATRIX_JSON" ]; then + HIL_MATRIX_JSON=$(python .github/scripts/hil_ci_set_matrix.py test/hil/tinyusb.json test/hil/hfp.json) + fi echo "hil_matrix=$HIL_MATRIX_JSON" echo "hil_matrix=$HIL_MATRIX_JSON" >> $GITHUB_OUTPUT @@ -271,23 +344,55 @@ jobs: strategy: fail-fast: false matrix: + # These names are the bucket keys of .github/scripts/hil_ci_set_matrix.py: every + # non-esptool roster board must land in one of them (esptool boards go to + # 'esp-idf', built by hil-build-esp below). hil_ci_set_matrix.py rejects a + # board whose "toolchain" is not a bucket, so a new bucket must be added in + # both places. + # + # INVARIANT the PR-scoped skip cascade rests on: hil_run_tinyusb / hil_run_hfp + # true => hil-build has at least one non-empty leg. It holds because those + # flags count only non-esptool boards and every such board builds here. It + # would break if an esptool board were added to hfp.json, because + # hil_args_hfp is NOT flasher-split: the hfp leg of hil-tinyusb would want to + # run while hil-build (and therefore that leg) skipped. toolchain: - 'arm-gcc' - 'riscv-gcc' - - 'esp-idf' with: build-system: 'cmake' toolchain: ${{ matrix.toolchain }} build-args: ${{ toJSON(fromJSON(needs.set-matrix.outputs.hil_json)[matrix.toolchain]) }} upload-artifacts: true + # esp-idf builds are by far the slowest; keep them out of hil-build so the main + # hil-tinyusb run starts as soon as the fast toolchains finish (esp boards get + # their own hil-tinyusb-esp run gated only on this job) + hil-build-esp: + needs: [ check-paths, set-matrix ] + if: needs.check-paths.outputs.code_changed == 'true' && github.repository_owner == 'hathach' + uses: ./.github/workflows/build_util.yml + with: + build-system: 'cmake' + toolchain: 'esp-idf' + build-args: ${{ toJSON(fromJSON(needs.set-matrix.outputs.hil_json)['esp-idf']) }} + upload-artifacts: true + # --------------------------------------- # Hardware in the loop (HIL) # self-hosted on local VM, for attached hardware checkout HIL_JSON # --------------------------------------- hil-tinyusb: - needs: hil-build + needs: [ hil-build, set-matrix ] name: hil-tinyusb (${{ matrix.display }}) + # Above hil_test.py's pool guard (HIL_POOL_TIMEOUT, 60 min) so the guard fires first + # and still gets to write its report. The 30 min on top is what the job pays OUTSIDE + # the guard clock: workspace cleanup, checkout, the multi-board artifact merge and + # the D-state note before it; kill_worker_children, shutdown_pool's 30 s grace, the + # report write and the upload after it. On a multi-stray convoy that tail alone is + # minutes, and a ceiling below guard+tail cancels the job before hil_report.md exists + # -- the inversion this branch removes. Both legs share the script and the guard. + timeout-minutes: 90 strategy: fail-fast: false matrix: @@ -295,26 +400,39 @@ jobs: - display: tinyusb.json runner: [ self-hosted, X64, hathach, hardware-in-the-loop ] hil_json: test/hil/tinyusb.json + # esptool-flashed (espressif) boards run in hil-tinyusb-esp, + # gated on the slow esp-idf build + test_args: '--exclude-flasher esptool' - display: hfp.json runner: [ self-hosted, Linux, X64, hifiphile ] hil_json: test/hil/hfp.json + test_args: '' runs-on: ${{ matrix.runner }} env: + # HIL_POOL_TIMEOUT deliberately unset: hil_test.py's 60 min default is below every + # ceiling here, so ceiling > guard holds by construction. Pin it to SHORTEN a run + # only -- pinning it above a ceiling re-inverts the two. HIL_JSON: ${{ matrix.hil_json }} steps: - - name: Set HIL report dir (sibling of workspace; persists across run attempts) - run: echo "HIL_REPORT_DIR=$(dirname "$GITHUB_WORKSPACE")/hil-report" >> "$GITHUB_ENV" + - name: Set HIL report dir (per run+job; persists across run attempts) + run: | + # one report dir per (run id, job): re-run attempts find their own report/spec, + # and interleaved runs of other PRs/jobs on the same runner cannot clobber them + BASE="$(dirname "$GITHUB_WORKSPACE")/hil-report" + # prune per-run dirs older than 2 weeks + find "$BASE" -mindepth 1 -maxdepth 1 -type d -mtime +14 -exec rm -rf {} + 2>/dev/null || true + echo "HIL_REPORT_DIR=$BASE/${GITHUB_RUN_ID}-$(basename "${{ matrix.display }}" .json)" >> "$GITHUB_ENV" - - name: Get Skip Boards from previous run + - name: Get re-run spec from previous attempt if: github.run_attempt != '1' run: | - if [ -f "${{ env.HIL_JSON }}.skip" ]; then - SKIP_BOARDS=$(cat "${{ env.HIL_JSON }}.skip") - else - SKIP_BOARDS="" - fi - echo "SKIP_BOARDS=$SKIP_BOARDS" - echo "SKIP_BOARDS=$SKIP_BOARDS" >> $GITHUB_ENV + # the report dir is keyed by run id, so a spec here can only have been + # written by an earlier attempt of THIS run + SPEC="$HIL_REPORT_DIR/$(basename "${{ env.HIL_JSON }}").failed" + RERUN_ARGS="" + [ -f "$SPEC" ] && RERUN_ARGS=$(cat "$SPEC") + echo "RERUN_ARGS=$RERUN_ARGS" + echo "RERUN_ARGS=$RERUN_ARGS" >> $GITHUB_ENV - name: Clean workspace run: | @@ -333,15 +451,33 @@ jobs: merge-multiple: true - name: Test on actual hardware + # Single attempt per test (--retry 1), no in-run second pass: a broken fixture + # fails fast instead of holding the runner (and other PRs' HIL jobs) for hours. + env: + # tinyusb.json minus the esptool boards (they run in hil-tinyusb-esp): each + # job gates on its own flasher subset, never on a rig-wide flag + SEL_ARGS_TINYUSB: ${{ needs.set-matrix.outputs.hil_args_tinyusb }} + SEL_RUN_TINYUSB: ${{ needs.set-matrix.outputs.hil_run_tinyusb }} + SEL_ARGS_HFP: ${{ needs.set-matrix.outputs.hil_args_hfp }} + SEL_RUN_HFP: ${{ needs.set-matrix.outputs.hil_run_hfp }} run: | - python3 test/hil/hil_test.py ${{ env.HIL_JSON }} $SKIP_BOARDS || \ - (if [ -f "${{ env.HIL_JSON }}.skip" ]; then - SKIP_BOARDS=$(cat "${{ env.HIL_JSON }}.skip") - echo "Re-running with SKIP_BOARDS=$SKIP_BOARDS" - python3 test/hil/hil_test.py ${{ env.HIL_JSON }} $SKIP_BOARDS - else - exit 1 - fi) + case "$HIL_JSON" in + *tinyusb.json) SEL_ARGS="$SEL_ARGS_TINYUSB"; SEL_RUN="$SEL_RUN_TINYUSB" ;; + *hfp.json) SEL_ARGS="$SEL_ARGS_HFP"; SEL_RUN="$SEL_RUN_HFP" ;; + esac + if [ "$SEL_RUN" = "false" ]; then + echo "HIL skipped by PR selection (no affected boards on this rig)" + # leave a marker so the combined PR comment says so instead of dropping the + # section (and leaving a stale table from an earlier push in its place) + mkdir -p "$HIL_REPORT_DIR" + echo "_Skipped by PR selection: no affected boards on this rig._" > "$HIL_REPORT_DIR/hil_report.md" + exit 0 + fi + # a re-run spec is already a subset of the selection (only the boards/tests + # that failed); -b/-bt accumulate, so keeping SEL_ARGS here would re-run the + # entire original selection instead of just what failed + if [ -n "$RERUN_ARGS" ]; then SEL_ARGS=''; fi + python3 test/hil/hil_test.py --retry 1 ${{ matrix.test_args }} $SEL_ARGS ${{ env.HIL_JSON }} $RERUN_ARGS - name: Upload HIL report if: always() && github.event_name == 'pull_request' @@ -353,6 +489,92 @@ jobs: overwrite: true # --------------------------------------- + # Hardware in the loop (HIL) - espressif boards only + # Same rig as hil-tinyusb (tinyusb.json) but gated only on the slow esp-idf build, + # so the main run does not wait for it. Per-board flocks arbitrate the shared rig; + # the runner has a single job slot, so the two HIL jobs never overlap - adding a + # second slot would double the per-controller flash/usbtest budgets. + # --------------------------------------- + hil-tinyusb-esp: + needs: [ hil-build-esp, set-matrix ] + name: hil-tinyusb (tinyusb-esp.json) + runs-on: [ self-hosted, X64, hathach, hardware-in-the-loop ] + # above hil_test.py's pool guard (60 min) with room for the pre-pool checkout + # and the post-guard sweep + report upload, so its own guard still writes a report; + # only a job wedged past that (unkillable D-state worker) hits this ceiling, which + # must exist because the runner has one job slot and holds every queued job hostage + timeout-minutes: 90 + env: + HIL_JSON: test/hil/tinyusb.json + TEST_ARGS: '--flasher esptool' + steps: + - name: Set HIL report dir (per run+job; persists across run attempts) + run: | + # one report dir per (run id, job): re-run attempts find their own report/spec, + # and interleaved runs of other PRs/jobs on the same runner cannot clobber them + BASE="$(dirname "$GITHUB_WORKSPACE")/hil-report" + # prune per-run dirs older than 2 weeks + find "$BASE" -mindepth 1 -maxdepth 1 -type d -mtime +14 -exec rm -rf {} + 2>/dev/null || true + echo "HIL_REPORT_DIR=$BASE/${GITHUB_RUN_ID}-tinyusb-esp" >> "$GITHUB_ENV" + + - name: Get re-run spec from previous attempt + if: github.run_attempt != '1' + run: | + # the report dir is keyed by run id, so a spec here can only have been + # written by an earlier attempt of THIS run + SPEC="$HIL_REPORT_DIR/$(basename "${{ env.HIL_JSON }}").failed" + RERUN_ARGS="" + [ -f "$SPEC" ] && RERUN_ARGS=$(cat "$SPEC") + echo "RERUN_ARGS=$RERUN_ARGS" + echo "RERUN_ARGS=$RERUN_ARGS" >> $GITHUB_ENV + + - name: Clean workspace + run: | + echo "Cleaning up for the first run" + rm -rf "${{ github.workspace }}" + mkdir -p "${{ github.workspace }}" + + - name: Checkout TinyUSB + uses: actions/checkout@v6 + + - name: Download Artifacts + uses: actions/download-artifact@v5 + with: + pattern: binaries-esp-idf-* + path: cmake-build + merge-multiple: true + + - name: Test on actual hardware + env: + # esptool subset of tinyusb.json: this job must gate on its own boards, not + # on the rig-wide flag (which would run a filter matching zero boards) + SEL_ARGS: ${{ needs.set-matrix.outputs.hil_args_tinyusb_esp }} + SEL_RUN: ${{ needs.set-matrix.outputs.hil_run_tinyusb_esp }} + run: | + if [ "$SEL_RUN" = "false" ]; then + echo "HIL skipped by PR selection (no affected esptool boards)" + # leave a marker so the combined PR comment says so instead of dropping the + # section (and leaving a stale table from an earlier push in its place) + mkdir -p "$HIL_REPORT_DIR" + echo "_Skipped by PR selection: no affected esptool boards._" > "$HIL_REPORT_DIR/hil_report.md" + exit 0 + fi + # a re-run spec is already a subset of the selection (only the boards/tests + # that failed); -b/-bt accumulate, so keeping SEL_ARGS here would re-run the + # entire original selection instead of just what failed + if [ -n "$RERUN_ARGS" ]; then SEL_ARGS=''; fi + python3 test/hil/hil_test.py --retry 1 $TEST_ARGS $SEL_ARGS ${{ env.HIL_JSON }} $RERUN_ARGS + + - name: Upload HIL report + if: always() && github.event_name == 'pull_request' + uses: actions/upload-artifact@v7 + with: + name: hil-report-tinyusb-esp.json + path: ${{ env.HIL_REPORT_DIR }}/hil_report.md + if-no-files-found: ignore + overwrite: true + + # --------------------------------------- # Hardware in the loop (HIL) # self-hosted by HFP, build with IAR toolchain, for attached hardware checkout test/hil/hfp.json # Since IAR Token secret is not passed to forked PR, only build non-forked PR @@ -364,7 +586,13 @@ jobs: github.repository_owner == 'hathach' && !(github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true) runs-on: [ self-hosted, Linux, X64, hifiphile ] - timeout-minutes: 30 + # Unlike the hil-tinyusb jobs, this one BUILDS with IAR in the same job before running + # hil_test.py -- hfp.json's 3 boards, 4 variant entries, "up to 30 minutes" (see the + # comment above the selection step). The ceiling has to cover build + the 60 min pool + # guard + overhead, or GitHub cancels before the guard can write its report -- the + # inversion this branch removes. 30 + 60 = 90; the remaining 30 is the full-history + # checkout, get_deps, the post-guard sweep and the report upload. + timeout-minutes: 120 env: IAR_LMS_BEARER_TOKEN: ${{ secrets.IAR_LMS_BEARER_TOKEN }} PYTHONUNBUFFERED: '1' @@ -381,23 +609,87 @@ jobs: - name: Checkout TinyUSB uses: actions/checkout@v6 + with: + # full history: the "HIL selection" step below needs + # merge-base(HEAD, origin/<base_ref>) for PR-scoped selection + fetch-depth: 0 + + # Computed BEFORE the build: the IAR build is four boards and up to 30 minutes on + # a runner that hil-tinyusb (hfp.json) also needs, so an unaffected PR must release + # it immediately instead of building everything and then skipping. The selection + # also narrows what gets built. + # This job has no needs: on set-matrix (it must run even if that unrelated job + # fails), so it computes its own selection instead of reading set-matrix's outputs. + - name: HIL selection (PR only) + if: github.event_name == 'pull_request' + env: + BASE_REF: ${{ github.base_ref }} + run: | + # Best-effort: this job is deliberately decoupled from set-matrix so unrelated + # failures cannot kill hfp coverage - a selector failure here must likewise + # fall back to the full hfp matrix (no hil_select.json, no SEL_* vars), never + # fail the job. + if ! python3 test/hil/test/test_hil_select.py; then + echo "::warning::hil_select unit suite failed - running the full hfp matrix" + rm -f hil_select.json + exit 0 + fi + if ! python3 test/hil/helper/hil_select.py --base "origin/$BASE_REF" test/hil/hfp.json > hil_select.json; then + echo "::warning::hil_select failed - running the full hfp matrix" + rm -f hil_select.json + exit 0 + fi + # hil_select.json is passed to hil_ci_set_matrix.py --select below to scope the + # build; it already honours full=true by ignoring the board list. + # The hil_test.py args go to a file, never to $GITHUB_ENV: they are derived + # from roster board names, which a PR can edit. Only SEL_RUN (a literal + # true/false computed here, needed by the step-level `if:`) goes to the env. + if ! SEL_RUN=$(python3 -c ' + import json + s = json.load(open("hil_select.json")) + a = s["args"]["hfp.json"] + open("hil_sel_args.txt", "w").write(a) + print("true" if (s["full"] or a) else "false") + '); then + echo "::warning::hil_select output unusable - running the full hfp matrix" + rm -f hil_select.json hil_sel_args.txt + exit 0 + fi + echo "SEL_RUN=$SEL_RUN" + echo "SEL_RUN=$SEL_RUN" >> $GITHUB_ENV - name: Get build boards + if: env.SEL_RUN != 'false' run: | - MATRIX_JSON=$(python test/hil/hil_ci_set_matrix.py test/hil/hfp.json) - BUILD_ARGS=$(echo $MATRIX_JSON | jq -r '.["arm-gcc"] | join(" ")') + if [ -f hil_select.json ]; then + MATRIX_JSON=$(python .github/scripts/hil_ci_set_matrix.py --select "$(cat hil_select.json)" test/hil/hfp.json) + else + MATRIX_JSON=$(python .github/scripts/hil_ci_set_matrix.py test/hil/hfp.json) + fi + # Each variant carries its own --build-name/--cflag, which are global to a + # single build.py invocation — so keep one matrix entry per line and build + # them one at a time (joining would leak a variant's flags onto every board). + echo "$MATRIX_JSON" | jq -r '.["arm-gcc"][]' > hil_build_entries.txt + cat hil_build_entries.txt + BUILD_ARGS=$(echo "$MATRIX_JSON" | jq -r '.["arm-gcc"] | join(" ")') echo "BUILD_ARGS=$BUILD_ARGS" echo "BUILD_ARGS=$BUILD_ARGS" >> $GITHUB_ENV - name: Get Dependencies + if: env.SEL_RUN != 'false' run: python3 tools/get_deps.py $BUILD_ARGS - name: Build + if: env.SEL_RUN != 'false' + # Bounded SEPARATELY from the job. This is the only HIL job that builds inline + # (hil-tinyusb downloads artifacts), and the job ceiling went 30 -> 120 to give the + # HIL step room -- which would hand a stalled IAR build the whole two hours on the + # shared self-hosted runner, never reaching hil_test.py or the report upload. That + # is the stranded-runner-with-no-report failure this branch exists to prevent. + # Typical full build here is a few minutes; 30 leaves generous headroom. + timeout-minutes: 30 run: | - # Each variant carries its own --build-name/--cflag, which are global to a - # single build.py invocation — so build one matrix entry at a time rather - # than joining them (joining would leak a variant's flags onto every board). - readarray -t ENTRIES < <(python test/hil/hil_ci_set_matrix.py test/hil/hfp.json | jq -r '.["arm-gcc"][]') + readarray -t ENTRIES < hil_build_entries.txt for entry in "${ENTRIES[@]}"; do echo "+ tools/build.py --toolchain iar $entry" python3 tools/build.py --toolchain iar $entry @@ -405,7 +697,21 @@ jobs: - name: Test on actual hardware (hardware in the loop) run: | - python3 test/hil/hil_test.py hfp.json + if [ "$SEL_RUN" = "false" ]; then + echo "HIL skipped by PR selection (no affected boards on this rig)" + # leave a marker so the combined PR comment says so instead of dropping + # the section (and leaving a stale table from an earlier push) + echo "_Skipped by PR selection: no affected boards on this rig._" > hil_report.md + exit 0 + fi + # empty/absent on a non-PR event or a selector fallback -> full hfp matrix + SEL_ARGS=$(cat hil_sel_args.txt 2>/dev/null || true) + # --retry 1, like the other two HIL legs. The pool guard is a FLAT 3600s and + # does NOT scale with max_retry, so argparse's default of 3 would multiply the + # serialized usbtest tail (hfp.json runs four batteries) by three against an + # unchanged guard -- on a runner with a single job slot that queues every other + # job behind it. + python3 test/hil/hil_test.py --retry 1 $SEL_ARGS hfp.json - name: Upload HIL report if: always() && github.event_name == 'pull_request' diff --git a/.github/workflows/build_util.yml b/.github/workflows/build_util.yml index 2532caebe..02f16488a 100644 --- a/.github/workflows/build_util.yml +++ b/.github/workflows/build_util.yml @@ -39,6 +39,21 @@ on: jobs: family: + # PR-scoped HIL selection can produce an empty build-args list for a toolchain + # (e.g. a dwc2-only change with no riscv boards affected); an empty matrix + # vector fails the job outright ("Matrix vector 'arg' does not contain any + # values"), so skip cleanly instead. + # + # Why that is safe for callers: GitHub SKIPS the dependents of a skipped + # `needs:` job, so this only works because the caller (hil-build) is itself a + # *matrix* job - a matrix with one skipped leg and one successful leg + # aggregates to success, and its dependents run. + # + # NOT covered: if every leg is empty the whole caller job skips, and so does + # everything that needs it. That is fine only because an all-empty selection + # means no board was selected for those rigs, so the rig jobs would have had + # nothing to run anyway (see the invariant on hil-build in build.yml). + if: inputs.build-args != '[]' runs-on: ${{ inputs.os }} strategy: fail-fast: false @@ -99,7 +114,7 @@ jobs: if: inputs.upload-artifacts == true && inputs.code-changed == true uses: actions/upload-artifact@v7 with: - name: binaries-${{ matrix.arg }} + name: binaries-${{ inputs.toolchain }}-${{ matrix.arg }} path: | cmake-build/cmake-build-*/*/*/*.elf cmake-build/cmake-build-*/*/*/*.bin diff --git a/.github/workflows/pr_comment.yml b/.github/workflows/pr_comment.yml index 4d50817b4..868e56405 100644 --- a/.github/workflows/pr_comment.yml +++ b/.github/workflows/pr_comment.yml @@ -101,7 +101,16 @@ jobs: shopt -s nullglob dirs=(hil-reports/hil-report-*) if [ ${#dirs[@]} -eq 0 ]; then + # No rig produced a report: PR selection matched no board anywhere, or the + # HIL jobs did not run at all. Post it rather than exiting, so a table from + # an earlier push is replaced instead of being left to look current. echo "No HIL reports found" + { + echo "## Hardware-in-the-loop (HIL) Test Report" + echo + echo "_No HIL run for this push (no affected boards, or hardware testing did not run)._" + } > hil_combined.md + echo "found=true" >> "$GITHUB_OUTPUT" exit 0 fi { diff --git a/.github/workflows/pre-commit.yml b/.github/workflows/pre-commit.yml index b9bfaf9b6..09f912bd3 100644 --- a/.github/workflows/pre-commit.yml +++ b/.github/workflows/pre-commit.yml @@ -24,10 +24,13 @@ jobs: - name: Get Dependencies run: | - gem install ceedling + # pinned: 1.1.0 breaks mock preprocessing on this project ("Failed to read + # _build/test/preprocess/.../raw/*.h for comment stripping"); revisit on next release + gem install ceedling -v 1.0.1 #cd test/unit-test #ceedling test:all + # runs --all-files, so the hil-test hook fires here regardless of its `files:` scope - name: Run pre-commit uses: pre-commit/[email protected] diff --git a/.gitignore b/.gitignore index 61358d118..8773322e4 100644 --- a/.gitignore +++ b/.gitignore @@ -60,6 +60,7 @@ BrowseInfo README_processed.rst docs/examples/ .worktrees +.claude/worktrees/ cmake-metrics/ # Directories fetched by tools/get_deps.py - not to be committed lib/CMSIS_5/ @@ -77,9 +78,11 @@ hw/mcu/artery/ hw/mcu/broadcom/ hw/mcu/bridgetek/ft9xx/ft90x-sdk/ hw/mcu/gd/ +hw/mcu/geehy/ hw/mcu/hpmicro/ hw/mcu/infineon/ hw/mcu/microchip/ +hw/mcu/puya/ hw/mcu/mindmotion/ hw/mcu/nordic/nrfx/ hw/mcu/nuvoton/ diff --git a/.idea/cmake.xml b/.idea/cmake.xml index 14dbdfe66..23e8af7ea 100644 --- a/.idea/cmake.xml +++ b/.idea/cmake.xml @@ -5,9 +5,9 @@ <configuration PROFILE_NAME="raspberrypi_zero" ENABLED="false" CONFIG_NAME="Debug" GENERATION_OPTIONS="-DBOARD=raspberrypi_zero -DLOG=1" /> <configuration PROFILE_NAME="raspberrypi_zero2" ENABLED="false" CONFIG_NAME="Debug" GENERATION_OPTIONS="-DBOARD=raspberrypi_zero2 -DLOG=1" /> <configuration PROFILE_NAME="raspberrypi_cm4" ENABLED="false" CONFIG_NAME="Debug" GENERATION_OPTIONS="-DBOARD=raspberrypi_cm4 -DLOG=1" /> - <configuration PROFILE_NAME="raspberry_pi_pico" ENABLED="true" CONFIG_NAME="Debug" GENERATION_OPTIONS="-DBOARD=raspberry_pi_pico -DLOG=1" /> + <configuration PROFILE_NAME="raspberry_pi_pico" ENABLED="false" CONFIG_NAME="Debug" GENERATION_OPTIONS="-DBOARD=raspberry_pi_pico -DLOG=1" /> <configuration PROFILE_NAME="raspberry_pi_pico-pio_host" ENABLED="false" CONFIG_NAME="Debug" GENERATION_OPTIONS="-DBOARD=raspberry_pi_pico -DLOG=1 -DCFLAGS_CLI="-DCFG_TUH_RPI_PIO_USB=1"" /> - <configuration PROFILE_NAME="raspberry_pi_pico2" ENABLED="true" CONFIG_NAME="Debug" GENERATION_OPTIONS="-DBOARD=raspberry_pi_pico2 -DLOG=1" /> + <configuration PROFILE_NAME="raspberry_pi_pico2" ENABLED="false" CONFIG_NAME="Debug" GENERATION_OPTIONS="-DBOARD=raspberry_pi_pico2 -DLOG=1" /> <configuration PROFILE_NAME="raspberry_pi_pico2-pio_host" ENABLED="false" CONFIG_NAME="Debug" GENERATION_OPTIONS="-DBOARD=raspberry_pi_pico2 -DLOG=1 -DCFLAGS_CLI="-DCFG_TUH_RPI_PIO_USB=1"" /> <configuration PROFILE_NAME="feather_rp2040" ENABLED="false" CONFIG_NAME="Debug" GENERATION_OPTIONS="-DBOARD=pico_sdk -DPICO_BOARD=adafruit_feather_rp2040 -DLOG=1" /> <configuration PROFILE_NAME="feather_rp2040_max3421" ENABLED="false" CONFIG_NAME="Debug" GENERATION_OPTIONS="-DBOARD=feather_rp2040_max3421 -DLOG=1" /> diff --git a/.idea/codeStyles/Project.xml b/.idea/codeStyles/Project.xml new file mode 100644 index 000000000..35c56fc87 --- /dev/null +++ b/.idea/codeStyles/Project.xml @@ -0,0 +1,10 @@ +<component name="ProjectCodeStyleConfiguration"> + <code_scheme name="Project" version="173"> + <RiderCodeStyleSettings> + <option name="/Default/CodeStyle/CodeFormatting/CppClangFormat/EnableClangFormatSupport/@EntryValue" value="true" type="bool" /> + </RiderCodeStyleSettings> + <clangFormatSettings> + <option name="ENABLED" value="true" /> + </clangFormatSettings> + </code_scheme> +</component>
\ No newline at end of file diff --git a/.idea/codeStyles/codeStyleConfig.xml b/.idea/codeStyles/codeStyleConfig.xml new file mode 100644 index 000000000..79ee123c2 --- /dev/null +++ b/.idea/codeStyles/codeStyleConfig.xml @@ -0,0 +1,5 @@ +<component name="ProjectCodeStyleConfiguration"> + <state> + <option name="USE_PER_PROJECT_SETTINGS" value="true" /> + </state> +</component>
\ No newline at end of file diff --git a/.idea/debugServers/lpc43s67.xml b/.idea/debugServers/lpc43s67.xml new file mode 100644 index 000000000..14af905e2 --- /dev/null +++ b/.idea/debugServers/lpc43s67.xml @@ -0,0 +1,13 @@ +<component name="DebugServers"> + <jlink-debug-target name="lpc43s67" uniqueID="832869b0-741e-4a53-9f01-68a8ed75e67c"> + <debugger version="1"> + <debugger kind="GDB" isBundled="true" /> + <env /> + </debugger> + <gdbserver exe="/usr/bin/JLinkGDBServerCLExe" /> + <console port="19021" /> + <target device="LPC43S67_M4" reset-before="false" frequency="12000" /> + <connection extended-remote="false" port="4444" warmup-ms="500" /> + <swo /> + </jlink-debug-target> +</component>
\ No newline at end of file diff --git a/.idea/debugServers/lpc55s28.xml b/.idea/debugServers/lpc55s28.xml new file mode 100644 index 000000000..afaae7368 --- /dev/null +++ b/.idea/debugServers/lpc55s28.xml @@ -0,0 +1,13 @@ +<component name="DebugServers"> + <jlink-debug-target name="lpc55s28" uniqueID="51878df1-0b10-471d-b848-25b9d159d83b"> + <debugger version="1"> + <debugger kind="GDB" isBundled="true" /> + <env /> + </debugger> + <gdbserver exe="/usr/bin/JLinkGDBServerCLExe" /> + <console port="19021" /> + <target device="LPC55S28" reset-before="false" /> + <connection extended-remote="false" port="4444" warmup-ms="500" /> + <swo /> + </jlink-debug-target> +</component>
\ No newline at end of file diff --git a/.idea/debugServers/nrf52840.xml b/.idea/debugServers/nrf52840.xml new file mode 100644 index 000000000..11312a551 --- /dev/null +++ b/.idea/debugServers/nrf52840.xml @@ -0,0 +1,13 @@ +<component name="DebugServers"> + <jlink-debug-target name="nrf52840" uniqueID="8be5d7a2-4241-4988-a733-5c870423dac9"> + <debugger version="1"> + <debugger kind="GDB" isBundled="true" /> + <env /> + </debugger> + <gdbserver exe="/usr/bin/JLinkGDBServerCLExe" /> + <console port="19021" /> + <target device="nRF52840_xxAA" reset-before="false" /> + <connection extended-remote="false" port="4444" warmup-ms="500" /> + <swo /> + </jlink-debug-target> +</component>
\ No newline at end of file diff --git a/.idea/debugServers/nrf54lm20.xml b/.idea/debugServers/nrf54lm20.xml new file mode 100644 index 000000000..3a0e40eed --- /dev/null +++ b/.idea/debugServers/nrf54lm20.xml @@ -0,0 +1,13 @@ +<component name="DebugServers"> + <jlink-debug-target name="nrf54lm20" uniqueID="7e47d1f2-72af-46d0-9038-09ff79138f1e"> + <debugger version="1"> + <debugger kind="GDB" isBundled="true" /> + <env /> + </debugger> + <gdbserver exe="/usr/bin/JLinkGDBServerCLExe" /> + <console port="19021" /> + <target device="nRF54LM20A_M33" reset-before="false" frequency="12000" /> + <connection extended-remote="false" port="4444" warmup-ms="500" /> + <swo /> + </jlink-debug-target> +</component>
\ No newline at end of file diff --git a/.idea/debugServers/sam70.xml b/.idea/debugServers/sam70.xml new file mode 100644 index 000000000..6659a857d --- /dev/null +++ b/.idea/debugServers/sam70.xml @@ -0,0 +1,13 @@ +<component name="DebugServers"> + <jlink-debug-target name="sam70" uniqueID="1e732be4-71f3-41be-b79b-e91778969278"> + <debugger version="1"> + <debugger kind="GDB" isBundled="true" /> + <env /> + </debugger> + <gdbserver exe="/usr/bin/JLinkGDBServerCLExe" /> + <console port="19021" /> + <target device="ATSAME70Q21B" reset-before="false" frequency="16000" /> + <connection extended-remote="false" port="4444" warmup-ms="500" /> + <swo /> + </jlink-debug-target> +</component>
\ No newline at end of file diff --git a/.idea/debugServers/stm32f407.xml b/.idea/debugServers/stm32f407.xml new file mode 100644 index 000000000..660c182e1 --- /dev/null +++ b/.idea/debugServers/stm32f407.xml @@ -0,0 +1,13 @@ +<component name="DebugServers"> + <jlink-debug-target name="stm32f407" uniqueID="aa9cccb2-ff90-4f91-a5ec-07fbd834a0b8" selected="true"> + <debugger version="1"> + <debugger kind="GDB" isBundled="true" /> + <env /> + </debugger> + <gdbserver exe="/usr/bin/JLinkGDBServerCLExe" /> + <console port="19021" /> + <target device="STM32F407VG" reset-before="false" frequency="12000" /> + <connection extended-remote="false" port="4444" warmup-ms="500" /> + <swo /> + </jlink-debug-target> +</component>
\ No newline at end of file diff --git a/.idea/debugServers/stm32h7s3.xml b/.idea/debugServers/stm32h7s3.xml new file mode 100644 index 000000000..8ba7a5b70 --- /dev/null +++ b/.idea/debugServers/stm32h7s3.xml @@ -0,0 +1,13 @@ +<component name="DebugServers"> + <jlink-debug-target name="stm32h7s3" uniqueID="05f9124a-7d2e-47a3-8c55-665273739888"> + <debugger version="1"> + <debugger kind="GDB" isBundled="true" /> + <env /> + </debugger> + <gdbserver exe="/usr/bin/JLinkGDBServerCLExe" /> + <console port="19021" /> + <target device="STM32H7S3L8" reset-before="false" frequency="16000" /> + <connection extended-remote="false" port="4444" warmup-ms="500" /> + <swo /> + </jlink-debug-target> +</component>
\ No newline at end of file diff --git a/.idea/debugServers/stm32l476.xml b/.idea/debugServers/stm32l476.xml new file mode 100644 index 000000000..457908d01 --- /dev/null +++ b/.idea/debugServers/stm32l476.xml @@ -0,0 +1,13 @@ +<component name="DebugServers"> + <jlink-debug-target name="stm32l476" uniqueID="13bf3c09-867a-47a2-938a-de86490345be"> + <debugger version="1"> + <debugger kind="GDB" isBundled="true" /> + <env /> + </debugger> + <gdbserver exe="/usr/bin/JLinkGDBServerCLExe" /> + <console port="19021" /> + <target device="STM32L476VG" reset-before="false" /> + <connection extended-remote="false" port="4444" warmup-ms="500" /> + <swo /> + </jlink-debug-target> +</component>
\ No newline at end of file diff --git a/.idea/debugServers/wch_riscv.xml b/.idea/debugServers/wch_riscv.xml index 2e147f1b6..0b2b83b2e 100644 --- a/.idea/debugServers/wch_riscv.xml +++ b/.idea/debugServers/wch_riscv.xml @@ -4,7 +4,7 @@ <debugger kind="GDB" isBundled="true" /> <env /> </debugger> - <gdbserver dir="$CMakeProjectDir$" exe="$USER_HOME$/app/riscv-openocd-wch/src/openocd" args="-f hw/bsp/ch32v20x/wch-riscv.cfg"> + <gdbserver dir="$CMakeProjectDir$" exe="openocd" args="-f target/wch-riscv.cfg"> <env /> </gdbserver> <console enabled="true" port="4444" /> diff --git a/.idea/editor.xml b/.idea/editor.xml new file mode 100644 index 000000000..07792cad7 --- /dev/null +++ b/.idea/editor.xml @@ -0,0 +1,49 @@ +<?xml version="1.0" encoding="UTF-8"?> +<project version="4"> + <component name="BackendCodeEditorSettings"> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppClassCanBeFinal/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppConstexprIfDiscardedBranch/@EntryIndexedValue" value="HINT" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFAArrayIndexOutOfBounds/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFAConstantConditions/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFAConstantFunctionResult/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFAConstantParameter/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFADeletedPointer/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFAEndlessLoop/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFAInfiniteRecursion/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFAInvalidatedMemory/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFALocalValueEscapesFunction/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFALocalValueEscapesScope/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFALoopConditionNotUpdated/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFAMemoryLeak/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFANotInitializedField/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFANullDereference/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFATimeOver/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFAUnreachableCode/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFAUnreachableFunctionCall/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFAUnreadVariable/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDFAUnusedValue/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDefinitionsOrder/@EntryIndexedValue" value="HINT" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppDeprecatedOverridenMethod/@EntryIndexedValue" value="WARNING" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppEnforceCVQualifiersOrder/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppEnforceCVQualifiersPlacement/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppEnforceDoStatementBraces/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppEnforceForStatementBraces/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppEnforceFunctionDeclarationStyle/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppEnforceIfStatementBraces/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppEnforceTypeAliasCodeStyle/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppEnforceWhileStatementBraces/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppInconsistentNaming/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppLocalVariableWithNonTrivialDtorIsNeverUsed/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppModulePartitionWithSeveralPartitionUnits/@EntryIndexedValue" value="WARNING" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppOverrideWithDifferentVisibility/@EntryIndexedValue" value="WARNING" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppRedundantFwdClassOrEnumSpecifier/@EntryIndexedValue" value="SUGGESTION" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppRedundantQualifierADL/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppReferenceToOverriddenVirtualFunction/@EntryIndexedValue" value="WARNING" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppRemoveRedundantBraces/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppTabsAreDisallowed/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppUnnecessaryWhitespace/@EntryIndexedValue" value="DO_NOT_SHOW" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppUseDesignatedInitializers/@EntryIndexedValue" value="HINT" type="string" /> + <option name="/Default/CodeInspection/Highlighting/InspectionSeverities/=CppUseInternalLinkage/@EntryIndexedValue" value="SUGGESTION" type="string" /> + <option name="/Default/CodeStyle/CodeFormatting/CppClangFormat/EnableClangFormatSupport/@EntryValue" value="true" type="bool" /> + </component> +</project>
\ No newline at end of file diff --git a/.idea/hil-pool-check.iml b/.idea/hil-pool-check.iml new file mode 100644 index 000000000..4c9423543 --- /dev/null +++ b/.idea/hil-pool-check.iml @@ -0,0 +1,2 @@ +<?xml version="1.0" encoding="UTF-8"?> +<module classpath="CIDR" type="CPP_MODULE" version="4" />
\ No newline at end of file diff --git a/.idea/misc.xml b/.idea/misc.xml new file mode 100644 index 000000000..7ed4f1ab9 --- /dev/null +++ b/.idea/misc.xml @@ -0,0 +1,11 @@ +<?xml version="1.0" encoding="UTF-8"?> +<project version="4"> + <component name="CMakeWorkspace" PROJECT_DIR="$PROJECT_DIR$/examples"> + <contentRoot DIR="$PROJECT_DIR$" /> + </component> + <component name="CidrRootsConfiguration"> + <excludeRoots> + <file path="$PROJECT_DIR$/hw/mcu" /> + </excludeRoots> + </component> +</project> diff --git a/.idea/modules.xml b/.idea/modules.xml new file mode 100644 index 000000000..063c94f51 --- /dev/null +++ b/.idea/modules.xml @@ -0,0 +1,8 @@ +<?xml version="1.0" encoding="UTF-8"?> +<project version="4"> + <component name="ProjectModuleManager"> + <modules> + <module fileurl="file://$PROJECT_DIR$/.idea/hil-pool-check.iml" filepath="$PROJECT_DIR$/.idea/hil-pool-check.iml" /> + </modules> + </component> +</project>
\ No newline at end of file diff --git a/.idea/runConfigurations/k64f.xml b/.idea/runConfigurations/k64f.xml deleted file mode 100644 index 6db0dd74e..000000000 --- a/.idea/runConfigurations/k64f.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="k64f" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="nxp" PROGRAM_PARAMS="-device "MK64FN1M0xxx12" -if swd -speed 10000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/kl25.xml b/.idea/runConfigurations/kl25.xml deleted file mode 100644 index bb7e1707b..000000000 --- a/.idea/runConfigurations/kl25.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="kl25" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="nxp" PROGRAM_PARAMS="-device "MKL25Z128xxx4" -if swd -speed 50000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/lpc1857.xml b/.idea/runConfigurations/lpc1857.xml deleted file mode 100644 index ef8178e08..000000000 --- a/.idea/runConfigurations/lpc1857.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="lpc1857" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="nxp" PROGRAM_PARAMS="-device "lpc1857" -if swd -speed 50000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="device_info" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="device_info"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/lpc4088.xml b/.idea/runConfigurations/lpc4088.xml deleted file mode 100644 index 6c6886f30..000000000 --- a/.idea/runConfigurations/lpc4088.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="lpc4088" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="nxp" PROGRAM_PARAMS="-device "lpc4088" -if swd -speed 50000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/lpc54628.xml b/.idea/runConfigurations/lpc54628.xml deleted file mode 100644 index 4b871d543..000000000 --- a/.idea/runConfigurations/lpc54628.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="lpc54628" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="nxp" PROGRAM_PARAMS="-device "LPC54628J512" -if swd -speed 50000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/lpc55s69.xml b/.idea/runConfigurations/lpc55s69.xml deleted file mode 100644 index 7ab9fac66..000000000 --- a/.idea/runConfigurations/lpc55s69.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="lpc55s69" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="nxp" PROGRAM_PARAMS="-device "lpc55s69" -if swd -speed 50000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/mcx947.xml b/.idea/runConfigurations/mcx947.xml deleted file mode 100644 index 2a9805145..000000000 --- a/.idea/runConfigurations/mcx947.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="mcx947" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="nxp" PROGRAM_PARAMS="-device "MCXN947_M33_0" -if swd -speed 50000 -port 25321 -nogui -singlerun -jlinkscriptfile $ProjectFileDir$/hw/bsp/mcx/debug.jlinkscript" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/nrf52840.xml b/.idea/runConfigurations/nrf52840.xml deleted file mode 100644 index 5a4f4837b..000000000 --- a/.idea/runConfigurations/nrf52840.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="nrf52840" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="nrf" PROGRAM_PARAMS="-select usb=752001685 -device "nrf52840_xxaa" -if swd -speed 8000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/nrf5340.xml b/.idea/runConfigurations/nrf5340.xml deleted file mode 100644 index bf1cb2938..000000000 --- a/.idea/runConfigurations/nrf5340.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="nrf5340" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="nrf" PROGRAM_PARAMS="-select usb=752001685 -device "nrf5340_xxaa_app" -if swd -speed 16000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/ra2a1.xml b/.idea/runConfigurations/ra2a1.xml deleted file mode 100644 index d50b3d729..000000000 --- a/.idea/runConfigurations/ra2a1.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="ra2a1" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="renesas" PROGRAM_PARAMS="-device "R7FA2A1AB" -if swd -speed 20000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/ra4m1.xml b/.idea/runConfigurations/ra4m1.xml deleted file mode 100644 index 0cccb60d2..000000000 --- a/.idea/runConfigurations/ra4m1.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="ra4m1" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="renesas" PROGRAM_PARAMS="-device "R7FA4M1AB" -if swd -speed 50000 -port 25321 -nogui -singlerun -jlinkscriptfile $PROJECT_DIR$/hw/bsp/ra/debug.jlinkscript" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/ra6m1.xml b/.idea/runConfigurations/ra6m1.xml deleted file mode 100644 index 5efd47753..000000000 --- a/.idea/runConfigurations/ra6m1.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="ra6m1" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="renesas" PROGRAM_PARAMS="-device "R7FA6M1AD" -if swd -speed 50000 -port 25321 -nogui -singlerun -jlinkscriptfile $PROJECT_DIR$/hw/bsp/ra/debug.jlinkscript" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/ra6m5.xml b/.idea/runConfigurations/ra6m5.xml deleted file mode 100644 index 713fc68cc..000000000 --- a/.idea/runConfigurations/ra6m5.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="ra6m5" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="renesas" PROGRAM_PARAMS="-device "R7FA6M5BH" -if swd -speed 50000 -port 25321 -nogui -singlerun -jlinkscriptfile $PROJECT_DIR$/hw/bsp/ra/debug.jlinkscript" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/rt1010.xml b/.idea/runConfigurations/rt1010.xml deleted file mode 100644 index c3582512c..000000000 --- a/.idea/runConfigurations/rt1010.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="rt1010" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="nxp" PROGRAM_PARAMS="-device "MIMXRT1011xxx5A" -if swd -speed 50000 -port 25321 -nogui -singlerun -jlinkscriptfile $ProjectFileDir$/hw/bsp/imxrt/debug.jlinkscript" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/rt1060.xml b/.idea/runConfigurations/rt1060.xml deleted file mode 100644 index 649fe6dac..000000000 --- a/.idea/runConfigurations/rt1060.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="rt1060" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="nxp" PROGRAM_PARAMS="-device "MIMXRT1062xxx5A" -if swd -speed 50000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="device_info" TARGET_NAME="device_info" version="1" RUN_TARGET_PROJECT_NAME="device_info" RUN_TARGET_NAME="device_info"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/samd21g18.xml b/.idea/runConfigurations/samd21g18.xml deleted file mode 100644 index 2ea822493..000000000 --- a/.idea/runConfigurations/samd21g18.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="samd21g18" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="microchip" PROGRAM_PARAMS="-device "ATSAMD21G18" -if swd -speed 50000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/samd51j19.xml b/.idea/runConfigurations/samd51j19.xml deleted file mode 100644 index b6cbe253a..000000000 --- a/.idea/runConfigurations/samd51j19.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="samd51j19" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="microchip" PROGRAM_PARAMS="-device "ATSAMD51J19A" -if swd -speed 50000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/stlink.xml b/.idea/runConfigurations/stlink.xml deleted file mode 100644 index e84445add..000000000 --- a/.idea/runConfigurations/stlink.xml +++ /dev/null @@ -1,10 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="stlink" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" PROGRAM_PARAMS="-p 49578 -cp "$USER_HOME$/st/stm32cubeide_1.16.1/plugins/com.st.stm32cube.ide.mcu.externaltools.cubeprogrammer.linux64_2.1.400.202404281720/tools/bin" --frequency 8000 --swd" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" CONFIG_NAME="raspberry_pi_pico" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::49578" executable="$USER_HOME$/st/stm32cubeide_1.16.1/plugins/com.st.stm32cube.ide.mcu.externaltools.stlink-gdb-server.linux64_2.1.400.202404281720/tools/bin/ST-LINK_gdbserver" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD"> - <debugger kind="GDB" isBundled="true" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/stm32g474.xml b/.idea/runConfigurations/stm32g474.xml deleted file mode 100644 index 600b1e555..000000000 --- a/.idea/runConfigurations/stm32g474.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="stm32g474" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="stm32" PROGRAM_PARAMS="-device "stm32g474re" -if swd -speed 50000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/stm32h563.xml b/.idea/runConfigurations/stm32h563.xml deleted file mode 100644 index 9c0ffc2ec..000000000 --- a/.idea/runConfigurations/stm32h563.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="stm32h563" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="stm32" PROGRAM_PARAMS="-device "stm32h563zi" -if swd -speed 50000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/stm32h743.xml b/.idea/runConfigurations/stm32h743.xml deleted file mode 100644 index 1565e92cd..000000000 --- a/.idea/runConfigurations/stm32h743.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="stm32h743" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="stm32" PROGRAM_PARAMS="-device "stm32h743xi" -if swd -speed 50000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc_hid" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc_hid"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/stm32u5a5.xml b/.idea/runConfigurations/stm32u5a5.xml deleted file mode 100644 index 92a1293be..000000000 --- a/.idea/runConfigurations/stm32u5a5.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="stm32u5a5" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="stm32" PROGRAM_PARAMS="-device "stm32u5a5zj" -if swd -speed 50000 -port 25321 -nogui -singlerun" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.idea/runConfigurations/uno_r4.xml b/.idea/runConfigurations/uno_r4.xml deleted file mode 100644 index c69e2939c..000000000 --- a/.idea/runConfigurations/uno_r4.xml +++ /dev/null @@ -1,11 +0,0 @@ -<component name="ProjectRunConfigurationManager"> - <configuration default="false" name="uno_r4" type="com.jetbrains.cidr.embedded.customgdbserver.type" factoryName="com.jetbrains.cidr.embedded.customgdbserver.factory" folderName="renesas" PROGRAM_PARAMS="-device "R7FA4M1AB" -if swd -speed 20000 -port 25321 -nogui -singlerun -jlinkscriptfile $PROJECT_DIR$/hw/bsp/ra/debug.jlinkscript" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="tinyusb_examples" TARGET_NAME="cdc_msc" version="1" RUN_TARGET_PROJECT_NAME="tinyusb_examples" RUN_TARGET_NAME="cdc_msc"> - <custom-gdb-server version="1" gdb-connect="tcp::25321" executable="/usr/bin/JLinkGDBServer" warmup-ms="0" download-type="ALWAYS" reset-cmd="monitor reset" reset-type="AFTER_DOWNLOAD" connection-mode="REMOTE" gdbserver-enable="false" console-enabled="false" console-port="0"> - <debugger kind="GDB" isBundled="true" /> - <extra-debugger-params debugger-working-dir="" debugger-env-pass-parent="true" custom-script="" /> - </custom-gdb-server> - <method v="2"> - <option name="CLION.COMPOUND.BUILD" enabled="true" /> - </method> - </configuration> -</component>
\ No newline at end of file diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index f4a297289..3d9c8482b 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -33,6 +33,13 @@ repos: - repo: local hooks: + - id: unique-example-pids + name: unique example USB PIDs + files: usb_descriptors\.c$ + entry: python3 tools/check_example_pids.py + pass_filenames: false + language: system + - id: unit-test name: unit-test files: ^(src/|test/unit-test/) @@ -41,6 +48,27 @@ repos: types_or: [c, header] language: system + # Two hooks, split by what each suite actually reads. The full discovery run costs + # ~55s (deliberate hang/timeout simulations); only test_hil_select (~0.1s) reads + # hw/bsp (board.cmake), src (portable dirs + class include graph) and examples + # (tusb_config.h per test) -- renaming a board, port dir or example breaks it without + # touching test/hil, and catching that here beats waiting for pre-commit CI. + # No types_or: the rig rosters (*.json) are inputs too. + # examples/device/mtp/src is in scope: test_hil_bounded parses README_TXT_CONTENT + # and md5-checks the logo header from there as its MTP fixtures. + - id: hil-test + name: hil-test + files: ^(test/hil/|examples/device/mtp/src/) + entry: python3 -m unittest discover -s test/hil/test + pass_filenames: false + language: system + - id: hil-select-test + name: hil-select-test + files: ^(hw/bsp/|src/|examples/) + entry: python3 test/hil/test/test_hil_select.py + pass_filenames: false + language: system + # - id: build-fuzzer # name: build-fuzzer # files: ^(src/|test/fuzz/) diff --git a/AGENTS.md b/AGENTS.md index f69083697..681311eb9 100644..120000 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,213 +1 @@ -# TinyUSB Agent Instructions - -TinyUSB is a cross-platform USB Host/Device stack for embedded systems: memory-safe (no dynamic allocation) and thread-safe (ISR events deferred to task context). - -Reference these instructions first; fall back to search/bash only when reality diverges. - -## Behavioral Guidelines - -Bias toward caution over speed. For trivial tasks, use judgment. - -- **Think first** — state assumptions; ask if unclear; present alternatives instead of picking silently. -- **Simplicity** — no features, abstractions, flexibility, or error handling beyond what was asked. If 200 lines could be 50, rewrite. -- **Surgical changes** — touch only what the task requires; match existing style; don't refactor working code; mention unrelated dead code rather than deleting it. Remove only orphans *your* changes created. -- **Goal-driven** — turn tasks into verifiable goals ("write failing test, make it pass"). For multi-step work, state a brief `step → verify` plan. -- **Worktrees** — default to a git worktree (`git worktree add`) for any branch or multi-step work; never switch the shared primary checkout's branch. Sessions run concurrently: switching the primary checkout mid-flight disrupts other sessions and can silently point a review, build, or commit at the wrong diff. Only trivial one-shot fixes may skip this. - -## Ground Rules - -- **Language/style:** C99, 2-space indent (no tabs), snake_case helpers, `UPPER_CASE` macros. Public APIs use `tud_`/`tuh_`; macros use `TU_`. Headers self-contained with `#if CFG_TUSB_MCU` guards. -- **Safety:** no dynamic allocation; defer ISR work to task context; use `TU_ASSERT()` for error checks; always check return values; include order: C stdlib → tusb common → drivers → classes. -- **Layout:** `src/` core, `hw/{mcu,bsp}/` MCU+BSP, `examples/{device,host,dual}/`, `test/{unit-test,fuzz,hil}/`, `docs/`, `tools/`. -- **Commits/PRs:** imperative mood, scoped changes, link issues, include test/build evidence. After opening a PR, monitor it and drive it to green: address automated review comments (Copilot/Codex/Claude) and fix any failing CI builds, pushing follow-up commits until checks pass and review threads are resolved. Useful: `gh pr checks <num> --watch`, `gh pr view <num> --comments`. -- **Formatting/lint:** `clang-format` (`.clang-format`), `codespell` (`.codespellrc`), run `pre-commit run --all-files` before submitting. - -## Bootstrap - -```bash -sudo apt-get install -y gcc-arm-none-eabi # ARM toolchain (2-5 min, one-time) -python3 tools/get_deps.py [FAMILY|-b BOARD] # fetch deps into lib/, hw/mcu/ (<1 s) -. $HOME/code/esp-idf/export.sh # Espressif only: before any build/flash/monitor -``` - -## Build - -Single example (CMake+Ninja, recommended, 1-3 s): -```bash -cd examples/device/cdc_msc && mkdir -p build && cd build -cmake -DBOARD=raspberry_pi_pico -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel .. -cmake --build . -``` - -All examples for a board (15-20 s; some objcopy failures are non-critical). Use `cmake-build-<board>` as the build dir — HIL tests expect that exact name: -```bash -cd examples -cmake -B cmake-build-raspberry_pi_pico -DBOARD=raspberry_pi_pico -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel . -cmake --build cmake-build-raspberry_pi_pico -``` - -Single example with Make: -```bash -cd examples/device/cdc_msc && make BOARD=raspberry_pi_pico all -``` - -Espressif (only ESP-IDF examples like `cdc_msc_freertos`): -```bash -. $HOME/code/esp-idf/export.sh -cd examples/device/cdc_msc_freertos -idf.py -DBOARD=espressif_s3_devkitc build -``` - -**Build options** (CMake `-D…` / Make `…=…`): -- Debug: `CMAKE_BUILD_TYPE=Debug` / `DEBUG=1` -- Logging: `LOG=2` (add `LOGGER=rtt` for RTT) -- Root hub port: `RHPORT_DEVICE=1` -- Speed: `RHPORT_DEVICE_SPEED=OPT_MODE_FULL_SPEED` - -## Flash - -```bash -# JLink -ninja cdc_msc-jlink # CMake -make BOARD=<board> flash-jlink # Make - -# OpenOCD -ninja cdc_msc-openocd # CMake -make BOARD=<board> flash-openocd # Make - -# UF2 -ninja cdc_msc-uf2 # CMake -make BOARD=<board> all uf2 # Make - -ninja -t targets # list CMake targets - -# Espressif (after . $HOME/code/esp-idf/export.sh) -idf.py -DBOARD=<board> flash -idf.py -DBOARD=<board> monitor -``` - -## GDB Debugging - -Look up `JLINK_DEVICE` / `OPENOCD_OPTION` in `hw/bsp/*/boards/*/board.cmake` (CMake builds) or `board.mk` (Make builds). - -**JLink — Terminal 1:** -```bash -JLinkGDBServer -device stm32h743xi -if SWD -speed 4000 -port 2331 -swoport 2332 -telnetport 2333 -nogui -``` - -**OpenOCD — Terminal 1:** -```bash -openocd -f interface/stlink.cfg -f target/stm32h7x.cfg -# or with a J-Link interface: -openocd -f interface/jlink.cfg -f target/stm32h7x.cfg -# rp2040/rp2350 via CMSIS-DAP: -openocd -f interface/cmsis-dap.cfg -f target/rp2040.cfg -c "adapter speed 5000" -``` - -**Terminal 2 — connect GDB** (replace `<port>` with `2331` for JLinkGDBServer or `3333` for OpenOCD): -```bash -arm-none-eabi-gdb /tmp/build/firmware.elf -(gdb) target remote :<port> -(gdb) monitor reset halt -(gdb) load -(gdb) break main # optional, to stop at entry -(gdb) continue -``` - -**RTT logging:** build with `LOG=2 LOGGER=rtt`, flash, then run JLinkGDBServer with `-RTTTelnetPort 19021`, and in another terminal `JLinkRTTClient` (pipe to `tee rtt.log` or use `timeout 20s JLinkRTTClient > rtt.log` for non-interactive capture). - -## Testing - -**Unit (Ceedling, Unity+CMock, ~4 s):** -```bash -sudo gem install ceedling -cd test/unit-test && ceedling test:all # or ceedling test:test_fifo -``` - -**HIL (2-5 min):** invoke the `hil` skill (`.claude/skills/hil/SKILL.md`) for the full procedure (local vs remote mode, config selection, SSH copy steps, debugging tips). Requires pre-built examples — see Build → "All examples for a board". - -## Documentation - -Sphinx docs in `docs/` (reStructuredText `.rst` or Markdown `.md` via MyST). Use the `build-doc` skill (`.claude/skills/build-doc/SKILL.md`) to build/preview locally (`sphinx-build`) and to regenerate auto-generated files (`tools/gen_doc.py` + `tools/gen_presets.py`) after adding a board or dependency. - -## Code Size Metrics - -Verify size impact before committing. Invoke the `code-size` skill (`.claude/skills/code-size/SKILL.md`) — it wraps `tools/metrics_compare_base.py` to handle the base-vs-branch worktree + build + compare flow. - -Quick reference: -```bash -# Single example, one board: -python3 tools/metrics_compare_base.py -b raspberry_pi_pico -e device/cdc_msc -# Add --bloaty for section/symbol breakdown. - -# All examples, one board: -python3 tools/metrics_compare_base.py -b raspberry_pi_pico - -# All arm-gcc CI families combined (pre-merge sweep, 4-8 min): -python3 tools/metrics_compare_base.py --ci -``` - -Reports land in `cmake-metrics/<board>/metrics_compare.md` (per-board) and `cmake-metrics/_combined/metrics_compare.md` (with `--combined`/`--ci`). - -## Static Analysis (PVS-Studio) - -Requires `compile_commands.json`, which the examples build exports by default -(`hw/bsp/family_support.cmake` sets `CMAKE_EXPORT_COMPILE_COMMANDS ON`). The -`pvs` skill (`.claude/skills/pvs/SKILL.md`) wraps the build + analyze flow for a -board; the commands below are the underlying steps. - -```bash -# Whole project: -pvs-studio-analyzer analyze \ - -f examples/cmake-build-raspberry_pi_pico/compile_commands.json \ - -R .PVS-Studio/.pvsconfig \ - -o pvs-report.log -j12 \ - --security-related-issues \ - --misra-c-version 2023 --misra-cpp-version 2008 --use-old-parser - -# Specific files: -S takes a plaintext list (one path per line), not paths directly: -printf 'src/foo.c\nsrc/bar.c\n' > files.txt -pvs-studio-analyzer analyze \ - -f examples/cmake-build-raspberry_pi_pico/compile_commands.json \ - -R .PVS-Studio/.pvsconfig \ - -S files.txt \ - -o pvs-report.log -j12 \ - --security-related-issues \ - --misra-c-version 2023 --misra-cpp-version 2008 --use-old-parser - -plog-converter -a GA:1,2 -t errorfile pvs-report.log # view results -``` - -Takes ~10-30 s. (`--dump-files` adds preprocessed `.PVS-Studio.i/.cfg` dumps next -to every source for false-positive debugging — omit it for normal runs.) - -## Validation After Changes - -1. `pre-commit run --all-files` — format, spell, unit tests (10-15 s). -2. Build at least one board's full example set (Build → "All examples for a board") for modules you touched. -3. Run relevant unit tests; add fuzz/HIL coverage for parsers or protocol state machines. - -**Boards good for local testing:** -- `stm32f407disco` — no external SDK -- `raspberry_pi_pico` — Pico SDK required -- Others: see `hw/bsp/FAMILY/boards/` - -Device examples need real hardware to validate runtime behavior; must at least build. - -## Release - -Cutting a release — version bump, regenerated files, the per-release changelog, validation, and the maintainer's commit/tag/GitHub-release — is handled by the `make-release` skill (`.claude/skills/make-release/SKILL.md`). - -## References - -- MCU reference manuals, datasheets, schematics: `$HOME/Documents/calibre-library`. -- Supported MCUs/boards: `hw/bsp/` and `docs/reference/boards.rst`. -- USB classes: `src/class/{cdc,hid,msc,audio,…}/` — each has `*_device.c` and `*_host.c`. -- Key files: `src/tusb.h`, `src/tusb_config.h`, `tools/get_deps.py`, `tools/build.py`, `test/unit-test/project.yml`. - -## Common Build Issues - -- Missing compiler → install `gcc-arm-none-eabi`. -- Missing deps → `python3 tools/get_deps.py FAMILY`. -- Unknown board → check `hw/bsp/FAMILY/boards/`. -- `objcopy` errors in full builds are often non-critical; retry the single example. +CLAUDE.md
\ No newline at end of file diff --git a/CLAUDE.md b/CLAUDE.md index 47dc3e3d8..762473714 120000..100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1 +1,139 @@ -AGENTS.md
\ No newline at end of file +# TinyUSB Agent Instructions + +TinyUSB is a cross-platform USB Host/Device stack for embedded systems: memory-safe (no dynamic allocation) and thread-safe (ISR events deferred to task context). + +Reference these instructions first; fall back to search/bash only when reality diverges. + +## Behavioral Guidelines + +Bias toward caution over speed. For trivial tasks, use judgment. + +- **Think first** — state assumptions; ask if unclear; present alternatives instead of picking silently. +- **Simplicity** — no features, abstractions, flexibility, or error handling beyond what was asked. If 200 lines could be 50, rewrite. +- **Surgical changes** — touch only what the task requires; match existing style; don't refactor working code; mention unrelated dead code rather than deleting it. Remove only orphans *your* changes created. +- **Goal-driven** — turn tasks into verifiable goals ("write failing test, make it pass"). For multi-step work, state a brief `step → verify` plan. +- **Worktrees** — default to a git worktree for any branch or multi-step work; never switch the shared primary checkout's branch. Sessions run concurrently: switching the primary checkout mid-flight disrupts other sessions and can silently point a review, build, or commit at the wrong diff. Only trivial one-shot fixes may skip this. Standard location: `.worktrees/<branch-name>` at the repo root (gitignored), e.g. `git worktree add .worktrees/my-branch -b my-branch`. In a new worktree, symlink the dependency dirs (`lib/*`, `hw/mcu/*`, `tools/linkermap` — the keys of `deps_all` in `tools/get_deps.py`) to the primary checkout instead of re-cloning them; only if the branch needs a different dep revision, replace that one symlink with a real dir and run `get_deps.py` for it. + +## Ground Rules + +- **Language/style:** C99, 2-space indent (no tabs), snake_case helpers, `UPPER_CASE` macros. Public APIs use `tud_`/`tuh_`; macros use `TU_`. Headers self-contained with `#if CFG_TUSB_MCU` guards. +- **Safety:** no dynamic allocation; defer ISR work to task context; use `TU_ASSERT()` for error checks; always check return values; include order: C stdlib → tusb common → drivers → classes. +- **Layout:** `src/` core, `hw/{mcu,bsp}/` MCU+BSP, `examples/{device,host,dual}/`, `test/{unit-test,fuzz,hil}/`, `docs/`, `tools/`. +- **Commits/PRs:** imperative mood, scoped changes, link issues, include test/build evidence. After opening a PR, drive it to green: address automated review comments (Copilot/Codex/Claude) and fix failing CI, pushing follow-ups until checks pass and threads resolve. Useful: `gh pr checks <num> --watch`, `gh pr view <num> --comments`. +- **Deferred work:** work that is worth doing but is a *separate scope* from the current PR — it deserves its own PR, written by a different session. Write it as a **handoff** with the `superpowers:writing-plans` skill, one doc per follow-up, in `docs/superpowers/followup/pr<NNN>-<topic>.md` (the PR it was split out of, so the origin stays traceable). Say what is already established (with citations/measurements), what remains, and why it was split out. Delete the doc when its PR lands. Never bundle unrelated follow-ups into one file. +- **Formatting/lint:** `clang-format` (`.clang-format`), `codespell` (`.codespellrc`); run `pre-commit run --all-files` before submitting. + +## Bootstrap + +```bash +sudo apt-get install -y gcc-arm-none-eabi # ARM toolchain (2-5 min, one-time) +python3 tools/get_deps.py [FAMILY|-b BOARD] # fetch deps into lib/, hw/mcu/ (<1 s) +. $HOME/code/esp-idf/export.sh # Espressif only: before any build/flash/monitor +``` + +## Build + +Single example (CMake+Ninja, recommended, 1-3 s): +```bash +cd examples/device/cdc_msc && mkdir -p build && cd build +cmake -DBOARD=raspberry_pi_pico -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel .. && cmake --build . +``` + +All examples for a board (15-20 s; some objcopy failures are non-critical). The build dir **must** be `cmake-build-<board>` — HIL tests expect that exact name: +```bash +cd examples +cmake -B cmake-build-raspberry_pi_pico -DBOARD=raspberry_pi_pico -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel . && cmake --build cmake-build-raspberry_pi_pico +``` + +- **Make:** `cd examples/device/cdc_msc && make BOARD=raspberry_pi_pico all` +- **Espressif** (ESP-IDF examples only, e.g. `cdc_msc_freertos`): after `export.sh`, `cd examples/device/cdc_msc_freertos && idf.py -DBOARD=espressif_s3_devkitc build` +- **Options** (CMake `-D…` / Make `…=…`): `CMAKE_BUILD_TYPE=Debug`/`DEBUG=1`; `LOG=2` (`LOGGER=rtt` for RTT); `RHPORT_DEVICE=1`; `RHPORT_DEVICE_SPEED=OPT_MODE_FULL_SPEED` + +## Flash + +```bash +ninja cdc_msc-jlink # CMake; Make: make BOARD=<board> flash-jlink +ninja cdc_msc-openocd # CMake; Make: make BOARD=<board> flash-openocd +ninja cdc_msc-uf2 # CMake; Make: make BOARD=<board> all uf2 +ninja -t targets # list CMake targets +``` +Espressif (after `export.sh`): `idf.py -DBOARD=<board> flash` / `… monitor`. + +## GDB Debugging + +Look up `JLINK_DEVICE` / `OPENOCD_OPTION` in `hw/bsp/*/boards/*/board.cmake` (CMake) or `board.mk` (Make). + +Terminal 1 — start a gdbserver: +```bash +JLinkGDBServer -device stm32h743xi -if SWD -speed 4000 -port 2331 -nogui # JLink → :2331 +openocd -f interface/stlink.cfg -f target/stm32h7x.cfg # OpenOCD → :3333 +openocd -f interface/cmsis-dap.cfg -f target/rp2040.cfg -c "adapter speed 5000" # rp2040/rp2350 +``` +Terminal 2 — connect (`<port>`: 2331 JLink, 3333 OpenOCD): +```bash +arm-none-eabi-gdb build/your_app.elf +(gdb) target remote :<port> # then: monitor reset halt → load → continue +``` +**RTT:** build `LOG=2 LOGGER=rtt`, run JLinkGDBServer with `-RTTTelnetPort 19021`, then `JLinkRTTClient` (`timeout 20s JLinkRTTClient > rtt.log` for non-interactive capture). + +## Testing + +**Unit (Ceedling, Unity+CMock, ~4 s):** +```bash +sudo gem install ceedling +cd test/unit-test && ceedling test:all # or ceedling test:test_fifo +``` + +**HIL (2-5 min):** invoke the `hil` skill (`.claude/skills/hil/SKILL.md`) — local vs remote mode, config selection, SSH copy steps, debugging. Requires pre-built examples (Build → "All examples for a board"). + +## Documentation + +Sphinx docs in `docs/` (`.rst`, or `.md` via MyST). Use the `build-doc` skill (`.claude/skills/build-doc/SKILL.md`) to build/preview locally and regenerate auto-generated files (`tools/gen_doc.py` + `tools/gen_presets.py`) after adding a board or dependency. + +## Code Size Metrics + +Verify size impact before committing with the `code-size` skill (`.claude/skills/code-size/SKILL.md`) — it wraps `tools/metrics_compare_base.py` for the base-vs-branch worktree + build + compare. Scopes: single example (`-e device/cdc_msc -b <board>`, add `--bloaty`), all examples on a board (`-b <board>`), or all arm-gcc CI families (`--ci`). Reports land in `cmake-metrics/<board>/metrics_compare.md` (and `_combined/` for `--ci`). + +## Static Analysis (PVS-Studio) + +Use the `pvs` skill (`.claude/skills/pvs/SKILL.md`) — it builds the examples with an exported `compile_commands.json` and runs SAST + MISRA C:2023/C++:2008 for a board, emitting readable + SARIF output (~10-30 s). The examples build exports `compile_commands.json` by default. + +## Validation After Changes + +1. `pre-commit run --all-files` — format, spell, unit tests, HIL suites (~55 s; the + HIL hooks deliberately exercise real timeouts and hangs). +2. Build at least one board's full example set (Build → "All examples for a board") for modules you touched. +3. Run relevant unit tests; add fuzz/HIL coverage for parsers or protocol state machines. + +**Boards good for local testing:** +- `stm32f407disco` — no external SDK +- `raspberry_pi_pico` — Pico SDK required +- Others: see `hw/bsp/FAMILY/boards/` + +Device examples need real hardware to validate runtime behavior; must at least build. + +## Release + +Cutting a release — version bump, regenerated files, the per-release changelog, validation, and the maintainer's commit/tag/GitHub-release — is handled by the `make-release` skill (`.claude/skills/make-release/SKILL.md`). + +## References + +- MCU reference manuals, datasheets, schematics: before answering register/bitfield/pinout/errata/timing questions from memory or the web — or changing a specific dcd/hcd driver — use the `read-doc` skill (`.claude/skills/read-doc/SKILL.md`) to cross-check against the maintainer's document library; tell the user if the needed document is missing (skill no-ops if the library is absent). Never search the library tree directly — the skill owns its location and search. +- Linux kernel behaviour (usbfs, usbtest, sysfs attributes, device locks, D state): never + infer it from symptoms — read the source for the *running* version. It refutes as often + as it confirms: it has killed two plausible dcd theories and corrected a recovery skill's + own attribute list. + ```bash + V=$(uname -r | grep -oE '^[0-9]+\.[0-9]+\.[0-9]+') # on the rig: ssh ci.lan uname -r + curl -fsSL "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/plain/drivers/usb/core/sysfs.c?h=v$V" + ``` +- Supported MCUs/boards: `hw/bsp/` and `docs/reference/boards.rst`. +- USB classes: `src/class/{cdc,hid,msc,audio,…}/` — each has `*_device.c` and `*_host.c`. +- Key files: `src/tusb.h`, `src/tusb_config.h`, `tools/get_deps.py`, `tools/build.py`, `test/unit-test/project.yml`. + +## Common Build Issues + +- Missing compiler → install `gcc-arm-none-eabi`. +- Missing deps → `python3 tools/get_deps.py FAMILY`. +- Unknown board → check `hw/bsp/FAMILY/boards/`. +- `objcopy` errors in full builds are often non-critical; retry the single example. diff --git a/README.rst b/README.rst index e5806bafc..dea352532 100644 --- a/README.rst +++ b/README.rst @@ -184,7 +184,7 @@ Supported CPUs | | MAX32 650, 666, 690, | ✅ | | ✅ | musb | 1-dir ep | | | MAX78002 | | | | | | +--------------+-----------------------------+--------+------+-----------+------------------------+--------------------+ -| Artery AT32 | F403a_407, F413 | ✅ | | | stm32_fsdev | 512 USB RAM | +| Artery AT32 | F403a_407, F413 | ✅ | | | stm32_fsdev | 768 USB RAM | | +-----------------------------+--------+------+-----------+------------------------+--------------------+ | | F415, F435_437, F423, | ✅ | ✅ | | dwc2 | | | | F425, F45x | | | | | | @@ -203,6 +203,8 @@ Supported CPUs | +-----------------------------+--------+------+-----------+------------------------+--------------------+ | | S31 | ✅ | ✅ | ✅ | dwc2 | | +--------------+-----------------------------+--------+------+-----------+------------------------+--------------------+ +| Geehy APM32 | F072 | ✅ | | ❌ | stm32_fsdev | 1KB USB RAM | ++--------------+-----------------------------+--------+------+-----------+------------------------+--------------------+ | GigaDevice | GD32VF103 | ✅ | | ❌ | dwc2 | | +--------------+-----------------------------+--------+------+-----------+------------------------+--------------------+ | HPMicro | HPM6750 | ✅ | ✅ | ✅ | ci_hs, ehci | | @@ -239,9 +241,7 @@ Supported CPUs +--------------+---------+-------------------+--------+------+-----------+------------------------+--------------------+ | NXP | iMXRT | RT 10xx, 11xx | ✅ | ✅ | ✅ | ci_hs, ehci | | | +---------+-------------------+--------+------+-----------+------------------------+--------------------+ -| | Kinetis | KL | ✅ | 🟡 | ❌ | ci_fs, khci | | -| | +-------------------+--------+------+-----------+------------------------+--------------------+ -| | | K32L2 | ✅ | | ❌ | khci | ci_fs variant | +| | Kinetis | KL, K32L | ✅ | 🟡 | ❌ | ci_fs | | | +---------+-------------------+--------+------+-----------+------------------------+--------------------+ | | LPC | 11u, 13, 15 | ✅ | ❌ | ❌ | lpc_ip3511 | | | | +-------------------+--------+------+-----------+------------------------+--------------------+ @@ -261,6 +261,8 @@ Supported CPUs | +---------+-------------------+--------+------+-----------+------------------------+--------------------+ | | RW61x | ✅ | ✅ | ✅ | ci_hs, ehci | | +--------------+-----------------------------+--------+------+-----------+------------------------+--------------------+ +| Puya | PY32F071, PY32F072 | ✅ | ❌ | ❌ | musb | 1-dir ep | ++--------------+-----------------------------+--------+------+-----------+------------------------+--------------------+ | Raspberry Pi | RP2040, RP2350 | ✅ | ✅ | ❌ | rp2040, pio_usb | | +--------------+-----+-----------------------+--------+------+-----------+------------------------+--------------------+ | Renesas | RX | 63N, 65N, 72N | ✅ | ✅ | ❌ | rusb2 | | diff --git a/docs/getting_started.rst b/docs/getting_started.rst index 7fcc2f5d1..bce028ccb 100644 --- a/docs/getting_started.rst +++ b/docs/getting_started.rst @@ -181,7 +181,7 @@ Some examples require udev permissions to access USB devices: .. code-block:: bash - $ cp `examples/device/99-tinyusb.rules <https://github.com/hathach/tinyusb/tree/master/examples/device/99-tinyusb.rules>`_ /etc/udev/rules.d/ + $ sudo cp examples/device/99-tinyusb-examples.rules /etc/udev/rules.d/ $ sudo udevadm control --reload-rules && sudo udevadm trigger Next Steps diff --git a/docs/reference/boards.rst b/docs/reference/boards.rst index 148175671..8b0f798ba 100644 --- a/docs/reference/boards.rst +++ b/docs/reference/boards.rst @@ -81,6 +81,15 @@ espressif_s3_devkitm Espresif S3 DevKitM espressif https://do espressif_saola_1 Espresif S2 Saola 1 espressif https://docs.espressif.com/projects/esp-dev-kits/en/latest/esp32s2/esp32-s2-saola-1/index.html ========================= ============================== ========= ======================================================================================================== ====== +Geehy +----- + +==================== ==================== ========= ============================================== ====== +Board Name Family URL Note +==================== ==================== ========= ============================================== ====== +apm32f072_dev_board APM32F072 Dev Board apm32f0xx +==================== ==================== ========= ============================================== ====== + GigaDevice ---------- diff --git a/docs/reference/device_issues.rst b/docs/reference/device_issues.rst index ae9cd55f1..b95a3fc1e 100644 --- a/docs/reference/device_issues.rst +++ b/docs/reference/device_issues.rst @@ -20,6 +20,51 @@ Most severe issues are: - USB.5: In USB full-speed host mode, linked list on done queue is broken. - USB.15: USB high-speed device in endpoint TX data corruption +NXP i.MX RT1015/RT1020/RT1024/RT1050/RT1060/RT1064 +----------------------------------------------------- +**Severity: High** when an isochronous IN endpoint is used behind a hub + +Reference: ERR050101 "USB: Endpoint conflict issue in device mode", listed in the errata sheet of +every part above - `IMXRT1015CE`_, `IMXRT1020CE`_, `IMXRT1024CE`_, `IMXRT1050CE`_, `IMXRT1060CE`_ +and `IMXRT1064CE`_. On RT1060 and RT1064 it applies to rev A silicon only and is fixed in rev B; on +RT1015, RT1020, RT1024 and RT1050 it is marked *no fix scheduled*, so all silicon is affected. +RT1010, RT116x, RT117x and RT118x do not list it. + +.. _IMXRT1015CE: https://www.nxp.com/docs/en/errata/IMXRT1015CE.pdf +.. _IMXRT1020CE: https://www.nxp.com/docs/en/errata/IMXRT1020CE.pdf +.. _IMXRT1024CE: https://www.nxp.com/docs/en/errata/IMXRT1024CE.pdf +.. _IMXRT1050CE: https://www.nxp.com/docs/en/errata/IMXRT1050CE.pdf +.. _IMXRT1060CE: https://www.nxp.com/docs/en/errata/IMXRT1060CE.pdf +.. _IMXRT1064CE: https://www.nxp.com/docs/en/errata/IMXRT1064CE.pdf + +While an isochronous IN endpoint is active, an IN token addressed to *that same endpoint number on +another device sharing the host* can silently unprime one of this device's OUT endpoints - control, +bulk, interrupt or isochronous alike. NXP states the unpriming cannot be detected by software and +raises no interrupt, so the endpoint simply stops answering OUT tokens and the transfer never +completes. Typically seen when the device is behind a hub with other devices attached. + +Workaround: give isochronous IN endpoints a number that no other device on the same host uses for +any IN endpoint - endpoints 1-3 are used by nearly every composite device, so choose a high number +(``examples/device/usbtest`` uses endpoint 7 on this family for that reason). Devices without an +isochronous IN endpoint are unaffected. + +NXP LPC55S2x/LPC552x +--------------------------------- +**Severity: Low** (both need specific conditions) + +Reference: `LPC55S2x Errata Sheet`_ USB.3, USB.5 + +.. _LPC55S2x Errata Sheet: https://www.nxp.com/docs/en/errata/ES_LPC55S2x.pdf + +USB.3: As a high-speed device behind certain full-speed hubs, the device does not correctly detect +the host's KJ chirp sequence and can behave erratically due to wrong speed detection. The documented +workaround is to set the FORCE_FS bit in DEVCMDSTAT on bus reset when the reported link speed is +full speed. TinyUSB does not implement this workaround. + +USB.5: An isochronous IN endpoint sending a 1024-byte maximum-packet-size packet raises no endpoint +interrupt and its command/status entry is not updated. Workaround: cap the isochronous IN maximum +packet size at 1023 bytes in the descriptor. + WCH CH32F20x/CH32V20x/CH32V30x --------------------------------- **Severity: Medium** @@ -33,3 +78,14 @@ Reference: `CH32V30X Reference Manual`_ USBFS/USBHS controller chapter Data corruption may occur on isochronous endpoints. Due to the lacking of FIFO for interrupt status registers, later completed transfer will overwrite `INT_ST` and `RX_LEN` register if previous transfer processing is not completed. Other types of transfers are not affected. + +Puya PY32F071/072 +--------------------------------- +**Severity: Very Low** + +Reference: `PY32F07x Reference Manual` USBD chapter + +The USB device controller (MUSB-like) has 5 application endpoints EP1-EP5 with fixed FIFO sizes +shared between IN and OUT of the same endpoint number: EP1 = 512 B, EP2-4 = 128 B, EP5 = 64 B. +This is much lower than the max ISO ep size of 1024 for high EP numbers. +Place large isochronous endpoints on EP1 and size descriptors accordingly. diff --git a/docs/superpowers/followup/pr3803-flasher-recover.md b/docs/superpowers/followup/pr3803-flasher-recover.md new file mode 100644 index 000000000..e9fff7480 --- /dev/null +++ b/docs/superpowers/followup/pr3803-flasher-recover.md @@ -0,0 +1,280 @@ +# `flasher_recover` Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Give the 15 HIL boards whose flasher cannot reach its probe past a poisoned usbfs +node a second, convoy-safe flasher used only for recovery. + +**Architecture:** An optional roster key `flasher_recover` beside `flasher`. +`hil_flash.recover_flasher(board)` picks it when present; `hil_test` substitutes it into the +`--recover-board` JSON so `usbtest.py` never learns a second entry exists. Delivery over +openocd's jlink driver is convoy-safe by construction, but the flash command form must +differ from the one `flash_openocd` uses, so the recovery gets its own flasher name. + +**Tech Stack:** Python 3.13 stdlib, openocd 0.12.0+dev (build 0ce743125 on ci.lan), +libjaylink, J-Link probes. + +## Global Constraints + +- Roster JSON: `test/hil/tinyusb.json`. `flasher_recover` is OPTIONAL; absent means today's + behaviour (`recover_flasher` returns the primary). +- Never change the shape of `board['flasher']` — it is read as a dict in `hil_flash`, + `hil_test`, `usbtest`, `hil_pool_check`, `hil_select` and the roster lint, and is shipped + as JSON to a subprocess. +- Flasher dispatch is by name: `getattr(hil_flash, f'flash_{name}')` / `reset_{name}`. +- `RECOVER_FLASH_TIMEOUT = 90`, `RECOVER_RESET_TIMEOUT = 30` (`usbtest.py`). Any board whose + flash cannot finish inside 90 s is not a candidate. +- Tests run offline: `cd test/hil && python3 test/test_hil_select.py`. + +## What is already established + +**Landed on PR #3803 and inert without roster entries:** `hil_flash.recover_flasher()`, +`convoy_safe()` accepting openocd-over-jlink, `hil_test` substituting the recovery flasher +into `--recover-board`, and `test_hil_select.FlasherRecoverEntry` (4 tests). + +**Verified in source:** +- openocd's jlink driver ignores `adapter usb vid_pid` — `jlink.c` never reads + `adapter_usb_get_vids/pids`; selection is `adapter serial` / USB address / usb location. + Do NOT lint a jlink recovery entry for `vid_pid`. +- It is convoy-safe anyway: libjaylink `discovery_usb.c` returns early unless + `idVendor == 0x1366` and the PID is in its table, and only THEN calls `libusb_open`. A + wedged `cafe:4010` DUT is never opened. +- CMSIS-DAP stays pin-gated: `cmsis_dap_usb_bulk.c:107` skips before `libusb_open`, and + `id_filter` is only `vids[0] || pids[0]`. + +**Measured on ci.lan 2026-08-17**, base args +`-f interface/jlink.cfg -c "transport select swd" -c "adapter speed 4000" -f target/<cfg>`: + +| Board | target cfg | flash | reset | +|--------------------------|--------------|-------|-------| +| stm32f407disco | stm32f4x | OK | OK | +| stm32f072disco | stm32f0x | OK | OK | +| stm32f723disco | stm32f7x | OK | OK | +| stm32l476disco | stm32l4x | OK | OK | +| feather_nrf52840_express | nrf52 | OK | OK | +| metro_m4_express | atsame5x | OK | OK | +| frdm_k64f | k60 | OK | OK | + +`frdm_k64f` is host-only (`tests.device == false`) — verify its reset over UART +(`/dev/serial/by-id/usb-SEGGER_J-Link_000621000000-if00`), never by USB disconnect. + +**Excluded, with reasons:** `lpcxpresso11u37` — 118 s for 24 KB at 1 MHz with a verify +mismatch, versus 0.277 s via JLinkExe; cannot fit `RECOVER_FLASH_TIMEOUT`. +`mimxrt1064_evk`, `ra4m1_ek`, `nrf54lm20dk` — no target config exists in this openocd +build, so they cannot be covered at all. **The board that wedges most (mimxrt1064_evk) is +therefore still uncovered by this work.** + +**The blocker this plan solves:** `flash_openocd` issues `program <fw> verify reset exit`, +which fails over the jlink transport on BOTH families tried (`stm32f4x`, `stm32f0x`) with +`Examination failed` → `auto_probe failed`, with or without a preceding `init; reset halt`. +Every successful flash above used the explicit sequence in Task 1. + +**Why this is a separate PR:** it adds a roster capability and a new flasher backend, which +is a different scope from containing a wedge; and it needs bench time on seven boards. + +## File Structure + +- `test/hil/hil_flash.py` — add `flash_openocd_seq` / `reset_openocd_seq`; extend + `convoy_safe` to accept the new name. This is the only file that learns the command form. +- `test/hil/tinyusb.json` — seven `flasher_recover` entries. +- `test/hil/test/test_hil_select.py` — extend `FlasherRecoverEntry`; add a roster lint. + +--- + +### Task 1: `openocd_seq` flasher backend + +**Files:** +- Modify: `test/hil/hil_flash.py` (beside `flash_openocd`, ~line 100) +- Test: `test/hil/test/test_hil_select.py` + +**Interfaces:** +- Consumes: `_openocd_cmd_base(flasher)`, `hil_util.run_cmd`. +- Produces: `flash_openocd_seq(board, firmware, timeout=None)`, + `reset_openocd_seq(board, timeout=None)`, both returning + `subprocess.CompletedProcess`; `convoy_safe()` returns True for + `{'name': 'openocd_seq', 'args': '...interface/jlink.cfg...'}`. + +- [ ] **Step 1: Write the failing test** + +```python + def test_openocd_seq_is_convoy_safe_over_jlink(self): + self.assertTrue(hil_flash.convoy_safe( + {'name': 'openocd_seq', 'args': '-f interface/jlink.cfg -f target/stm32f4x.cfg'})) + + def test_openocd_seq_uses_explicit_flash_commands_not_program(self): + """`program` fails over the jlink transport: Examination failed -> auto_probe + failed, measured on stm32f4x and stm32f0x.""" + seen = {} + real = hil_util.run_cmd + hil_util.run_cmd = lambda cmd, **k: seen.setdefault('cmd', cmd) or real('true') + try: + hil_flash.flash_openocd_seq( + {'flasher': {'name': 'openocd_seq', 'uid': 'X', 'args': '-f interface/jlink.cfg'}}, + '/tmp/fw.elf', timeout=5) + finally: + hil_util.run_cmd = real + self.assertIn('flash write_image erase /tmp/fw.elf', seen['cmd']) + self.assertIn('verify_image /tmp/fw.elf', seen['cmd']) + self.assertNotIn('program ', seen['cmd']) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd test/hil && python3 test/test_hil_select.py FlasherRecoverEntry -v` +Expected: FAIL — `module 'hil_flash' has no attribute 'flash_openocd_seq'` + +- [ ] **Step 3: Write minimal implementation** + +```python +def flash_openocd_seq(board, firmware, timeout=None): + # Explicit commands, NOT `program`: over the jlink transport `program` fails at the + # flash bank probe ("Examination failed" -> "auto_probe failed"), measured on + # stm32f4x and stm32f0x, with or without a preceding reset halt. This sequence + # succeeded on all seven candidate boards. + flasher = board['flasher'] + verify = f' -c "verify_image {firmware}"' if flasher.get('verify', True) else '' + return hil_util.run_cmd( + f'{_openocd_cmd_base(flasher)} -c "init" -c "reset halt" ' + f'-c "flash write_image erase {firmware}"{verify} -c "reset run" -c "shutdown"', + timeout=timeout) + + +def reset_openocd_seq(board, timeout=None): + flasher = board['flasher'] + return hil_util.run_cmd( + f'{_openocd_cmd_base(flasher)} -c "init" -c "reset run" -c "shutdown"', + timeout=timeout) +``` + +In `convoy_safe`, replace `if name != 'openocd':` with: + +```python + if name not in ('openocd', 'openocd_seq'): + return False +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd test/hil && python3 test/test_hil_select.py FlasherRecoverEntry -v` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add test/hil/hil_flash.py test/hil/test/test_hil_select.py +git commit -m "hil: add openocd_seq flasher for convoy-safe recovery delivery" +``` + +--- + +### Task 2: Roster entries for the seven validated boards + +**Files:** +- Modify: `test/hil/tinyusb.json` +- Test: `test/hil/test/test_hil_select.py` + +**Interfaces:** +- Consumes: `flash_openocd_seq` / `reset_openocd_seq` from Task 1. +- Produces: seven boards for which `hil_flash.convoy_safe(hil_flash.recover_flasher(b))` + is True. + +- [ ] **Step 1: Write the failing test** + +```python + def test_roster_recover_entries_are_convoy_safe_and_named_openocd_seq(self): + import json, pathlib + roster = json.loads((pathlib.Path(__file__).parent.parent / 'tinyusb.json').read_text()) + recover = [b for b in roster['boards'] if 'flasher_recover' in b] + self.assertGreaterEqual(len(recover), 7) + for b in recover: + f = b['flasher_recover'] + self.assertEqual(f['name'], 'openocd_seq', b['name']) + self.assertIn('interface/jlink.cfg', f['args'], b['name']) + self.assertIn('adapter speed', f['args'], b['name']) # required; see below + self.assertTrue(hil_flash.convoy_safe(f), b['name']) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd test/hil && python3 test/test_hil_select.py FlasherRecoverEntry -v` +Expected: FAIL — `0 >= 7` + +- [ ] **Step 3: Add the entries** + +`adapter speed` is REQUIRED: without it examination fails outright on the jlink driver. +Add to each board below, using the SAME `uid` as its primary jlink entry: + +```json +"flasher_recover": { + "name": "openocd_seq", + "uid": "<same probe serial as flasher.uid>", + "args": "-f interface/jlink.cfg -c \"transport select swd\" -c \"adapter speed 4000\" -f target/<cfg>.cfg" +} +``` + +| Board | `uid` | `<cfg>` | +|--------------------------|----------------|-----------| +| stm32f407disco | 000773661813 | stm32f4x | +| stm32f072disco | 779541626 | stm32f0x | +| stm32f723disco | 000776606156 | stm32f7x | +| stm32l476disco | 777632258 | stm32l4x | +| feather_nrf52840_express | 681295394 | nrf52 | +| metro_m4_express | 123456 | atsame5x | +| frdm_k64f | 000621000000 | k60 | + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd test/hil && python3 test/test_hil_select.py -v` +Expected: PASS, and no other selector test regresses. + +- [ ] **Step 5: Commit** + +```bash +git add test/hil/tinyusb.json test/hil/test/test_hil_select.py +git commit -m "hil: give seven J-Link boards a convoy-safe recovery flasher" +``` + +--- + +### Task 3: Bench validation on the rig + +**Files:** none — this task produces evidence, not code. + +- [ ] **Step 1: Confirm the rig is idle and take the locks** + +```bash +ssh [email protected] 'if pgrep -f "[h]il_test.py" >/dev/null; then echo BUSY; exit 1; fi' +ssh [email protected] 'cd ~/actions-runner/_work/tinyusb/tinyusb && \ + nohup timeout 900 python3 test/hil/helper/hil_lock.py hold <boards...> --reason "flasher_recover validation" &' +``` + +Guard with `if`, never `cmd && echo || echo` — that form only gates the echo and will take +locks during a live CI run. + +- [ ] **Step 2: For each board, flash then reset through the recovery entry** + +```bash +python3 test/hil/hil_test.py -b <board> test/hil/tinyusb.json # normal path still works +``` + +Then force the recovery path by running usbtest with the recovery flags and a firmware that +hangs a case, or drive `hil_flash.flash_openocd_seq` / `reset_openocd_seq` directly. + +- [ ] **Step 3: Verify** + +Device boards: `sudo dmesg` shows `USB disconnect` then a fresh enumeration. +`frdm_k64f`: UART shows the boot banner (see above). +Every flash must finish well inside `RECOVER_FLASH_TIMEOUT` (90 s). + +- [ ] **Step 4: Release locks and record the results in the PR body** + +--- + +## Out of scope, and why + +- **`mimxrt1064_evk`** needs an i.MX RT target config that this openocd build does not + have. Sourcing or writing one is its own investigation; until then the board with the + most wedges has no automated recovery. +- **Changing `flash_openocd`** to the explicit form would cover these boards without a new + name, but `program` is what nine pinned CMSIS-DAP boards use in CI daily and no CMSIS-DAP + image could be built in the originating worktree (no pico-sdk) to re-validate it. diff --git a/docs/superpowers/followup/pr3803-hil-blindness-reporting.md b/docs/superpowers/followup/pr3803-hil-blindness-reporting.md new file mode 100644 index 000000000..69ff939b0 --- /dev/null +++ b/docs/superpowers/followup/pr3803-hil-blindness-reporting.md @@ -0,0 +1,185 @@ +# Blindness Reporting Gaps Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make a HIL worker's sysfs blindness reach the report in the two cases where it +currently does not — an untested producer, and a board that raises. + +**Architecture:** A worker returns `hil_util.sysfs_blind()` as the last field of its result +tuple; `_blind_note()` turns that into a report banner. Two holes: nothing tests the +producer, and a board that raises returns no tuple at all, so its blindness is lost. + +**Tech Stack:** Python 3.13 stdlib, multiprocessing Pool with `maxtasksperchild=1`. + +## Global Constraints + +- A blind worker answers `SYSFS_UNKNOWN` for every attribute, so its "device not found" + means "could not tell". The report must say so or a red cell reads as a broken board. +- `maxtasksperchild=1`: one worker per board, so the flag is per-board and must not be + smeared across boards. +- Tests: `cd test/hil && python3 test/test_hil_bounded.py`. + +## What is already established + +- `hil_test.test_board` returns `(..., hil_util.sysfs_blind(), stray)`; `_blind_note(mret)` + renders the banner; wired into all three report paths. +- **The producer is provably untested**: replacing `hil_util.sysfs_blind()` with `False` in + the return leaves all tests green. Nothing drives `test_board` — it needs a board dict, a + real flock, a flasher and `test_example` per test. +- Blindness fired for real on ci.lan: four workers went blind in one run, and cells failed + *because* of it (`Printer device not found ... (this worker is blind)`). + +**Why this is a separate PR:** closing it means making `test_board` testable, which is a +refactor of the harness's orchestration layer — a different scope from the containment +work, and the reason the gap was accepted rather than papered over. + +## File Structure + +- `test/hil/hil_test.py` — extract the result-tuple assembly from `test_board` so it can be + built and asserted without running a board; carry blindness out of the raise path. +- `test/hil/test/test_hil_bounded.py` — tests for both. + +--- + +### Task 1: Make the result tuple assembly testable + +**Files:** +- Modify: `test/hil/hil_test.py` (`test_board`, the `return (name, err_count, ...)` at the + end of the try block) +- Test: `test/hil/test/test_hil_bounded.py` + +**Interfaces:** +- Produces: `_board_result(name, err_count, failed_tests, rows, t_total, board_wide_fail)` + returning the 7-tuple `(name, err_count, failed, rows, t_total, blind, stray)`, reading + `hil_util.sysfs_blind()` and `hil_health.kill_own_children()` itself. + +- [ ] **Step 1: Write the failing test** + +```python +class BoardResultCarriesBlindness(unittest.TestCase): + def test_a_blind_worker_reports_it(self): + from helper import hil_util, hil_health + self.addCleanup(setattr, hil_util, 'sysfs_blind', hil_util.sysfs_blind) + self.addCleanup(setattr, hil_health, 'kill_own_children', hil_health.kill_own_children) + hil_util.sysfs_blind = lambda: True + hil_health.kill_own_children = lambda: 0 + row = hil_test._board_result('b', 0, [], [], 1.0, False) + self.assertTrue(row[5], 'blindness did not reach the result tuple') + self.assertIn('b', hil_test._blind_note([row])) + + def test_a_sighted_worker_does_not(self): + from helper import hil_util, hil_health + self.addCleanup(setattr, hil_util, 'sysfs_blind', hil_util.sysfs_blind) + self.addCleanup(setattr, hil_health, 'kill_own_children', hil_health.kill_own_children) + hil_util.sysfs_blind = lambda: False + hil_health.kill_own_children = lambda: 0 + row = hil_test._board_result('b', 0, [], [], 1.0, False) + self.assertFalse(row[5]) + self.assertEqual(hil_test._blind_note([row]), '') +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd test/hil && python3 test/test_hil_bounded.py BoardResultCarriesBlindness -v` +Expected: FAIL — `module 'hil_test' has no attribute '_board_result'` + +- [ ] **Step 3: Write minimal implementation** + +```python +def _board_result(name, err_count, failed_tests, rows, t_total, board_wide_fail): + """Assemble a worker's result tuple. Separate from test_board so the two fields only + the WORKER can answer -- its process-global blindness latch and what it could not kill + -- are testable without running a board.""" + stray = hil_health.kill_own_children() + return (name, err_count, [] if board_wide_fail else sorted(set(failed_tests)), + rows, t_total, hil_util.sysfs_blind(), stray) +``` + +Replace the tail of `test_board` with: + +```python + return _board_result(name, err_count, failed_tests, rows, t_total, board_wide_fail) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd test/hil && python3 test/test_hil_bounded.py -v` +Expected: PASS, and the existing `BlindWorkerReachesTheReport` tests still pass. + +- [ ] **Step 5: Verify the mutation is now caught** + +Replace `hil_util.sysfs_blind()` with `False` inside `_board_result` and re-run; the suite +MUST fail. Restore it. + +- [ ] **Step 6: Commit** + +```bash +git add test/hil/hil_test.py test/hil/test/test_hil_bounded.py +git commit -m "test/hil: make the worker result tuple testable, covering blindness" +``` + +--- + +### Task 2: Carry blindness out of the worker-raise path + +**Files:** +- Modify: `test/hil/hil_test.py` (`test_board`'s except/finally, and `main`'s worker-raise + handler that builds synthetic rows) +- Test: `test/hil/test/test_hil_bounded.py` + +**Interfaces:** +- Consumes: `_board_result` from Task 1. +- Produces: a board that raises still contributes a row whose blindness field is accurate. + +- [ ] **Step 1: Write the failing test** + +```python + def test_a_board_that_raises_still_reports_blindness(self): + """The result tuple is returned inside a try whose finally only releases the lock, + so a board that dies by exception contributed nothing -- and its blindness, the + thing that most explains its failure, was lost with it.""" + from helper import hil_util + self.addCleanup(setattr, hil_util, 'sysfs_blind', hil_util.sysfs_blind) + hil_util.sysfs_blind = lambda: True + row = hil_test._board_result_on_error('b', RuntimeError('boom')) + self.assertTrue(row[5]) + self.assertIn('b', hil_test._blind_note([row])) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd test/hil && python3 test/test_hil_bounded.py BoardResultCarriesBlindness -v` +Expected: FAIL — no `_board_result_on_error` + +- [ ] **Step 3: Write minimal implementation** + +```python +def _board_result_on_error(name, exc): + """A row for a board that died by exception. err_count 1, no per-test detail, but the + blindness and stray fields are still accurate -- they explain the failure more often + than the exception text does.""" + rows = [(name, {BOUNDARY_CELL: f'{REPORT_CELL["fail"]} {type(exc).__name__}'}, None)] + return _board_result(name, 1, [], rows, 0.0, True) +``` + +Wrap the body of `test_board` so the exception path returns it instead of propagating. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd test/hil && python3 test/test_hil_bounded.py -v` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add test/hil/hil_test.py test/hil/test/test_hil_bounded.py +git commit -m "test/hil: keep a raising board's blindness in the report" +``` + +--- + +## Caution + +`test_board`'s `finally` releases the board flock. Any restructuring MUST keep that +release on every path, including the new error path — a leaked flock locks the board until +the host reboots. diff --git a/docs/superpowers/followup/pr3803-hil-iar-rerun-spec.md b/docs/superpowers/followup/pr3803-hil-iar-rerun-spec.md new file mode 100644 index 000000000..fe377f741 --- /dev/null +++ b/docs/superpowers/followup/pr3803-hil-iar-rerun-spec.md @@ -0,0 +1,118 @@ +# IAR HIL Leg Re-run Spec Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Let the `hil-hfp-iar` CI leg re-run only its failed boards, as the other two HIL +legs already do. + +**Architecture:** `hil_test.py` writes a `<config>.failed` spec into `HIL_REPORT_DIR`; a +workflow step reads it on the next attempt and passes the boards back as arguments. The IAR +leg passes `--retry 1` like the others but sets no `HIL_REPORT_DIR` and has no read-back +step, so its spec is written into the workspace and never read. + +**Tech Stack:** GitHub Actions YAML, self-hosted runner. + +## Global Constraints + +- `.github/workflows/build.yml`. The two working legs are `hil-tinyusb` (matrix) — see its + `Set HIL report dir (per run+job; persists across run attempts)` and `Get re-run spec from + previous attempt` steps — and they are the pattern to copy. +- The report dir must be keyed by run id AND job so a matrix leg does not collide with + another, and must survive across run attempts (that is the whole point). +- The IAR leg is the only HIL job that BUILDS inline; its `Build` step is bounded at + `timeout-minutes: 30` under a 120-minute job ceiling. Do not disturb that. + +## What is already established + +- Verified by reading the workflow: `hil-hfp-iar` has neither `HIL_REPORT_DIR` nor a + `Get re-run spec` step, while passing `--retry 1`. +- Consequence: a GitHub re-run of that job re-tests its whole matrix. **This is not a + regression** — that leg never had the mechanism — and the unread spec costs only a file. +- The report artifact upload for that leg is named `hil-report-hfp-iar`. + +**Why this is a separate PR:** it is CI plumbing with no code change, it needs a real +re-run on the self-hosted runner to prove, and it duplicates ~15 lines of workflow that +would be better factored — a decision worth making on its own. + +## File Structure + +- `.github/workflows/build.yml` — the `hil-hfp-iar` job only. + +--- + +### Task 1: Give the IAR leg a persistent report dir and a re-run spec + +**Files:** +- Modify: `.github/workflows/build.yml` (job `hil-hfp-iar`) + +**Interfaces:** +- Consumes: `hil_test.py`'s existing `--report-dir` / `.failed` behaviour — no code change. +- Produces: `env.HIL_REPORT_DIR` for the job, and `$RERUN_ARGS` for the test step. + +- [ ] **Step 1: Copy the two steps from `hil-tinyusb`, before the Build step** + +```yaml + - name: Set HIL report dir (per run+job; persists across run attempts) + run: | + BASE=$HOME/hil-reports + echo "HIL_REPORT_DIR=$BASE/${GITHUB_RUN_ID}-hfp-iar" >> "$GITHUB_ENV" + + - name: Get re-run spec from previous attempt + run: | + SPEC="$HIL_REPORT_DIR/hfp.json.failed" + if [ -f "$SPEC" ]; then + echo "RERUN_ARGS=$(cat "$SPEC")" >> "$GITHUB_ENV" + echo "re-running only: $(cat "$SPEC")" + fi +``` + +Match the exact spec filename `hil_test.py` writes for this leg's config — read +`_write_failed_spec` and the `failed_fname` construction rather than assuming. + +- [ ] **Step 2: Pass the spec to the test step** + +```yaml + python3 test/hil/hil_test.py --retry 1 $SEL_ARGS hfp.json $RERUN_ARGS +``` + +`--retry 1` stays FIRST so argparse's last-wins keeps any explicit override working. + +- [ ] **Step 3: Point the artifact upload at the report dir** + +```yaml + path: ${{ env.HIL_REPORT_DIR }}/hil_report.md +``` + +- [ ] **Step 4: Validate the YAML** + +Run: `python3 -c "import yaml,sys; d=yaml.safe_load(open('.github/workflows/build.yml')); j=d['jobs']['hil-hfp-iar']; print(j['timeout-minutes'], [s.get('name') for s in j['steps']])"` +Expected: the ceiling is still 120, the Build step still carries `timeout-minutes: 30`, and +the two new steps appear before Build. + +- [ ] **Step 5: Commit** + +```bash +git add .github/workflows/build.yml +git commit -m "ci: let the IAR HIL leg re-run only its failed boards" +``` + +--- + +### Task 2: Prove it on a real re-run + +**Files:** none — evidence only. + +- [ ] **Step 1:** Push and let `hil-hfp-iar` run to a failure (or force one). +- [ ] **Step 2:** Confirm `$HIL_REPORT_DIR/hfp.json.failed` exists on the runner after the + job. +- [ ] **Step 3:** Use GitHub's "Re-run failed jobs" and confirm the log line + `re-running only: ...` and that only those boards are tested. +- [ ] **Step 4:** Record the run URL in the PR body. + +--- + +## Consider first + +Three jobs would then carry the same ~15 lines. Factoring them into a composite action, or +computing the report dir inside `hil_test.py` from `GITHUB_RUN_ID`, may be the better +change — decide that before copying the block a third time. diff --git a/docs/superpowers/followup/pr3803-pci-rebind-stranding.md b/docs/superpowers/followup/pr3803-pci-rebind-stranding.md new file mode 100644 index 000000000..de1f7163b --- /dev/null +++ b/docs/superpowers/followup/pr3803-pci-rebind-stranding.md @@ -0,0 +1,157 @@ +# `pci-rebind` Stranding Investigation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Settle when a PCI unbind/rebind of an xHCI controller strands it driverless, so +the `usb-kernel-recover` skill can state a rule instead of a hypothesis. + +**Architecture:** No product code. This is a controlled reproduction against the rig's +kernel, ending in a documentation change and — if the boundary turns out to be +detectable — a guard in `usb_recover.sh`. + +**Tech Stack:** Linux 6.12.96 (ci.lan), Renesas uPD720201 xHCI, `usb_recover.sh`. + +## Global Constraints + +- ci.lan is a live CI rig. Take every affected board's lock first + (`hil_lock.py hold --all --reason ...`) and confirm no `hil_test.py` is running, with an + `if`, not an `&&` chain. +- A stranded controller takes every fixture on it offline; recovery is + `usb_recover.sh pci-bind <addr>` or, failing that, a PVE **host** power cycle — an + operator action. Do not start this without being able to reach the host. +- The rig has two Renesas controllers plus an AMD one; pick the controller with the fewest + fixtures for the experiment. + +## What is already established + +**The skill claimed, unconditionally, that `pci-rebind`'s re-bind hangs on the D-state URB +and leaves the controller with no driver.** That claim was generalised from ONE observation +and was used to delete `pci-rebind` and `pci-bind` from `usb_recover.sh` entirely. + +**It was refuted in the field on 2026-08-17.** After `hub-cycle 17-2.7` failed to clear a +wedge, `pci-rebind 0000:05:00.0` recovered the controller in about one second: + +``` +02:34:41 remove, state 4 / USB bus 18 deregistered +02:34:41 remove, state 1 / USB bus 17 deregistered +02:34:42 xHCI Host Controller / new USB bus registered, assigned bus number 1 +02:34:42 new USB bus registered, assigned bus number 2 +``` + +Both actions were restored, with the guidance scoped to failure mode: **dead controller → +use it; device-lock convoy → do not**. Buses renumbered 17/18 → 1/2, which is why rig-wide +operations need every board's lock. + +**What is NOT known:** why the earlier attempt stranded and this one did not. The leading +hypothesis is that it turns on whether a live D-state URB exists **on that controller** at +the moment of the re-bind — but in the 02:34 incident the wedged board (17-2.7) was on that +very controller, which weakens it. An alternative is that `hub-cycle` had already cleared +the holder, leaving only a dead controller. + +**Why this is a separate PR:** it is an experiment that risks taking the rig offline, and +its output is a documentation change plus possibly a guard — a different scope from any +code change. + +## File Structure + +- `.claude/skills/usb-kernel-recover/SKILL.md` — replace the hypothesis in section 3b and + the Common-mistakes entry with whatever the experiment establishes. +- `.claude/skills/usb-kernel-recover/scripts/usb_recover.sh` — only if the boundary is + detectable from userspace. + +--- + +### Task 1: Reproduce a controller-scoped D-state wedge + +**Files:** none. + +- [ ] **Step 1: Establish the safety net** + +```bash +ssh [email protected] 'if pgrep -f "[h]il_test.py" >/dev/null; then echo BUSY; exit 1; fi' +# hold ALL boards on the target controller +``` + +Confirm host access to pve.lan before continuing. + +- [ ] **Step 2: Create a wedge deliberately** + +Run `usbtest.py` against a board known to hang (`mimxrt1064_evk` has wedged eight times, +TEST 9/10/24/27), or drive `testusb` directly until a case does not return. + +- [ ] **Step 3: Confirm the holder and its controller** + +```bash +ps -eo pid,stat,etimes,wchan:22,args | awk '$2 ~ /D/' +sudo cat /proc/<pid>/stack # usbdev_ioctl + [usbtest] = the owner +readlink -f /sys/bus/usb/devices/usb<N> # bus -> PCI addr +``` + +Record whether the holder is on the SAME controller you will rebind. + +--- + +### Task 2: Rebind and record the outcome + +**Files:** none. + +- [ ] **Step 1: Rebind, with a bounded observer** + +```bash +timeout 120 sudo usb_recover.sh pci-rebind <addr>; echo "rc=$?" +``` + +- [ ] **Step 2: Record which of the three outcomes occurred** + +1. Re-bind completes, controller recovers (as on 2026-08-17). +2. Re-bind hangs; `/sys/bus/pci/devices/<addr>/driver` is gone → **stranded**. +3. Re-bind completes but the wedge persists. + +Capture `sudo journalctl -k --since ...` around the attempt either way. + +- [ ] **Step 3: If stranded, recover** + +```bash +sudo usb_recover.sh pci-bind <addr> +``` + +If that hangs too, the only remaining step is a PVE host power cycle — an operator action. + +- [ ] **Step 4: Repeat at least three times** + +One observation is what produced the wrong rule in the first place. Vary whether a D-state +holder is live on that controller at rebind time; that is the hypothesis under test. + +--- + +### Task 3: Write down what was learned + +**Files:** +- Modify: `.claude/skills/usb-kernel-recover/SKILL.md` + +- [ ] **Step 1: Replace section 3b's scoping with the measured rule** + +State the condition under which stranding occurs, with the journal lines. If the experiment +does NOT reproduce stranding, say that too, with the attempt count — "not reproduced in N +attempts" is a better record than an unexplained warning. + +- [ ] **Step 2: If the boundary is detectable, guard the script** + +For example, refuse `pci-rebind` when a D-state holder exists on that controller, since the +holder is enumerable from `/proc` and the controller from `readlink`. Only add this if the +experiment shows it predicts the outcome. + +- [ ] **Step 3: Commit** + +```bash +git add .claude/skills/usb-kernel-recover/ +git commit -m "skills: replace the pci-rebind stranding hypothesis with measurement" +``` + +--- + +## Abort criteria + +Stop and hand back to the operator if: a rebind strands the controller and `pci-bind` does +not recover it; `uhubctl` starts hanging (the convoy has spread to the hub locks); or a CI +run starts while the rig is in a broken state. diff --git a/docs/superpowers/followup/pr3803-usbtest-recovery-reserve.md b/docs/superpowers/followup/pr3803-usbtest-recovery-reserve.md new file mode 100644 index 000000000..eb8959520 --- /dev/null +++ b/docs/superpowers/followup/pr3803-usbtest-recovery-reserve.md @@ -0,0 +1,175 @@ +# usbtest Recovery Reserve Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make the post-hang recovery reserve a derived, asserted property instead of an +accident of four independently-set constants. + +**Architecture:** `hil_test` passes `--budget` and `--outer-timeout` to `usbtest.py`, which +decides at runtime whether a recovery still fits. Today the reserve survives only because +the four numbers happen to line up; nothing ties them together or fails when they stop. + +**Tech Stack:** Python 3.13 stdlib. + +## Global Constraints + +- `usbtest.py`: `RECOVER_FLASH_TIMEOUT = 90`, `RECOVER_RESET_TIMEOUT = 30`. +- `hil_test.py`: `USBTEST_BATTERY_BUDGET = 260`, `USBTEST_RECOVERY_BUDGET = 250`, + `USBTEST_OVERSHOOT = 120`; `outer = BATTERY_BUDGET + (RECOVERY_BUDGET if recovery else + OVERSHOOT)`, used for both the child's `--outer-timeout` and the parent's `run_cmd` bound. +- All five are env-overridable via `hil_util.pos_int_env`, so a rig can change them. +- Tests: `cd test/hil && python3 test/test_hil_health.py` and `test_hil_bounded.py`. + +## What is already established + +The reserve holds at the shipped values, checked by hand: + +- The battery checks its budget BEFORE dispatching a case, so it can overshoot by one + case — worst case `260 + 60 + 5 = 325 s`. +- Recovery is gated on `_time_left() >= RECOVER_RESET_TIMEOUT`, where + `_time_left() = outer_timeout - elapsed - 35`; with `outer = 510` that allows recovery + until `elapsed = 445 s`, and the reflash until `385 s`. +- So ~60 s of margin survives, and recovery does fire. + +**The defect is structural, not arithmetic:** lower `--outer-timeout`, raise `--timeout`, or +raise `USBTEST_BATTERY_BUDGET` via the env and the reserve silently disappears. The failure +mode is a skipped reflash that leaves the D-state holder for the next job — the exact thing +the containment exists to prevent — with no error anywhere. + +**Why this is a separate PR:** it changes the timing contract between `hil_test` and +`usbtest.py`, which affects every board's run duration, so it wants its own review and a +full rig run. + +## File Structure + +- `test/hil/usbtest.py` — a `reserve_ok()` predicate plus a startup assertion. +- `test/hil/hil_test.py` — derive the battery budget from the outer bound rather than + setting both independently. +- `test/hil/test/test_hil_health.py` — tests. + +--- + +### Task 1: Assert the reserve at startup + +**Files:** +- Modify: `test/hil/usbtest.py` (constants block, and `main()` after argparse) +- Test: `test/hil/test/test_hil_health.py` + +**Interfaces:** +- Produces: `usbtest.reserve_ok(budget, outer, case_timeout)` returning bool. + +- [ ] **Step 1: Write the failing test** + +```python +class RecoveryReserveIsChecked(unittest.TestCase): + """The battery may overshoot its budget by ONE already-started case, so the outer bound + must leave room for that overshoot AND a bounded recovery afterwards.""" + + def setUp(self): + import usbtest + self.u = usbtest + + def test_the_shipped_numbers_leave_room(self): + self.assertTrue(self.u.reserve_ok(budget=260, outer=510, case_timeout=60)) + + def test_a_tighter_outer_bound_is_rejected(self): + self.assertFalse(self.u.reserve_ok(budget=260, outer=380, case_timeout=60)) + + def test_a_longer_case_timeout_is_rejected(self): + self.assertFalse(self.u.reserve_ok(budget=260, outer=510, case_timeout=200)) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd test/hil && python3 test/test_hil_health.py RecoveryReserveIsChecked -v` +Expected: FAIL — `module 'usbtest' has no attribute 'reserve_ok'` + +- [ ] **Step 3: Write minimal implementation** + +```python +def reserve_ok(budget: int, outer: int, case_timeout: int) -> bool: + """Does `outer` leave room for the battery's worst case AND a bounded recovery? + + The budget is checked BEFORE dispatch, so the battery can run to + `budget + case_timeout + 5` (the +5 is run_case's reap). _time_left() subtracts a + further 35 s of fixed tail. A reflash needs RECOVER_FLASH_TIMEOUT beyond that. + """ + worst_case_end = budget + case_timeout + 5 + return outer - worst_case_end - 35 >= RECOVER_FLASH_TIMEOUT +``` + +In `main()`, after parsing args: + +```python + if args.budget and args.outer_timeout and not reserve_ok( + args.budget, args.outer_timeout, args.timeout): + print(f'warning: --outer-timeout {args.outer_timeout} leaves no room for a bounded ' + f'recovery after a --budget {args.budget} battery with --timeout ' + f'{args.timeout} cases; a HUNG board will be left wedged', file=sys.stderr) +``` + +Warn, do not exit: a caller that deliberately runs without recovery is legitimate. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd test/hil && python3 test/test_hil_health.py RecoveryReserveIsChecked -v` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add test/hil/usbtest.py test/hil/test/test_hil_health.py +git commit -m "usbtest: check the recovery reserve instead of assuming it" +``` + +--- + +### Task 2: Derive the outer bound from one place + +**Files:** +- Modify: `test/hil/hil_test.py` (constants block ~line 227, and `test_device_usbtest`) +- Test: `test/hil/test/test_hil_bounded.py` + +**Interfaces:** +- Consumes: `usbtest.reserve_ok` semantics (duplicate the arithmetic, do not import + usbtest — `hil_test` must not import it). +- Produces: an assertion at module import that the shipped constants satisfy the reserve. + +- [ ] **Step 1: Write the failing test** + +```python + def test_the_shipped_constants_satisfy_the_reserve(self): + """Whatever the env overrides, the pair hil_test computes must leave recovery room: + outer - (budget + case_timeout + 5) - 35 >= 90.""" + outer = hil_test.USBTEST_BATTERY_BUDGET + hil_test.USBTEST_RECOVERY_BUDGET + self.assertGreaterEqual(outer - (hil_test.USBTEST_BATTERY_BUDGET + 60 + 5) - 35, 90) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Temporarily set `HIL_USBTEST_RECOVERY_BUDGET=100` and run; expect FAIL. Unset. + +- [ ] **Step 3: Add the guard** + +```python +# The recovery reserve is a PROPERTY of these two, not a coincidence: the battery may +# overshoot its budget by one already-started case (checked before dispatch), and a bounded +# reflash needs 90 s after a 35 s fixed tail. Env overrides make this checkable at import +# rather than discoverable when a wedge is left unrecovered. +if USBTEST_RECOVERY_BUDGET - 60 - 5 - 35 < 90: + print(f'warning: HIL_USBTEST_RECOVERY_BUDGET={USBTEST_RECOVERY_BUDGET} leaves no room ' + f'for a bounded reflash after a one-case overshoot; HUNG boards will stay wedged', + file=sys.stderr) +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cd test/hil && python3 test/test_hil_bounded.py -v` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add test/hil/hil_test.py test/hil/test/test_hil_bounded.py +git commit -m "hil: warn when the timeout constants leave no recovery reserve" +``` diff --git a/docs/superpowers/plans/2026-07-14-usb-target-debug-handoff.md b/docs/superpowers/plans/2026-07-14-usb-target-debug-handoff.md new file mode 100644 index 000000000..b56d035d9 --- /dev/null +++ b/docs/superpowers/plans/2026-07-14-usb-target-debug-handoff.md @@ -0,0 +1,125 @@ +# Hand-off: `usb-target-debug` skill + `target-debugger` agent + +**Status: agreed but NOT started.** Design discussion happened 2026-07-13 in session +`c31a4617-43b1-491d-9865-3e35f393996b` (post-merge of the agents/workflows harness, +PR #3762 / `ac595bc5c`). This document is the implementation brief for a fresh session. + +**Agreed sequencing: skill first → dogfood on 1-2 real HIL failures → then the agent +as its own small PR.** Do not build both at once — the agent charter's hard parts are +exactly what dogfooding the skill answers. + +## The gap being filled + +When HIL fails today, *what failed* is covered (hil-validate workflow, hil-operator +agent) but the deep *why* loop — instrument the target, capture on both sides, +correlate — has no skill and no agent. Every hard case so far (musb babble, rusb2 +FRDY wedge, ch32v307 Heisenbug) fell back to interactive main-session work. + +Why no existing agent can do it: + +- **hil-operator** (sonnet) is deliberately mechanical: lock → flash → `hil_test.py` + → recover. It never edits source, so it cannot inject instrumentation. +- **port-dev** can edit source but its charter is scoped changes verified by a + *build*; it has no hardware mandate. +- The host-side capture knowledge lives in skills (`usbmon`, `usb-debug`); the + device-side half exists only as CLAUDE.md recipes plus session memory. + +The skill completes the debugging trio: + +| Skill | Answers | Status | +|---|---|---| +| `usbmon` | what the host actually exchanged (URBs) | on master | +| `usb-debug` | why the host acted (dmesg / dynamic debug) | ships in PR #3758 (untracked copy in tree) | +| `usb-target-debug` | what the device did | **this hand-off** | + +## Part 1 — `usb-target-debug` skill (do this first) + +Create `.claude/skills/usb-target-debug/SKILL.md`. Match the style of +`.claude/skills/usbmon/SKILL.md` and `usb-debug/SKILL.md`: frontmatter `name` + +`description` where the description states concretely *when* to reach for it +(HIL test fails and host-side capture can't explain it; device silently NAKs, +wedges, or misbehaves; need TU_LOG/device-state evidence from real hardware). + +Playbook to codify — all techniques already proven on this rig: + +1. **TU_LOG capture** — build with `LOG=2` (add `LOGGER=rtt` for RTT); UART capture + from the board's debug serial; RTT via `JLinkGDBServer -RTTTelnetPort 19021` + + `JLinkRTTClient` (non-interactive: `timeout 20s JLinkRTTClient > rtt.log`). + Note which log level perturbs timing (see warning #6). +2. **GDB recipes per probe family** — J-Link, OpenOCD (ST-Link / CMSIS-DAP / + WCH-Link). Base connect/load recipes already exist in CLAUDE.md "GDB Debugging"; + the skill adds the debug-loop specifics: breakpoints in ISR context, dumping + endpoint/FIFO registers, watchpoints on driver state variables. +3. **RAM ring-buffer trace pattern** (used to crack the musb babble): instrument + the dcd/hcd with a small RAM ring of event records instead of TU_LOG when + printing perturbs timing; let the failure happen; halt and dump the ring via + GDB. Include a minimal C snippet (fixed-size struct ring, no allocation, + ISR-safe single-writer). +4. **J-Link PC-sampling** (nailed the rusb2 FRDY wedge): statistically sample PC + without halting to find where the core spins — the non-intrusive option when + halting or logging masks the bug. +5. **Dual-side capture**: usbmon on the host + RTT/ring-buffer on the target, + simultaneously; correlate host URBs against device events on one timeline. + This is the default posture for enumeration/transfer bugs, not an escalation. +6. **Warnings**: observation can mask the bug (the ch32v307 case changed behavior + under logging/debug — prefer ring-buffer over TU_LOG, PC-sampling over halting, + and say so explicitly); a J-Link core reset does NOT drop a DWC2 soft-connect + pullup, so a wedged DUT stays wedged on the host side (cross-ref + `usb-recover/SKILL.md`). +7. **Rig discipline**: hold the board lock for the whole manual session — + `python3 test/hil/board_lock.py hold <board> --reason "target debug: <bug>"` + … work … `release <board>`. Never stop the actions-runner. Board → probe + mapping via `test/hil/tinyusb.json`; `JLINK_DEVICE`/`OPENOCD_OPTION` via + `hw/bsp/*/boards/*/board.cmake` or `board.mk`. + +**Where to ship**: its own small PR (usb-recover/usb-debug already belong to +PR #3758 — don't grow that one), or fold into #3758 if it is still open and being +rebased anyway. User's call at the time. + +## Part 2 — `target-debugger` agent (later, after dogfooding) + +Create `.claude/agents/target-debugger.md` as its own PR once the skill has been +through at least one real debug session. + +Agreed charter outline: + +- **Frontmatter**: `model: opus`; omit `tools:` (= all tools — it must edit source + AND drive hardware). Note the registry supports no `effort` field — the agreed + opus/**xhigh** tier is requested per `agent()` call by whichever workflow or + session spawns it. +- **Loop**: instrument → build → flash under one held board lock → dual-side + capture (host usbmon + target RTT/ring-buffer/GDB) → correlate → refine + hypothesis → repeat. Deliberately serial: no fan-out win; the value is + backgrounding a long debug session and the codified playbook. +- **Strictly one instance**, holds the board lock for the entire session — its work + is exactly the "hardware work outside hil_test.py" case in the lock protocol. +- **Skills are its source of truth** (mirror hil-operator's pattern): read + `usb-target-debug`, `usbmon`, `usb-debug`, `usb-recover`, `hil` SKILL.md files + before acting. +- **Hard rule — instrumentation is temporary**: the instrumentation diff must be + reverted (or explicitly listed in the hand-back report) at session end; the *fix* + itself goes to port-dev. Keeps charters clean: this agent produces a diagnosis + and evidence, not a merged patch. + +Questions dogfooding must answer before the charter is written (do NOT guess these +now — that was the whole reason for skill-first): + +1. When to stop instrumenting and report a partial diagnosis vs keep digging. +2. Maximum board-lock hold time / check-in cadence for a backgrounded session. +3. What "revert instrumentation" means when a partial fix emerged mid-debug + (revert + attach diff? keep on a branch?). + +## Conventions and references for the implementing session + +- Skill style exemplars: `.claude/skills/usbmon/SKILL.md`, `usb-debug/SKILL.md`, + `usb-recover/SKILL.md` (the latter two are #3758's copies, present untracked). +- Agent style exemplars: `.claude/agents/hil-operator.md` (lock discipline, + skills-as-source-of-truth), `port-dev.md` (source-edit + verify charter). +- When the agent lands, update the harness spec's agent roster: + `docs/superpowers/specs/2026-07-09-claude-agents-workflows-design.md` + (convention: spec evolves in-repo; plans like this file are per-effort records). +- Agents register from `.claude/agents/*.md` at session start — a new agent file + is only visible to sessions launched after it exists. +- Past cases to mine for the skill's examples: musb babble (ring-buffer trace), + rusb2 FRDY wedge (J-Link PC-sampling), ch32v307 Heisenbug (observation + sensitivity) — details in session memory and the referenced session transcript. diff --git a/docs/superpowers/plans/2026-07-23-esp-target-debug-skill.md b/docs/superpowers/plans/2026-07-23-esp-target-debug-skill.md new file mode 100644 index 000000000..d08902111 --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-esp-target-debug-skill.md @@ -0,0 +1,74 @@ +# esp-target-debug Skill Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Create `.claude/skills/esp-target-debug/SKILL.md` (Espressif built-in USB-Serial-JTAG debug backend) with every recipe verified on the rig's P4, the S3 PHY boundary verified both ways, plus pointer edits in `target-debug` and the `target-debugger` agent. + +**Architecture:** Per spec `docs/superpowers/specs/2026-07-23-esp-target-debug-design.md`. Verification-first: hardware gates 1–6 run before the skill text lands, so only proven content ships unmarked. One lock session per board. + +**Tech Stack:** ESP-IDF at `$HOME/code/esp-idf` (`export.sh` → `openocd-esp32`, `riscv32-esp-elf-gdb`, `xtensa-esp32s3-elf-gdb`, `esptool.py`), rig boards `espressif_p4_function_ev` (uid 6055F9F98715), `espressif_s3_devkitm` (uid 84F703C084E4). + +## Global Constraints + +- Worktree `/home/hathach/code/tinyusb/.claude/worktrees/improve-debug-skill-agent`, branch `claude/improve-debug-skill-agent`. +- Board-lock discipline per `hil` skill; reflash pristine firmware before release; evidence (command + output snippet) in commit message bodies. +- Formatting: aligned table columns, skill-name-only cross-references. +- Unverified content ships tagged `(untested)` or not at all. +- Espressif anything requires `. $HOME/code/esp-idf/export.sh` in that shell first. + +--- + +### Task 1: P4 recon + coexistence gate (spec gates 1) + +- [x] **Step 1: Environment + firmware recon** + +```bash +ls $HOME/code/esp-idf/export.sh && source $HOME/code/esp-idf/export.sh && which openocd riscv32-esp-elf-gdb +ls /home/hathach/code/tinyusb/examples/cmake-build-espressif_p4_function_ev 2>/dev/null || echo "no prebuilt" +lsusb -d 303a:1001 # USB-SJ devices present +``` +If no prebuilt firmware: build `device/cdc_msc_freertos` for the P4 (`idf.py -DBOARD=espressif_p4_function_ev build` in that example, per CLAUDE.md), else use the prebuilt binary. Identify the ELF path for gdb symbolization. + +- [x] **Step 2: Lock P4, ensure known firmware, confirm DUT traffic** + +```bash +python3 test/hil/board_lock.py hold espressif_p4_function_ev --reason "esp-target-debug verify: coexistence" +# flash known build (esptool/idf.py flash -p <port-by-uid>), settle, then confirm enumeration: +lsusb | grep -i cafe # TinyUSB VID on the DUT port +# generate traffic: echo > /dev/ttyACM<N> of the cdc, or timeout 5s cat +``` + +- [x] **Step 3: Attach openocd over USB-SJ while the device runs** + +```bash +openocd -f board/esp32p4-builtin.cfg -c 'adapter serial 60:55:F9:F9:87:15' & # gdb :3333 — USB-SJ iSerial = MAC with colons +riscv32-esp-elf-gdb -batch -ex 'target extended-remote :3333' -ex 'monitor halt' \ + -ex bt -ex 'monitor resume' <p4 elf> +``` +Expected: backtrace with symbols; after resume the CDC device still answers (re-run the traffic check). Record: does the DUT drop off the bus during halt (host URB timeouts — expected per target-debug) and does it recover on resume without re-enumeration? + +- [x] **Step 4: Release-or-continue checkpoint** — keep the lock for Task 2 (same session). No commit yet; evidence to `/tmp/esp_evidence.txt`. + +### Task 2: P4 budget, watchpoint, threads, console (spec gates 2–4) + +- [x] **Step 1: Breakpoint/watchpoint budget** — RISC-V trigger count: in gdb `monitor riscv info` or set watchpoints until rejection; verify a hardware watchpoint on a TinyUSB variable (e.g. `watch -l` on a usbd counter) reports and hits. +- [x] **Step 2: FreeRTOS threads** — `info threads` after halt; expect ESP-IDF tasks incl. the USB task; note whether it works at attach or needs run→stop (mirror the ARM finding). +- [x] **Step 3: Console during traffic** — OUTCOME: stock builds route the console to UART0 (the CP2102 flasher tty — boot log captured there); the USB-SJ CDC carries no log without sdkconfig `ESP_CONSOLE_USB_SERIAL_JTAG`, which stays (untested) in the skill. +- [x] **Step 4: Reflash pristine, release P4 lock.** Evidence appended to `/tmp/esp_evidence.txt`. + +### Task 3: P4 apptrace spike — GATED (spec gate 5) + +Budget 30 min. `openocd -c 'esp apptrace start ...'` against a firmware built with apptrace enabled? Stock HIL firmware has no apptrace init — if a code change would be required, that's the gate answer: land apptrace as `(untested — needs CONFIG_APPTRACE + firmware init)` with the recipe sketch. Only a working capture lands unmarked. + +### Task 4: S3 boundary (spec gate 6) + +- [x] **Step 1: Lock S3, flash `board_test`** (no TinyUSB → PHY free). Attach `openocd -f board/esp32s3-builtin.cfg -c 'adapter serial 84F703C084E4'` + `xtensa-esp32s3-elf-gdb`: halt + bt works. +- [x] **Step 2: Flash a USB device example** — record the exact failure: does 303a:1001 vanish from lsusb (PHY switched), does openocd fail to attach or die mid-session? Capture verbatim error. +- [x] **Step 3: Reflash pristine (a USB example — that is the CI-expected state), release.** + +### Task 5: Write the skill + integration edits + commit + +- [x] **Step 1: Write `.claude/skills/esp-target-debug/SKILL.md`** per spec section order (role/defer, PHY map with verified boundary symptoms, toolchain+attach with the real commands from Tasks 1–4, technique mapping table with verified annotations, rig deltas, external-JTAG TODO). Aligned tables. +- [x] **Step 2: `target-debug` pointer** (2 lines, after probe-mapping bullets) + `target-debugger` agent table row. +- [x] **Step 3: pre-commit, single commit** with evidence summary from `/tmp/esp_evidence.txt`. +- [x] **Step 4: Retrieval sanity** — one fresh-subagent scenario: "debug a TinyUSB hang on the rig's P4" routes to esp-target-debug (not JLink recipes); "same on S3 while cdc_msc runs" routes to the PHY boundary + external-JTAG TODO. diff --git a/docs/superpowers/plans/2026-07-23-target-debug-skill-enhancement.md b/docs/superpowers/plans/2026-07-23-target-debug-skill-enhancement.md new file mode 100644 index 000000000..36a3144c2 --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-target-debug-skill-enhancement.md @@ -0,0 +1,570 @@ +# target-debug Skill & target-debugger Agent Enhancement Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Extend `.claude/skills/target-debug/SKILL.md` (and its agent) with the full debugger facility arsenal from the J-Link, OpenOCD, and GDB manuals — breakpoint/watchpoint depth, OpenOCD RTT, vector catch + fault autopsy, SWO/ITM trace, flash verification — each recipe hardware-verified on the ci rig before it lands unmarked. + +**Architecture:** The skill's organizing spine is its intrusiveness table ("pick the least intrusive technique that can answer the question"); every new facility slots into that model with an honest cost row. Recipes keep the existing dense, copy-paste style. The skill's value is that its recipes are *proven on this rig* — so each task pairs drafting with a bounded hardware verification, and anything unverifiable lands tagged `(untested)` or is dropped. + +**Tech Stack:** arm-none-eabi-gdb 15.2, OpenOCD 0.12.0+dev, SEGGER J-Link V7.94b (`JLinkExe`, `JLinkGDBServer`, `JLinkSWOViewerCLExe`), ci rig boards from `test/hil/tinyusb.json` (10 jlink / 6 openocd / 1 stlink probes). + +**Reference manual:** "Debugging with GDB", **Tenth Edition** (for GDB 18.0.50) — prefer the calibre-library copy via the `read-doc` skill, but **verify the edition on the title page first**: the library also holds an outdated Ninth Edition (2002, GDB 5.1.1, txt) that predates `dprintf`/`watch -l` — do not use it. Fallback fetch: `curl -sL -o /tmp/gdb.pdf https://sourceware.org/gdb/current/onlinedocs/gdb.pdf` (HTML pages block fetchers; the PDF does not). Sections used by this plan: §5.1.2 Setting Watchpoints, §5.1.6 Break Conditions, §5.1.7 Breakpoint Command Lists, §5.1.8 Dynamic Printf (PDF page = book page + 18). NOTE: the manual documents GDB 18; the rig runs 15.2 — the installed `arm-none-eabi-gdb`'s `help <cmd>` is authoritative for feature availability. + +## Global Constraints + +- Worktree: `/home/hathach/code/tinyusb/.claude/worktrees/improve-debug-skill-agent`, branch `claude/improve-debug-skill-agent`. All paths below are relative to it. +- J-Link User Guide link must be exactly `https://kb.segger.com/UM08001_J-Link_/_J-Trace_User_Guide` (user-specified, verified live 2026-07-23). +- **Hardware-verify before landing**: a recipe is committed unmarked only with captured evidence from a rig board; otherwise tag it `(untested)` inline or drop it. Record evidence (command + output snippet) in the task's commit message body. +- Rig discipline (from `hil` + `target-debug` skills): `python3 test/hil/board_lock.py hold <board> --reason "skill-enhance verify: <what>"` before touching hardware, `release` after; reflash pristine firmware before release; NEVER stop the actions-runner; one J-Link client per probe at a time; we are ON host `ci` (config `test/hil/tinyusb.json`). +- Hardware tasks are strictly serial (one board session at a time). Bash timeouts ≥ 10 min for flash+debug cycles. +- Style: match the skill's existing voice — dense, recipe-first, caveats inline. Skill word budget after all tasks: ≤ 2 700 words (`wc -w`, currently 1 763). +- Run `pre-commit run --files <changed>` before every commit. No Co-Authored-By trailers. +- Board selection is runtime data (boards come/go, locks): resolve with the exact python snippet in Task 2 Step 2 and reuse `$JB` (jlink board) / `$OB` (openocd board) thereafter. + +--- + +### Task 1: Manuals reference block + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` (insert new `## Manuals` section immediately before `## Warnings`) + +**Interfaces:** +- Produces: `## Manuals` section that later tasks' text may reference as "see Manuals". + +- [x] **Step 1: Insert the Manuals section** + +In `.claude/skills/target-debug/SKILL.md`, find the line `## Warnings` and insert immediately before it: + +```markdown +## Manuals + +- J-Link / J-Trace User Guide (UM08001): <https://kb.segger.com/UM08001_J-Link_/_J-Trace_User_Guide> — flash breakpoints, RTT, SWO, monitor mode, Commander commands. +- OpenOCD User's Guide: <https://openocd.org/doc/html/index.html> — `rtt`, `bp`/`wp`, `cortex_m vector_catch` / `maskisr`, `itm`/`tpiu`. +- "Debugging with GDB" (the official manual; §5.1 covers break/watch/dprintf): + calibre library first (`read-doc` skill) — use the **Tenth Edition (GDB 18)** + copy, not the 2002 Ninth-Edition txt also present; fallback + `curl -sL -o /tmp/gdb.pdf https://sourceware.org/gdb/current/onlinedocs/gdb.pdf` + (the HTML mirror blocks fetchers; the PDF works). The installed + `arm-none-eabi-gdb`'s `help <cmd>` is authoritative for what this rig runs. + +``` + +- [x] **Step 2: Verify formatting and word count** + +Run: `cd /home/hathach/code/tinyusb/.claude/worktrees/improve-debug-skill-agent && grep -A5 '^## Manuals' .claude/skills/target-debug/SKILL.md && wc -w .claude/skills/target-debug/SKILL.md` +Expected: section present before `## Warnings`; word count ≤ 1 830. + +- [x] **Step 3: Commit** + +```bash +cd /home/hathach/code/tinyusb/.claude/worktrees/improve-debug-skill-agent +pre-commit run --files .claude/skills/target-debug/SKILL.md +git add .claude/skills/target-debug/SKILL.md +git commit -m "docs(target-debug): link J-Link UM08001, OpenOCD and GDB manuals" +``` + +--- + +### Task 2: Breakpoint & watchpoint arsenal (GDB + OpenOCD) + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` — extend the `## GDB — state autopsy and watchpoints` section +- Read-only reference: `test/hil/tinyusb.json` (board resolution) + +**Interfaces:** +- Consumes: nothing from other tasks. +- Produces: board env vars `$JB`, `$OB` resolution snippet (reused by Tasks 3-6); the "halt-per-hit cost model" wording that Task 7's table row cites. + +- [x] **Step 1: Draft the section extension** + +In `.claude/skills/target-debug/SKILL.md`, the GDB section currently ends with the paragraph beginning `While halted the device answers **nothing**`. Insert immediately BEFORE that paragraph: + +```markdown +**Hardware budget — read it off the chip, not from memory** (counts differ +per core: M0+ typically 4 bp/2 wp, M3/M4 6/4, M7 8/4): + +```gdb +p ((*(unsigned*)0xE0002000)>>4) & 0xF # FPB NUM_CODE = hw breakpoints (M7 adds bits[14:12]) +p (*(unsigned*)0xE0001000)>>28 # DWT_CTRL NUMCOMP = watchpoint comparators +``` + +- `hbreak`/`thbreak` force a hardware breakpoint (code in flash can't take a + software break unless the probe does flash breakpoints — J-Link does, + OpenOCD needs `bp <addr> 2 hw`); `tbreak` = one-shot. +- `watch -l <expr>` watches the *address* the expression evaluates to once — + cheap and what you almost always want; `rwatch`/`awatch` trap reads/any + access (hardware-only — they error rather than fall back). OpenOCD (telnet + :4444) adds a data-VALUE match GDB cannot express: `wp <addr> 4 w <value> + [mask]` — fires only when the written value matches (e.g. catch who writes + 0 into a busy flag, ignoring writes of 1). +- **Demand the word "Hardware" in the confirmation.** `watch` silently falls + back to a SOFTWARE watchpoint when no DWT comparator fits (expression too + wide/complex, budget exhausted): GDB then single-steps the whole program — + hundreds of times slower, certain USB death. `Watchpoint 2:` without + "Hardware" = delete it; `set can-use-hw-watchpoints 1` is the default but + narrowing the expression (`watch -l`, cast to a 4-byte int) is the real fix. +- Conditional breaks/watches (`break dcd_edpt_xfer if ep_addr==0x81`) are + evaluated by GDB on the HOST with our stubs — neither JLinkGDBServer nor + OpenOCD supports target-side agent expressions on Cortex-M — so every hit + is a halt+resume (~ms) whether the condition matches or not: fine + post-wedge or on cold paths, wrong under live USB traffic. +- `commands <bpnum> ... end` auto-runs GDB commands at each hit (start with + `silent`, end with `continue` for hands-free evidence collection) — same + halt-per-hit cost. +- `dprintf <loc>,"fmt",args` = printf without recompiling. Stay on the + default `dprintf-style gdb` (host prints): the `call` style runs the + target's own printf mid-halt and `agent` needs stub support — neither is + viable on these probes. Same cost model as conditional breaks; for + ISR-rate events use the RAM ring buffer instead. +- Stepping while the USB ISR fires between every step is chaos: OpenOCD + `cortex_m maskisr steponly` masks interrupts during single-steps only. + The bus keeps running either way — the host may still reset a device that + stops responding mid-step. +- While halted you can poke state to test a hypothesis (`set var + _usbd_dev.ep_status[2][1].busy = 0`) — but that invalidates the snapshot + as post-mortem evidence; dump first, poke after. +``` + +- [x] **Step 2: Resolve verification boards (runtime data)** + +```bash +cd /home/hathach/code/tinyusb +python3 - <<'EOF' +import json +cfg = json.load(open('test/hil/tinyusb.json')) +jl = [b['name'] for b in cfg['boards'] if b['flasher']['name']=='jlink'] +oo = [b['name'] for b in cfg['boards'] if b['flasher']['name']=='openocd'] +print('JLINK candidates:', jl) +print('OPENOCD candidates:', oo) +EOF +``` +Pick the first candidate of each that `python3 test/hil/board_lock.py status` shows unlocked; export as `JB=<jlink board>` `OB=<openocd board>`. Look up `flasher.uid` for each in `test/hil/tinyusb.json` (`JB_UID`, `OB_UID`) and `JLINK_DEVICE`/`OPENOCD_OPTION` from `hw/bsp/*/boards/$JB/board.cmake` (family via `ls -d hw/bsp/*/boards/$JB`). + +- [x] **Step 3: Hardware-verify the budget reads on both probe families** + +```bash +python3 test/hil/board_lock.py hold $JB --reason "skill-enhance verify: bp/wp budget" +printf 'mem32 E0002000, 1\nmem32 E0001000, 1\nqc\n' | \ + JLinkExe -device $JLINK_DEVICE -SelectEmuBySN $JB_UID -if swd -speed 4000 -autoconnect 1 -nogui 1 +python3 test/hil/board_lock.py release $JB +``` +Expected: two register values; decode NUM_CODE and NUMCOMP by hand and check they are plausible (2-8 range). Repeat for `$OB` via `openocd $OPENOCD_OPTION -c init -c 'mdw 0xE0002000' -c 'mdw 0xE0001000' -c shutdown` under its own lock. +If a register reads 0 on one board, note which core and adjust the skill text's example counts if contradicted. + +- [x] **Step 4: Hardware-verify dprintf + commands round-trip on $JB** + +With the board lock held and an already-flashed example (any; do not reflash), a JLinkGDBServer on :2331 (per CLAUDE.md GDB Debugging), run bounded — `commands` blocks cannot be passed via `-ex`, so use a command file: + +```bash +cat > /tmp/bpcmd.gdb <<'EOF' +target remote :2331 +set var $count=0 +watch -l *(unsigned*)&_usbd_dev +delete +dprintf tud_task_ext,"tick\n" +break tud_task_ext +commands 3 +silent +set var $count=$count+1 +continue +end +continue& +shell sleep 3 +interrupt +print $count +EOF +timeout 120 arm-none-eabi-gdb -batch -x /tmp/bpcmd.gdb \ + $(find examples/cmake-build-$JB -name 'cdc_msc.elf' | head -1) +``` +Expected: the `watch` line answers `Hardware watchpoint 1:` (the word +"Hardware" present — this is the skill's software-fallback check, then +deleted), "tick" lines printed, and `$count > 0`. (`tud_task_ext` is the real +symbol — `tud_task` is an inline wrapper; the breakpoint is number 3 after +the watchpoint and dprintf.) Kill the GDB server, reflash pristine +(`ninja`-flash target or `hil_test.py` flash path), release the lock. + +- [x] **Step 5: Apply the Step-1 text, run pre-commit, commit** + +```bash +cd /home/hathach/code/tinyusb/.claude/worktrees/improve-debug-skill-agent +pre-commit run --files .claude/skills/target-debug/SKILL.md +git add .claude/skills/target-debug/SKILL.md +git commit -m "docs(target-debug): breakpoint/watchpoint arsenal with halt-per-hit cost model + +Verified on <JB> (J-Link) + <OB> (OpenOCD): FPB/DWT budget reads, dprintf, +breakpoint command lists. <paste the two register values here>" +``` + +--- + +### Task 3: OpenOCD RTT — RTT is not J-Link-only + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` — `## TU_LOG capture` section + +**Interfaces:** +- Consumes: `$OB`, `$OB_UID`, `$OPENOCD_OPTION` from Task 2 Step 2. +- Produces: the corrected claim "RTT works on any OpenOCD-driven probe" that Task 7's agent text repeats. + +- [x] **Step 1: Replace the J-Link-only claim** + +In the `## TU_LOG capture` section, replace: + +```markdown +Build with `LOG=2` (`LOG=3` adds per-transfer noise and much more timing skew). +`LOGGER=rtt` routes it over the debug probe (J-Link only) — no UART wiring: +``` + +with: + +```markdown +Build with `LOG=2` (`LOG=3` adds per-transfer noise and much more timing skew). +`LOGGER=rtt` routes it over the debug probe — no UART wiring. SEGGER's host +tools need a J-Link, but OpenOCD serves the same RTT buffer on ST-Link / +CMSIS-DAP / WCH-Link boards: +``` + +- [x] **Step 2: Add the OpenOCD RTT recipe** + +Immediately after the existing J-Link/UART capture code block (ends with `... | tee /tmp/uart.log`), add: + +```markdown +```bash +# OpenOCD RTT (any probe OpenOCD drives) — in telnet :4444 (or -c equivalents): +rtt setup 0x20000000 0x8000 "SEGGER RTT" # search range = RAM ORIGIN + LENGTH (from the .ld / map file) +rtt start # after firmware booted; rerun after each reflash +rtt server start 19021 0 +# then: timeout 20s nc localhost 19021 > /tmp/rtt.log +``` + +OpenOCD polls the buffer (default 10 ms): bursty logs can drop lines a J-Link +would keep — prefer J-Link where both exist; the drain-model warning below +applies unchanged. +``` + +- [x] **Step 3: Hardware-verify on $OB** + +```bash +python3 test/hil/board_lock.py hold $OB --reason "skill-enhance verify: openocd rtt" +cd examples/device/cdc_msc && cmake -B build-rtt -DBOARD=$OB -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel -DLOG=2 -DLOGGER=rtt && cmake --build build-rtt +# flash it (ninja -C build-rtt cdc_msc-openocd), then: +openocd $OPENOCD_OPTION & # gdb :3333, telnet :4444 +{ echo 'rtt setup 0x20000000 0x8000 "SEGGER RTT"'; echo 'rtt start'; echo 'rtt server start 19021 0'; sleep 1; } | nc -q1 localhost 4444 +timeout 10s nc localhost 19021 > /tmp/ob_rtt.log; head /tmp/ob_rtt.log +``` +Expected: TinyUSB boot banner / log lines in `/tmp/ob_rtt.log`. Adjust the search range from the board's linker script if the control block isn't found ("rtt: No control block found") and mirror any correction into the Step-2 text. Kill openocd, reflash pristine cdc_msc (no LOG), release lock, delete `build-rtt`. + +- [x] **Step 4: Commit** + +```bash +pre-commit run --files .claude/skills/target-debug/SKILL.md +git add .claude/skills/target-debug/SKILL.md +git commit -m "docs(target-debug): RTT via OpenOCD on non-J-Link probes + +Verified on <OB>: rtt setup/start/server + nc capture of boot log. +<paste first captured log line>" +``` + +--- + +### Task 4: Vector catch + fault autopsy + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` — new section after `## GDB — state autopsy and watchpoints` + +**Interfaces:** +- Consumes: `$JB` from Task 2. (Corrected during execution: $OB/rp2040 is ARMv6-M — no CFSR/BFAR and only VC_HARDERR, so the full autopsy verify needs the ARMv7-M $JB; the payload is a bad LOAD because stores fault imprecisely with BFAR invalid.) +- Produces: section title `## Vector catch + fault autopsy` cited by Task 7's table row. + +- [x] **Step 1: Insert the new section** + +After the GDB section (i.e. before `## RAM ring-buffer trace`), insert: + +```markdown +## Vector catch + fault autopsy — catch the crash, not the wedge + +A "wedge" that is really a fault (HardFault loop, lockup) autopsies best AT +the faulting instruction, not minutes later. Arm before reproducing: + +```gdb +# tool-agnostic (any probe, incl. J-Link): DEMCR trap bits — halt on fault +set *(unsigned*)0xE000EDFC |= (1<<10)|(1<<9)|(1<<8)|(1<<7)|(1<<6)|(1<<5)|(1<<4) +# = VC_HARDERR|INTERR|BUSERR|STATERR|CHKERR|NOCPERR|MMERR; bit0 VC_CORERESET halts at reset +``` + +OpenOCD native form: `cortex_m vector_catch hard_err bus_err state_err chk_err mm_err`. +When it fires the core halts at the fault; decode: + +```gdb +p/x *(unsigned*)0xE000ED28 # CFSR — low byte MemManage, byte1 BusFault, top half UsageFault +p/x *(unsigned*)0xE000ED2C # HFSR — bit30 FORCED = an escalated lower-priority fault +p/x *(unsigned*)0xE000ED38 # BFAR — faulting address (valid if CFSR bit15 BFARVALID) +x/8wx $msp # stacked frame: r0 r1 r2 r3 r12 lr pc xpsr — pc = culprit +``` + +`arm-none-eabi-addr2line -e <elf> <stacked pc>` names the line. Caveats: a +vector-catch halt is still a halt (host-side URB timeouts apply); the bits +persist until power-cycle — clear them (`... &= ~0x7F1`) before handing the +board back; RISC-V ports have no DEMCR — use a breakpoint on the trap handler. +``` + +- [x] **Step 2: Hardware-verify with a deliberate fault on $JB (ARMv7-M)** + +Create the fault build (NOT committed): + +```bash +python3 test/hil/board_lock.py hold $JB --reason "skill-enhance verify: vector catch" +cd examples/device/cdc_msc # executed on $JB (stm32f407disco, ARMv7-M) via JLinkExe — see commit evidence +# temporary patch — revert after: fault 5 s after boot +python3 - <<'EOF' +import pathlib +p = pathlib.Path('src/main.c'); s = p.read_text() +import re +s = re.sub(r'\\nint main\\(void\\)', + '\\nstatic void _fault_after_5s(void){ static uint32_t t0=0; if(!t0) t0=tusb_time_millis_api();' + ' if(tusb_time_millis_api()-t0>5000) (void)*(volatile uint32_t*)0xCF000000u; }\\n\\nint main(void)', s, count=1) # board_millis is gone; helper must sit after the includes +s = s.replace('led_blinking_task();', 'led_blinking_task(); _fault_after_5s();', 1) +p.write_text(s) +EOF +grep -n '_fault_after_5s' src/main.c # expect 3 hits: definition + call + (none in decl block) +cmake -B build-fault -DBOARD=$JB -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel && cmake --build build-fault +``` +(If `app_led_task`/`board_millis` anchors differ in the current `main.c`, place the same 3-line helper on whatever per-loop task function exists — the fault line `*(volatile uint32_t*)0xCF000000u = 0;` is the payload.) +Flash `build-fault`, then: + +```bash +# executed variant: DEMCR armed + autopsy via JLinkExe command file on $JB (see commit c1d2d305f evidence); OpenOCD-native form: +openocd $OPENOCD_OPTION -c init -c 'cortex_m vector_catch hard_err bus_err' & +timeout 60 arm-none-eabi-gdb -batch -ex 'target remote :3333' -ex 'monitor reset run' \ + -ex 'shell sleep 8' -ex 'interrupt' \ + -ex 'p/x *(unsigned*)0xE000ED28' -ex 'p/x *(unsigned*)0xE000ED38' -ex 'x/8wx $msp' \ + build-fault/cdc_msc.elf +``` +Expected: halted in the fault path, CFSR BusFault bits set, **BFAR = 0xCF000000**, stacked pc addr2lines to `_fault_after_5s`. If the write is silently ignored on this core (some buses RAZ/WI), switch payload to a NULL-function call `((void(*)(void))0x1)();` and note UsageFault/INVSTATE instead. + +- [x] **Step 3: Clean up hardware state** + +`git checkout -- src/main.c`, delete `build-fault/`, clear DEMCR bits (`set *(unsigned*)0xE000EDFC &= ~0x7F1` via a final gdb attach or power-cycle note), reflash pristine cdc_msc, `board_lock.py release $JB`. + +- [x] **Step 4: Commit** + +```bash +pre-commit run --files .claude/skills/target-debug/SKILL.md +git add .claude/skills/target-debug/SKILL.md +git commit -m "docs(target-debug): vector catch + Cortex-M fault autopsy recipe + +Verified on <OB>: deliberate bad-address write halted via vector_catch, +CFSR=<val> BFAR=0xCF000000, stacked pc resolved by addr2line." +``` + +--- + +### Task 5: SWO/ITM experiment — exception trace & hardware PC sampling + +This is an EXPERIMENT task with an explicit gate: the section lands **unmarked only if packets are actually captured** on a rig board; otherwise it lands tagged `(untested — SWO wiring unconfirmed on this rig)`. Budget: 30 min of hardware time, then decide. + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` — new subsection inside the PC-sampling section (after the OpenOCD variant paragraph) + +**Interfaces:** +- Consumes: `$JB`, `$JB_UID`, `$JLINK_DEVICE` from Task 2. +- Produces: verified-or-tagged status consumed by Task 7's table row for SWO. + +- [x] **Step 1: Probe for SWO output (gate experiment)** + +```bash +python3 test/hil/board_lock.py hold $JB --reason "skill-enhance verify: SWO" +# arm DWT sources while the fw runs (background mem write, no halt): +printf 'w4 E0001000, 0x00011401\nqc\n' | JLinkExe -device $JLINK_DEVICE -SelectEmuBySN $JB_UID -if swd -speed 4000 -autoconnect 1 -nogui 1 +# EXCTRCENA(16)|PCSAMPLENA(12)|SYNCTAP(10)|CYCCNTENA(0); tune POSTPRESET[4:1] if PC samples flood — then hand the probe to the viewer: +timeout 20s JLinkSWOViewerCLExe -device $JLINK_DEVICE -usb $JB_UID -swofreq 4000000 -itmmask 0xFFFFFFFF | head -40 +``` +Gate: ANY decoded output (stimulus, PC samples, exception packets) = SWO wired on `$JB` → land unmarked with the observed invocation. No output → try one more J-Link board, then land tagged. Either way `release $JB` after reflashing nothing (this experiment flashes nothing). + +- [x] **Step 2: Insert the section (wording per gate outcome)** + +Append to the `## PC-sampling` section: + +```markdown +### SWO/ITM — hardware-timed trace on one pin (J-Link) + +If the board routes SWO (TRACESWO), DWT emits packets with ZERO code change: +**exception trace** (`DWT_CTRL` bit16 EXCTRCENA) — every IRQ enter/exit, +timestamped, the ISR-ordering evidence the ring buffer needs code for — and +**hardware PC sampling** (bit12 PCSAMPLENA), better histograms than DWT_PCSR +polling. Arm the bits, then give the probe to the viewer (one client rule): + +```bash +printf 'w4 E0001000, 0x00011401\nqc\n' | JLinkExe -device $JLINK_DEVICE -SelectEmuBySN <uid> ... +timeout 20s JLinkSWOViewerCLExe -device $JLINK_DEVICE -usb <uid> -swofreq 4000000 -itmmask 0xFFFFFFFF +``` + +SWO needs the pin physically wired to the probe — many rig boards route only +SWDIO/SWCLK. If the viewer shows nothing, that is the wiring, not the recipe. +``` + +If the gate FAILED on both boards, append ` (untested — SWO wiring unconfirmed on this rig)` to the subsection heading and keep the text. + +- [x] **Step 3: Commit** + +```bash +pre-commit run --files .claude/skills/target-debug/SKILL.md +git add .claude/skills/target-debug/SKILL.md +git commit -m "docs(target-debug): SWO exception-trace / hw PC-sampling recipe + +Gate result on <JB>: <captured packet types | no SWO output — tagged untested>." +``` + +--- + +### Task 6: Flash verification, FreeRTOS thread awareness, semihosting & monitor-mode notes + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` — `## Warnings` section + GDB section tail + +**Interfaces:** +- Consumes: `$JB`, `$JB_UID`, `$JLINK_DEVICE` from Task 2. +- Produces: warning-list entries cited in Task 7's retrieval test scenarios. + +- [x] **Step 1: Add flash-content verification to Warnings** + +In `## Warnings`, after the "A marginal link can fake a deterministic firmware bug" bullet, add: + +```markdown +- **"Flash OK" can lie** (silent no-op: old firmware keeps running after a + green flash). When behavior contradicts the code you think is flashed, + verify flash against the build: + `arm-none-eabi-objcopy -O binary fw.elf /tmp/fw.bin`, then J-Link + `verifybin /tmp/fw.bin,<flash-base>` (Commander) or OpenOCD + `verify_image /tmp/fw.bin <flash-base>` — a mismatch means reflash with + verification before debugging another minute. +``` + +- [x] **Step 2: Add FreeRTOS + semihosting + monitor-mode notes to the GDB section** + +Append to the end of the `## GDB — state autopsy and watchpoints` section (after the Task-2 additions): + +```markdown +FreeRTOS examples (`*_freertos`): add `-rtos GDBServer/RTOSPlugin_FreeRTOS` +to JLinkGDBServer (OpenOCD: `-rtos FreeRTOS` on the target) and `info +threads` / `thread <n>` shows every task's stack — a USB task blocked on a +queue vs. spinning is one `bt` away. Semihosting is never the answer here: +each call traps and halts the core — RTT does the same job without stopping. +**Monitor-mode debugging** (J-Link, M3+) can keep the USB ISR serviced while +you sit at a breakpoint — needs SEGGER's `JLINK_MONITOR.c`/ISR files compiled +in + `SetMonModeDebug=1`; not set up in this repo, reach for it when a bug +truly needs live breakpoints without killing the bus: +<https://kb.segger.com/Monitor_Mode_Debugging> (untested). +``` + +- [x] **Step 3: Hardware-verify verifybin + FreeRTOS awareness on $JB** + +```bash +python3 test/hil/board_lock.py hold $JB --reason "skill-enhance verify: verifybin+rtos" +# (a) verifybin positive path against whatever is flashed — first reflash a known build: +# flash examples/cmake-build-$JB/device/cdc_msc, then: +arm-none-eabi-objcopy -O binary examples/cmake-build-$JB/device/cdc_msc/cdc_msc.elf /tmp/fw.bin +printf 'verifybin /tmp/fw.bin,<flash-base from board .ld>\nqc\n' | \ + JLinkExe -device $JLINK_DEVICE -SelectEmuBySN $JB_UID -if swd -speed 4000 -autoconnect 1 -nogui 1 +# (b) rtos plugin: flash cdc_msc_freertos for $JB (build if missing), start +JLinkGDBServer -device $JLINK_DEVICE -select usb=$JB_UID -if swd -speed 4000 -port 2331 -nogui -rtos GDBServer/RTOSPlugin_FreeRTOS & +timeout 60 arm-none-eabi-gdb -batch -ex 'target remote :2331' -ex 'monitor halt' -ex 'info threads' \ + <path to cdc_msc_freertos.elf> +python3 test/hil/board_lock.py release $JB # after pristine reflash +``` +Expected: (a) `Verify successful.` (b) `info threads` lists FreeRTOS tasks (`usbd`, `IDLE`, ...). If the plugin errors ("Could not load RTOS plugin"), drop the JLinkGDBServer variant from the Step-2 text and keep only the OpenOCD `-rtos FreeRTOS` form tagged `(untested)`. + +- [x] **Step 4: Commit** + +```bash +pre-commit run --files .claude/skills/target-debug/SKILL.md +git add .claude/skills/target-debug/SKILL.md +git commit -m "docs(target-debug): flash verifybin, FreeRTOS thread awareness, monitor-mode pointer + +Verified on <JB>: verifybin 'Verify successful.'; info threads listed <n> tasks." +``` + +--- + +### Task 7: Intrusiveness table integration, agent update, retrieval test + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` — the technique/intrusiveness table +- Modify: `.claude/agents/target-debugger.md` — primary-playbook bullet + +**Interfaces:** +- Consumes: verified/untested status of every technique from Tasks 2-6. + +- [x] **Step 1: Extend the intrusiveness table** + +The table under `## Pick the least intrusive technique that can answer the question` currently has 5 rows (PC-sampling → GDB halt). Replace it with (keep the header row and any wording the earlier tasks did not contradict): + +```markdown +| Technique | Intrusiveness | Reach for it when | +|---|---|---| +| PC-sampling | none — no halt, no code change | core wedged/spinning somewhere unknown (rusb2 FRDY) | +| SWO exception trace / hw PC-sample | none — needs SWO pin wired | ISR ordering/timing with zero code change | +| Vector catch | none until a fault fires | crash-shaped wedges — autopsy AT the faulting pc | +| RAM ring-buffer | ~tens of cycles per event | ISR ordering/timing bugs (musb babble) | +| TU_LOG (RTT) | µs per line | logic bugs that survive logging (J-Link or OpenOCD rtt) | +| TU_LOG (UART) | ms per line — blocking write | same, when no debug-probe RTT path | +| dprintf / conditional breakpoint | halt+resume per hit (~ms) | low-rate probes post-wedge; never ISR-rate events | +| GDB halt / breakpoints | stops USB service entirely | post-mortem state autopsy once wedged | +``` + +If Task 5's gate failed, keep the SWO row but append ` (untested)` in its "Reach for it" cell. + +- [x] **Step 2: Update the agent's playbook bullet** + +In `.claude/agents/target-debugger.md`, replace: + +```markdown +- `.claude/skills/target-debug/SKILL.md` — your primary playbook: technique + choice by intrusiveness, channel choice by link topology, capture recipes, + GDB autopsy, all rig warnings. +``` + +with: + +```markdown +- `.claude/skills/target-debug/SKILL.md` — your primary playbook: technique + choice by intrusiveness, channel choice by link topology, capture recipes, + breakpoint/watchpoint budget and cost model, vector catch + fault autopsy, + GDB autopsy, all rig warnings. +``` + +- [x] **Step 3: Word-count and stale-reference check** + +Run: `wc -w .claude/skills/target-debug/SKILL.md` — expected ≤ 2 700. If over, trim prose (not recipes) until under. +Run: `grep -n 'J-Link only' .claude/skills/target-debug/SKILL.md` — expected: no output (Task 3 removed the claim). + +- [x] **Step 4: Retrieval test (skill-TDD GREEN gate)** + +Dispatch a fresh read-only subagent (Explore) that reads ONLY the updated `.claude/skills/target-debug/SKILL.md` and answers: + +1. "A CH32 board's firmware wedges; you suspect a HardFault loop. Least-intrusive next step?" — expected: vector catch (with the RISC-V caveat noted: CH32 is RISC-V → breakpoint on trap handler). +2. "You need RTT logs on an ST-Link-only board." — expected: OpenOCD `rtt setup/start/server`, NOT "impossible/J-Link only". +3. "Who is writing 0 into a busy flag, under live traffic?" — expected: OpenOCD value-match watchpoint `wp <addr> 4 w 0`, NOT a GDB conditional watch (halt-per-hit cost). +4. "Flash reported OK but behavior matches last week's build." — expected: verifybin/verify_image. +5. "You set `watch xfer_status[2][1]` and GDB answered `Watchpoint 2:` (no 'Hardware'). Proceed?" — expected: NO — software-watchpoint fallback single-steps the program; delete and narrow the expression. + +All five must route correctly; a miss = fix the text (usually the table row or a heading), re-test. + +- [x] **Step 5: Final commit** + +```bash +pre-commit run --files .claude/skills/target-debug/SKILL.md .claude/agents/target-debugger.md +git add .claude/skills/target-debug/SKILL.md .claude/agents/target-debugger.md +git commit -m "docs(target-debug): integrate new techniques into intrusiveness table; agent playbook bullet + +Retrieval test: 4/4 scenarios routed correctly." +``` + +--- + +## Deferred / out of scope (deliberate) + +- **ETM / J-Trace instruction trace** — no J-Trace hardware on the rig; UM08001 "Trace" chapter is linked for the day one arrives. +- **Monitor-mode debugging as a working recipe** — needs SEGGER monitor files compiled into firmware (a firmware feature, not a doc change); landed as a pointer + `(untested)` in Task 6. +- **ITM stimulus-port logging backend for TU_LOG** — would be a `lib/` + `LOGGER=itm` firmware feature; out of scope for a skill-doc plan. +- **GDB tracepoints (`trace`/`tfind`)** — need a tracing-capable stub; neither JLinkGDBServer nor OpenOCD implements them for Cortex-M. diff --git a/docs/superpowers/plans/2026-07-24-etm-trace-agent-integration.md b/docs/superpowers/plans/2026-07-24-etm-trace-agent-integration.md new file mode 100644 index 000000000..ebcc089fd --- /dev/null +++ b/docs/superpowers/plans/2026-07-24-etm-trace-agent-integration.md @@ -0,0 +1,299 @@ +# etm-trace Skill Tightening + target-debugger Agent Integration Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Fix post-rebase staleness in the etm-trace skill, add its +hardware-consent gate, and wire it into the target-debugger agent + target-debug +skill the same way usb-sniffer is wired in (hardware-gated, user-confirmed). + +**Architecture:** Three curated instruction files get surgical edits (this repo +treats skills/agents as curated docs — smallest possible diffs, no bulk +rewrites). The gate follows the two existing consent patterns: in the *skill* +(read by interactive sessions) it is "confirm with the user unless they asked"; +in the *agent* (which cannot ask mid-session) it is "only when your prompt +states it", mirroring the agent's existing lock-force rule. + +**Tech Stack:** Markdown instruction files, git, one subagent retrieval test. + +## Global Constraints + +- Work in the existing worktree `/home/hathach/code/tinyusb/.worktrees/add-etm-trace-skill` on branch `claude/add-etm-trace-skill` (already rebased onto PR 3786 — the base that renamed `usb-target-debug` → `target-debug` and rewrote `.claude/agents/target-debugger.md`). Never switch the primary checkout's branch. +- Commit messages: imperative mood, no `Co-Authored-By`/`Claude-Session` trailers (repo rule: hathach is sole author). +- Commits are SSH-signed automatically (keyring agent); if `git commit` fails with `fatal: failed to write commit object`, stop and report — do not commit unsigned. +- Run `pre-commit run --files <changed files>` before each commit; re-stage anything the hooks fix. +- Do not touch `.idea/`, `*.jdebug.user`, or `PICO2_TRACE_PCB_HANDOFF.md` (user-local files in the worktree). + +--- + +### Task 1: Tighten etm-trace SKILL.md — stale names + hardware-consent gate + +**Files:** +- Modify: `.claude/skills/etm-trace/SKILL.md` (lines ~12–33: cross-skill table, PC-sampling pointer, Requirements) +- No test file (instruction doc; verification = grep + subagent test in Task 4) + +**Interfaces:** +- Consumes: nothing from other tasks. +- Produces: the phrase `confirm with the user` gate bullet in Requirements that Task 2/3 rows reference by concept (no code interface). + +- [ ] **Step 1: Verify the stale references exist (the "failing test")** + +Run: +```bash +cd /home/hathach/code/tinyusb/.worktrees/add-etm-trace-skill +grep -n "usb-target-debug" .claude/skills/etm-trace/SKILL.md +``` +Expected: exactly 2 hits (the table row at ~line 15 and the PC-sampling +pointer at ~line 19). If 0 hits, the file was already fixed — skip Steps 2–3. + +- [ ] **Step 2: Apply the edits** + +Edit `.claude/skills/etm-trace/SKILL.md`. + +Replace this block (current content): + +```markdown +| Skill | Answers | +|--------------------|----------------------------------------------------------------------------| +| `usbmon` | what the host actually exchanged (URBs) | +| `usb-target-debug` | what the device did (logs, driver state, sampled PCs) | +| `usb-sniffer` | what crossed the wire | +| **`etm-trace`** | **exactly which instructions executed, when** (profile, coverage, history) | + +Use `usb-target-debug`'s DWT PC-sampling for a quick statistical profile; use +this skill for exact counts, coverage, or instruction-by-instruction history. +``` + +with: + +```markdown +| Skill | Answers | +|-----------------|----------------------------------------------------------------------------| +| `usbmon` | what the host actually exchanged (URBs) | +| `target-debug` | what the target did (logs, driver state, sampled PCs) | +| `usb-sniffer` | what crossed the wire | +| **`etm-trace`** | **exactly which instructions executed, when** (profile, coverage, history) | + +Use `target-debug`'s DWT PC-sampling for a quick statistical profile; use +this skill for exact counts, coverage, or instruction-by-instruction history. +``` + +Then in the `## Requirements` section, replace the first bullet: + +```markdown +- J-Trace on USB (`lsusb -d 1366:1020`) wired to the board's trace header; + select it by J-Link USB **nickname** (this rig: `jtrace`) — never commit + serials. +``` + +with: + +```markdown +- J-Trace on USB (`lsusb -d 1366:1020`) wired to the board's trace header; + select it by J-Link USB **nickname** (this rig: `jtrace`) — never commit + serials. +- **Physical setup is per-board and exclusive** (one J-Trace, moved between + boards; some rigs are fly-wired): unless the user just asked for trace on + this board or your task states it is wired, **confirm with the user** that + the J-Trace is connected to the target before flashing or capturing. +``` + +- [ ] **Step 3: Verify the edits** + +Run: +```bash +grep -c "usb-target-debug" .claude/skills/etm-trace/SKILL.md; grep -c "confirm with the user" .claude/skills/etm-trace/SKILL.md; grep -rn "usb-target-debug" .claude/skills/etm-trace/boards.md +``` +Expected: `0`, then `1` (or more), then no output from boards.md (it has no +stale names — do not edit it). + +- [ ] **Step 4: Commit** + +```bash +cd /home/hathach/code/tinyusb/.worktrees/add-etm-trace-skill +pre-commit run --files .claude/skills/etm-trace/SKILL.md +git add .claude/skills/etm-trace/SKILL.md +git commit -m "etm-trace: post-rename references, per-board hardware-consent gate + +target-debug replaced usb-target-debug in the debug-skill overhaul; update +the cross-skill table and PC-sampling pointer. Add the consent gate: the +J-Trace is a single probe moved between boards, so captures on a board the +user did not just ask about need explicit confirmation that it is wired." +``` +Expected: commit succeeds; `git log --format="%G?" -1` prints `G`. + +--- + +### Task 2: Add etm-trace to the target-debugger agent's skill table + +**Files:** +- Modify: `.claude/agents/target-debugger.md` (skill table, after the `usb-sniffer` row at ~line 24) + +**Interfaces:** +- Consumes: the etm-trace skill name and its `boards.md` (Task 1 keeps both valid). +- Produces: the agent-side gate wording ("only when your prompt states…") that Task 4's subagent test asserts. + +- [ ] **Step 1: Verify etm-trace is absent (the "failing test")** + +Run: +```bash +grep -c "etm-trace" .claude/agents/target-debugger.md +``` +Expected: `0`. + +- [ ] **Step 2: Add the table row** + +In `.claude/agents/target-debugger.md`, after this row: + +```markdown +| usb-sniffer | wire-level capture (hardware tap): host can't see the bus, usbmon vs target logs disagree, or TinyUSB is the host (no usbmon anywhere) | +``` + +insert: + +```markdown +| etm-trace | instruction-level ETM trace via SEGGER J-Trace (exact execution history, profile, coverage) when sampled PCs and logs cannot resolve the mechanism. Requires the J-Trace physically wired to THIS board (supported boards: the skill's boards.md) — use only when your prompt states the board is trace-wired or the user asked for it; otherwise name it in `notes` as the next technique | +``` + +(The gate is prompt-based, not ask-based: this agent cannot ask the user +mid-session — same pattern as the existing lock-force rule.) + +- [ ] **Step 3: Verify** + +Run: +```bash +grep -n "etm-trace" .claude/agents/target-debugger.md | wc -l; grep -n "prompt states the board is trace-wired" .claude/agents/target-debugger.md +``` +Expected: `1` match count; the gate phrase found once. + +- [ ] **Step 4: Commit** + +```bash +pre-commit run --files .claude/agents/target-debugger.md +git add .claude/agents/target-debugger.md +git commit -m "agents: target-debugger may escalate to etm-trace, prompt-gated + +Instruction-level trace outranks PC-sampling when samples cannot resolve a +mechanism, but the J-Trace is exclusive per-board hardware: the agent uses +it only when its prompt says the board is trace-wired or the user asked, +and otherwise proposes it in notes - mirroring the lock-force consent rule." +``` +Expected: commit succeeds, signature `G`. + +--- + +### Task 3: Cross-pointer row in target-debug's channel table + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` (channel table at ~lines 14–19) + +**Interfaces:** +- Consumes: skill name `etm-trace` (Task 1). +- Produces: nothing later tasks rely on. + +- [ ] **Step 1: Verify absence (the "failing test")** + +Run: +```bash +grep -c "etm-trace" .claude/skills/target-debug/SKILL.md +``` +Expected: `0`. + +- [ ] **Step 2: Add the row** + +In `.claude/skills/target-debug/SKILL.md`, after this row: + +```markdown +| `usb-sniffer` | what crossed the wire (PIDs, handshakes, resets) | hardware tap cabled in — role-agnostic | +``` + +insert: + +```markdown +| `etm-trace` | exactly which instructions executed (profile, coverage, history) | SEGGER J-Trace wired to this board's trace header — confirm with the user first | +``` + +- [ ] **Step 3: Verify table renders consistently** + +Run: +```bash +grep -A6 "| Skill | Answers" .claude/skills/target-debug/SKILL.md | head -8 +``` +Expected: five data rows, `etm-trace` last, pipes aligned with the header +(cosmetic alignment may differ; column count must be 3). + +- [ ] **Step 4: Commit** + +```bash +pre-commit run --files .claude/skills/target-debug/SKILL.md +git add .claude/skills/target-debug/SKILL.md +git commit -m "target-debug: list etm-trace as the instruction-level channel + +Fifth capture view alongside usbmon/kernel/target/wire: exact execution +history via J-Trace, existing only where the trace header is wired - +confirm with the user before reaching for it." +``` +Expected: commit succeeds, signature `G`. + +--- + +### Task 4: Subagent retrieval test of the agent gate + +**Files:** +- None modified; read-only test of `.claude/agents/target-debugger.md`. + +**Interfaces:** +- Consumes: Task 2's gate wording. + +- [ ] **Step 1: Run the pressure scenario** + +Dispatch a fresh general-purpose subagent with exactly this prompt: + +``` +Read /home/hathach/code/tinyusb/.worktrees/add-etm-trace-skill/.claude/agents/target-debugger.md and answer as if you were that agent. Scenario: your dispatch prompt said only "debug why cdc_msc wedges on ra6m5_ek under bulk traffic; board lock authorized". PC-sampling shows a tight spin in dcd_int_handler but cannot tell which branch path loops. The ra6m5_ek IS listed in etm-trace's boards.md as validated. Do you start an ETM capture now? Answer YES or NO with the governing sentence from the agent file, then say what you would do instead. +``` + +- [ ] **Step 2: Evaluate** + +Expected answer: **NO** — the prompt did not state the board is trace-wired +nor that the user asked; the agent quotes the gate row and proposes +etm-trace in the `notes` field of its output instead. If the subagent +answers YES or hedges, the gate wording is ambiguous: tighten the Task 2 row +(e.g. bold the "only when") and re-run this test once. + +- [ ] **Step 3: Record** + +No commit. Note the test outcome in the final summary to the user. + +--- + +### Task 5: Plan file + final verification + +**Files:** +- Create (already saved by the planner): `docs/superpowers/plans/2026-07-24-etm-trace-agent-integration.md` + +- [ ] **Step 1: Full-sweep verification** + +Run: +```bash +cd /home/hathach/code/tinyusb/.worktrees/add-etm-trace-skill +grep -rn "usb-target-debug" .claude/skills/etm-trace/ ; git log --oneline -4; git log --format="%G?" -3 | sort | uniq -c +``` +Expected: no stale references; three new commits on top of `52973317e`-era +history; all signatures `G`. + +- [ ] **Step 2: Commit the plan document** + +```bash +git add docs/superpowers/plans/2026-07-24-etm-trace-agent-integration.md +git commit -m "docs: plan for etm-trace tightening and target-debugger integration" +``` +Expected: commit succeeds (repo convention: plans are committed, cf. PR 3786's +`docs/superpowers/plans/`). + +--- + +## Self-Review + +- **Spec coverage:** "update/tighten etm-trace skill" → Task 1 (stale names = the concrete rot; consent gate added). "update target-debugger agent to make use of it" → Task 2. "like usb-sniffer… require jtrace and hardware setup on supported boards, confirm with user first or if user instruct to" → gate wording in Tasks 1 (skill: confirm-with-user), 2 (agent: prompt-gated because the agent cannot ask), 3 (channel table "confirm with the user first"). Covered. +- **Placeholders:** none — every step carries the exact text or command. +- **Consistency:** skill name `target-debug` and file paths match the post-3786 tree; `boards.md` name used consistently; gate phrasing intentionally differs between skill (interactive) and agent (prompt-gated) — that asymmetry is the design, documented in Task 2 Step 2. diff --git a/docs/superpowers/plans/2026-07-27-openocd-unified-fork.md b/docs/superpowers/plans/2026-07-27-openocd-unified-fork.md new file mode 100644 index 000000000..e0f1f9678 --- /dev/null +++ b/docs/superpowers/plans/2026-07-27-openocd-unified-fork.md @@ -0,0 +1,602 @@ +# Unified OpenOCD Fork (`hathach/openocd`) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** One OpenOCD fork at `hathach/openocd` (default branch `tinyusb`) that flashes, debugs and RTT-captures every TinyUSB rig target — RP2040, RP2350 (arm + riscv), all WCH CH32/CH5xx, Analog Devices MAX32, and Espressif — replacing the four separate OpenOCD trees on ci. + +**Architecture:** Fork `openocd-org/openocd` master (mainline is 1610 commits ahead of the RPi fork base and now the sole home of RISC-V support). Layer on top: 4 RP2350 TCL configs from the RPi fork, 1 ported max32665 TCL config from the ADI fork, the `wlinke` adapter + `sdi` transport + WCH flash drivers from `hathach/riscv-openocd-wch` (driving CH32 with **mainline's** riscv target if the DTM hypothesis holds), and ESP32-P4 TCL configs adapted from `espressif/openocd-esp32` onto mainline's generic-riscv ESP pattern. ESP32/S2/S3/C3/C6/H2 debug is already in mainline; ESP flash stays with esptool. + +**Tech Stack:** OpenOCD (autotools, C), TCL configs, GitHub CLI, TinyUSB HIL rig (`hil_test.py`, `board_lock.py`). + +## Global Constraints + +- Everything runs **on ci** (this machine *is* the rig — hostname `ci`); no SSH hop needed. +- Repo: `hathach/openocd`, default branch **`tinyusb`**, source clone at `~/app/openocd`, install prefix `$HOME/app/openocd_tinyusb`. +- **One commit per downstream fork** on the `tinyusb` branch: one for raspberrypi/openocd, one for analogdevicesinc/openocd, one for riscv-openocd-wch, one for espressif/openocd-esp32 (plus the initial README commit). Iterate with `git commit --amend` / squash before declaring a task done. +- No `Co-Authored-By: Claude` / `Claude-Session:` trailers in any commit. +- **`~/.local/bin/openocd_wch` (symlink) and `~/app/openocd_wch_new` stay untouched until Task 8's 4/4 WCH boards pass** — it is the rig's only CH32 flasher. Backup exists at `~/.local/bin/openocd_wch.bak-20260727`. +- Hold a board lock for every hardware step: `python3 test/hil/board_lock.py hold <board> --reason "openocd-unified verify"`; release after. **Never stop the actions-runner.** +- WCH RTT: always `rtt polling_interval 1`; **never `reset run` inside an SDI session** (target does not come back). +- `pkill -x openocd` — never `pkill -f` (pattern matches your own shell). +- `libjim-dev` is required to configure mainline; all build deps are already installed on ci (mainline was built here 2026-07-27). +- Back up before replacing `/usr/local/bin/openocd`; the current binary is the RPi-fork build (byte-identical to `~/app/openocd_rpi/src/openocd`). +- Do not modify the TinyUSB checkout at `~/code/tinyusb` except where a task explicitly says so (hil_test.py WCH cfg template, on a `claude/`-prefixed branch). Never `git stash -u` in a TinyUSB worktree. +- OpenOCD resolves its scripts dir relative to the **realpath** of the binary — repoint via symlink into an installed prefix, never a bare copy of the binary. + +## Reference: current state (measured 2026-07-27, in `OPENOCD_UNIFIED_FORK_HANDOFF.md`) + +| Tree on ci | Repo @ commit | Role | +| --- | --- | --- | +| `~/app/openocd_rpi` | raspberrypi/openocd @ `ebec9504d` (sdk-2.0.0) | rig default (`/usr/local/bin/openocd`) | +| `~/app/openocd_adi` | analogdevicesinc/openocd @ `5fc33af` | max32666fthr (`~/app/openocd_adi/src/openocd`) | +| `~/app/riscv-openocd-wch` | hathach/riscv-openocd-wch @ `ccb04d7` | CH32 flash+RTT (`~/.local/bin/openocd_wch`) | +| `~/app/openocd-mainline` | openocd-org/openocd @ `43441cd83` | candidate build, verified on pico/pico2/max32666fthr | + +Rig flasher entries (`test/hil/tinyusb.json`): `openocd` (pico ×3, fruit_jam, stm32h743nucleo, stm32g0b1nucleo), `openocd_adi` (max32666fthr), `openocd_wch` (nanoch32v203, ch32v103r_r1_1v0, ch32v307v_r1_1v0, ch582m_evt), `esptool` (espressif_s3_devkitm, espressif_p4_function_ev). + +--- + +### Task 1: Create `hathach/openocd`, `tinyusb` branch, README + +**Files:** +- Create: `~/app/openocd/` (clone), `~/app/openocd/README.md` + +**Interfaces:** +- Produces: GitHub repo `hathach/openocd` with default branch `tinyusb`; local clone `~/app/openocd` with remotes `origin` (hathach) and `upstream` (openocd-org). All later tasks commit to this clone's `tinyusb` branch. + +- [ ] **Step 1: Fork and clone** + +```bash +gh repo fork openocd-org/openocd --clone=false +git clone --recursive https://github.com/hathach/openocd.git ~/app/openocd +cd ~/app/openocd +git remote add upstream https://github.com/openocd-org/openocd.git +git checkout -b tinyusb origin/master +``` + +- [ ] **Step 2: Verify the clone is at mainline HEAD** + +Run: `cd ~/app/openocd && git log --oneline -1` +Expected: `43441cd83 server: add 'services' command to list service information` or newer. + +- [ ] **Step 3: Write `README.md`** (new file — GitHub renders it instead of mainline's plain-text `README`, and leaving `README` untouched keeps future rebases conflict-free) + +```markdown +# OpenOCD for the TinyUSB test rig + +One OpenOCD build that flashes, debugs and RTT-captures every board family on +the [TinyUSB](https://github.com/hathach/tinyusb) hardware-in-the-loop rig, so +the rig does not need four different OpenOCD trees. + +This is the `tinyusb` branch, tracking +[openocd-org/openocd](https://github.com/openocd-org/openocd) `master`. +Everything not listed below is unmodified mainline. + +## Cherry-picked / ported from + +| Source repo | What we took | +| --- | --- | +| [raspberrypi/openocd](https://github.com/raspberrypi/openocd) (`sdk-2.0.0`) | `tcl/target/rp2350-riscv.cfg`, `rp2350-rescue.cfg`, `rp2350-dbgkey-secure.cfg`, `rp2350-dbgkey-nonsecure.cfg`. The RP2040/RP2350 C flash driver is already better in mainline (`rp2xxx.c`). | +| [analogdevicesinc/openocd](https://github.com/analogdevicesinc/openocd) (`release`) | `tcl/target/max32665.cfg` (MAX32665/MAX32666), re-ported onto mainline's `max32xxx_common.cfg`. The fork's QSPI block is dropped — it is guarded by `QSPI_ENABLE`, which this part sets to 0. | +| [hathach/riscv-openocd-wch](https://github.com/hathach/riscv-openocd-wch) (originally [dragonlock2/miscboards](https://github.com/dragonlock2/miscboards) WCH SDK) | `wlinke` adapter driver, `sdi` single-wire transport, and the WCH flash drivers (`wch_riscv`, `wch_arm`) for CH32V/CH32F/CH5xx over WCH-Link/LinkE. | +| [espressif/openocd-esp32](https://github.com/espressif/openocd-esp32) | `tcl/target/esp32p4.cfg` + `tcl/board/esp32p4-builtin.cfg`, adapted to mainline's generic RISC-V ESP pattern. ESP32/S2/S3/C3/C6/H2 debug is already in mainline; ESP flash programming stays with `esptool`. | + +## Build + + ./bootstrap + ./configure --enable-jlink --enable-cmsis-dap --enable-stlink \ + --enable-wlinke --disable-werror + make -j$(nproc) + +`libjim-dev` is required — mainline no longer builds the bundled jimtcl by +default and configure hard-fails without it. +``` + +- [ ] **Step 4: Commit, push, set default branch** + +```bash +cd ~/app/openocd +git add README.md +git commit -m "README: purpose of the tinyusb branch and its downstream sources" +git push -u origin tinyusb +gh repo edit hathach/openocd --default-branch tinyusb \ + --description "OpenOCD for the TinyUSB test rig - one build for RP2040/RP2350, WCH CH32, MAX32 and ESP32 targets" +``` + +- [ ] **Step 5: Verify default branch** + +Run: `gh repo view hathach/openocd --json defaultBranchRef -q .defaultBranchRef.name` +Expected: `tinyusb` + +--- + +### Task 2: Build the fork on ci + +**Files:** +- Create: `~/app/openocd_tinyusb/` (install prefix) + +**Interfaces:** +- Consumes: `~/app/openocd` clone from Task 1. +- Produces: `~/app/openocd_tinyusb/bin/openocd` (installed binary + scripts at `~/app/openocd_tinyusb/share/openocd/scripts/`). Every later flash/verify step uses this path. + +- [ ] **Step 1: Configure and build** (same recipe that already worked for mainline on this box) + +```bash +cd ~/app/openocd +./bootstrap +./configure --prefix=$HOME/app/openocd_tinyusb \ + --enable-jlink --enable-cmsis-dap --enable-stlink --disable-werror +make -j$(nproc) && make install +``` + +- [ ] **Step 2: Verify version and adapters** + +Run: `~/app/openocd_tinyusb/bin/openocd --version 2>&1 | head -1` +Expected: `Open On-Chip Debugger 0.12.0+dev-...` with a `-g<sha>` matching `git -C ~/app/openocd rev-parse --short HEAD`. + +Run: `~/app/openocd_tinyusb/bin/openocd -c 'adapter list; shutdown' 2>&1 | grep -E 'cmsis-dap|jlink|stlink'` +Expected: all three listed. + +*(No commit — build products only.)* + +--- + +### Task 3: Import the 5 TCL configs — one commit per downstream fork + +**Files:** +- Create: `~/app/openocd/tcl/target/rp2350-riscv.cfg`, `rp2350-rescue.cfg`, `rp2350-dbgkey-secure.cfg`, `rp2350-dbgkey-nonsecure.cfg`, `max32665.cfg` +- Source of truth: `~/code/tinyusb/openocd-unified-configs/` (the copies already hardware-verified this week; the max32665 port is already written there) + +**Interfaces:** +- Consumes: `~/app/openocd` + install prefix from Task 2. +- Produces: `target/rp2350-riscv.cfg` and `target/max32665.cfg` resolvable via `find` in the installed scripts dir — Task 4 flashes with them. + +- [ ] **Step 1: Copy the RPi configs and commit (downstream commit #1)** + +```bash +cd ~/app/openocd +cp ~/code/tinyusb/openocd-unified-configs/rp2350-riscv.cfg \ + ~/code/tinyusb/openocd-unified-configs/rp2350-rescue.cfg \ + ~/code/tinyusb/openocd-unified-configs/rp2350-dbgkey-secure.cfg \ + ~/code/tinyusb/openocd-unified-configs/rp2350-dbgkey-nonsecure.cfg \ + tcl/target/ +git add tcl/target/rp2350-*.cfg +git commit -m "tcl/target: add RP2350 riscv/rescue/dbgkey configs from raspberrypi/openocd + +Taken from raspberrypi/openocd branch sdk-2.0.0 @ ebec9504d. These four +configs are the only things that fork has which mainline lacks - the +rp2040/rp2350 C driver was consolidated upstream as rp2xxx.c. All four +use only mainline-present commands (swj_newdap, dap create -adiv6, +target create riscv -ap-num, riscv set_enable_virt2phys). + +rp2350-riscv.cfg is what hw/bsp/rp2040/family.cmake requests when +PICO_PLATFORM=rp2350-riscv." +``` + +- [ ] **Step 2: Copy the ADI config and commit (downstream commit #2)** + +```bash +cd ~/app/openocd +cp ~/code/tinyusb/openocd-unified-configs/max32665.cfg tcl/target/ +git add tcl/target/max32665.cfg +git commit -m "tcl/target: add max32665 config ported from analogdevicesinc/openocd + +Ported from analogdevicesinc/openocd @ 5fc33af onto mainline's +max32xxx_common.cfg (the ADI fork calls the same file max32xxx.cfg). +The fork's QSPI block is dropped: it is guarded by QSPI_ENABLE, which +this part sets to 0, and it needs the ADI-only max32xxx_qspi driver. +Covers MAX32665/MAX32666 (both flash banks). Hardware-verified on +max32666fthr 2026-07-27." +``` + +- [ ] **Step 3: Install and verify the configs resolve** + +```bash +cd ~/app/openocd && make install +~/app/openocd_tinyusb/bin/openocd -c 'puts [find target/max32665.cfg]; puts [find target/rp2350-riscv.cfg]; shutdown' +``` +Expected: both paths under `~/app/openocd_tinyusb/share/openocd/scripts/target/` printed; exit without "Can't find". + +- [ ] **Step 4: Push** + +```bash +cd ~/app/openocd && git push +``` + +--- + +### Task 4: Hardware-verify every current-openocd board with the fork binary + +**Files:** +- No source changes. Uses `~/code/tinyusb` builds + `test/hil/hil_test.py`. + +**Interfaces:** +- Consumes: `~/app/openocd_tinyusb/bin/openocd` with Task 3 configs installed. +- Produces: evidence that the fork can replace `/usr/local/bin/openocd` (Task 5's gate). PATH shim dir `~/app/openocd_tinyusb/shim/` reused by later tasks. + +Boards (every `openocd`/`openocd_adi` flasher entry in `tinyusb.json`): +`raspberry_pi_pico`, `raspberry_pi_pico_w`, `raspberry_pi_pico2`, `adafruit_fruit_jam`, `stm32h743nucleo`, `stm32g0b1nucleo`, `max32666fthr`. +Already verified on plain mainline 2026-07-27: pico, pico2, max32666fthr (re-run anyway — the binary changed). + +- [ ] **Step 1: Build any missing firmware sets** (repeat per board without `examples/cmake-build-<board>`; `cmake-build-raspberry_pi_pico`, `-stm32g0b1nucleo`, `-max32666fthr` already exist) + +```bash +cd ~/code/tinyusb/examples +cmake -B cmake-build-raspberry_pi_pico2 -DBOARD=raspberry_pi_pico2 -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel . \ + && cmake --build cmake-build-raspberry_pi_pico2 +``` +(Same pattern for `raspberry_pi_pico_w`, `adafruit_fruit_jam`, `stm32h743nucleo`. If a board fails `get_deps`, run `python3 tools/get_deps.py -b <board>` first.) + +- [ ] **Step 2: Create the PATH shim** (lets `hil_test.py`'s hardcoded `openocd` resolve to the fork; symlink keeps scripts-dir resolution working because OpenOCD follows the realpath) + +```bash +mkdir -p ~/app/openocd_tinyusb/shim +ln -sf ~/app/openocd_tinyusb/bin/openocd ~/app/openocd_tinyusb/shim/openocd +``` + +- [ ] **Step 3: Smoke-flash one board directly** (fast signal before the full suite) + +```bash +python3 ~/code/tinyusb/test/hil/board_lock.py hold raspberry_pi_pico --reason "openocd-unified verify" +~/app/openocd_tinyusb/bin/openocd -c "adapter serial E6614103E72C1D2F" \ + -f interface/cmsis-dap.cfg -f target/rp2040.cfg -c "adapter speed 5000" \ + -c "program /home/hathach/code/tinyusb/examples/cmake-build-raspberry_pi_pico/device/cdc_msc/cdc_msc.elf verify reset exit" +``` +Expected: `** Verified OK **` then `** Resetting Target **`. Release the lock after (`board_lock.py release raspberry_pi_pico`). + +- [ ] **Step 4: Run the HIL suite for all 7 boards through the shim** + +```bash +cd ~/code/tinyusb +PATH=~/app/openocd_tinyusb/shim:$PATH \ +python3 test/hil/hil_test.py test/hil/tinyusb.json \ + -b raspberry_pi_pico -b raspberry_pi_pico_w -b raspberry_pi_pico2 \ + -b adafruit_fruit_jam -b stm32h743nucleo -b stm32g0b1nucleo +``` +Notes for the executor: +- `hil_test.py` takes the config as a positional arg and `-b` per board; it holds board locks itself (that is the board-lock protocol in CI — do not also hold manual locks around `hil_test.py` runs). +- max32666fthr is **not** in this run: its `flash_openocd_adi()` path uses the hardcoded `OPENCOD_ADI_PATH = ~/app/openocd_adi` (`hil_test.py:408`), which the shim can't intercept. Handle it in Step 4b instead. Do not edit `hil_test.py` for this — the adi path disappears at cutover (Task 10 flips `tinyusb.json`'s flasher entry to plain `openocd` with `-f interface/cmsis-dap.cfg -f target/max32665.cfg`). +- Expected: every board PASS in the report. Any failure: stop, diagnose (consult the `hil` skill), do not proceed to Task 5. + +- [ ] **Step 4b: max32666fthr — manual flash with the fork, then tests with `--skip-flash`** + +```bash +python3 ~/code/tinyusb/test/hil/board_lock.py hold max32666fthr --reason "openocd-unified verify" +~/app/openocd_tinyusb/bin/openocd -c "adapter serial E6614C311B597D32" \ + -f interface/cmsis-dap.cfg -f target/max32665.cfg \ + -c "program /home/hathach/code/tinyusb/examples/cmake-build-max32666fthr/device/cdc_msc/cdc_msc.elf verify reset exit" +python3 ~/code/tinyusb/test/hil/board_lock.py release max32666fthr +cd ~/code/tinyusb && python3 test/hil/hil_test.py test/hil/tinyusb.json -b max32666fthr -sf +``` +Expected: `** Verified OK **` on the flash, then PASS with `-sf` (tests run against the firmware just flashed). + +- [ ] **Step 5: RTT smoke on the pico** (mainline RTT was verified 2026-07-27; re-confirm on the fork build — `target-debug` skill has the full flow) + +Expected: RTT control block found, events stream, overflow 0. + +--- + +### Task 5: Repoint the rig default `openocd` + +**Files:** +- Modify: `/usr/local/bin/openocd` (→ symlink), remove Debian `openocd` package + +**Interfaces:** +- Consumes: Task 4 all-green. +- Produces: `which openocd` → fork for every rig user (hil_test.py, skills, CI). Rollback: restore `/usr/local/bin/openocd.rpi-backup-20260727`. + +- [ ] **Step 1: Back up and repoint** + +```bash +sudo cp -a /usr/local/bin/openocd /usr/local/bin/openocd.rpi-backup-20260727 +sudo ln -sf $HOME/app/openocd_tinyusb/bin/openocd /usr/local/bin/openocd +openocd --version 2>&1 | head -1 +``` +Expected: fork version string (matches Task 2 Step 2). + +- [ ] **Step 2: Drop the Debian openocd** (installed 2026-07-27 only to get a jlink-capable OpenOCD; the fork has `--enable-jlink`) + +```bash +sudo apt-get remove -y openocd +which -a openocd +``` +Expected: only `/usr/local/bin/openocd` remains. + +- [ ] **Step 3: Re-verify through the default path (no shim)** + +```bash +cd ~/code/tinyusb +python3 test/hil/hil_test.py test/hil/tinyusb.json -b raspberry_pi_pico -b stm32g0b1nucleo -b raspberry_pi_pico2 +``` +Expected: 3/3 PASS. If CI kicks a workflow mid-way, board locks arbitrate — just wait. + +--- + +### Task 6: WCH part 1 — port the `wlinke` adapter + `sdi` transport (compiles, detects probe) + +**Files (all in `~/app/openocd`, sources from `~/app/riscv-openocd-wch` @ `ccb04d7` — this copy already carries the GCC-14 fixes):** +- Create: `src/jtag/drivers/wlinke.c` (2041 lines, copy), `src/jtag/sdi.c` (~130 lines, port), `src/jtag/sdi.h` (if the fork has one — check `ls ~/app/riscv-openocd-wch/src/jtag/sdi*`) +- Modify: `src/transport/transport.h` (new transport id), `src/jtag/interface.h` (add `sdi_ops` to `struct adapter_driver` + `struct sdi_driver` decl), `src/jtag/interfaces.c` (register driver), `src/jtag/drivers/Makefile.am`, `src/jtag/Makefile.am`, `configure.ac` (`--enable-wlinke`) + +**Interfaces:** +- Consumes: fork clone + build tree. +- Produces: `openocd -c "adapter driver wlinke"` works; `wlink_*` C exports (`wlink_erase`, `wlink_write`, `wlink_getromram`, `wlink_reset`, `wlink_chip_reset`, `wlink_clean`, `wlink_flash_protect`, …) available for Task 8's flash driver; `sdi` transport selectable. Commit stays **amend-in-progress** — Tasks 6–8 squash into downstream commit #3. + +Port notes gathered up front (verified against both trees 2026-07-27): +- Fork wiring to replicate: `configure.ac:117` (adapter list entry `[[wlinke],[WLINKE Programmer],[WLINKE]]`), `:284-286` (`AC_ARG_ENABLE`), `:537`, `:737` (`AM_CONDITIONAL`); `src/jtag/drivers/Makefile.am:189` (`DRIVERFILES += %D%/wlinke.c`); `src/jtag/interfaces.c:154,274` (extern + table entry). +- Mainline transports are now a **fixed bitmask enum** (`src/transport/transport.h:19-25`: `TRANSPORT_JTAG BIT(0)` … `TRANSPORT_SWIM BIT(6)`, plus `TRANSPORT_VALID_MASK`), and `struct transport` selects by `unsigned int id`, not name. Add `#define TRANSPORT_SDI BIT(7)`, extend `TRANSPORT_VALID_MASK`, and port `sdi.c`'s `transport_register` to the id-based struct. +- **SWIM is the exact precedent** — ST's proprietary single-wire transport, wired upstream the same way this needs: `swim_ops` field at `src/jtag/interface.h:363`, its own transport bit, own command namespace. Mirror how `grep -rn swim src/transport/ src/jtag/interface.h src/jtag/swim.c` is structured wherever the fork's 0.11-era pattern no longer matches mainline. +- The fork's `sdi` op is a raw RISC-V DMI transfer: `adapter_driver->sdi_ops->transfer(iIndex, iAddr, iData, iOP, oAddr, oData, oOP)` (`src/jtag/sdi.c:20-22`) — keep that signature; Task 7 builds on it. +- `wlinke.c` includes `"cmsis_dap.h"`, `"hidapi.h"`, `"libusb_helper.h"` and (spuriously) `<windows.h>` — drop/guard the windows include; hidapi + libusb helpers exist in mainline's drivers dir. + +- [ ] **Step 1: Copy `wlinke.c` and `sdi.c` in; make the wiring edits above** + +- [ ] **Step 2: Reconfigure with wlinke and build** + +```bash +cd ~/app/openocd +./configure --prefix=$HOME/app/openocd_tinyusb \ + --enable-jlink --enable-cmsis-dap --enable-stlink --enable-wlinke --disable-werror +make -j$(nproc) && make install +``` +Expected: clean build (`--disable-werror` tolerates the fork's warning-dirty code; do fix outright errors). + +- [ ] **Step 3: Probe-detection test against real hardware** (nanoch32v203's WCH-LinkE, serial `EBCA8F0670AF`) + +```bash +python3 ~/code/tinyusb/test/hil/board_lock.py hold nanoch32v203 --reason "wlinke port bring-up" +~/app/openocd_tinyusb/bin/openocd -c "adapter driver wlinke" \ + -c "adapter serial EBCA8F0670AF" -c "transport select sdi" \ + -c "init" -c "shutdown" +``` +Expected: log lines identifying the WCH-Link probe (firmware version print from `wlink_init`), no crash. `init` may complain about missing target — probe identification is the pass signal. Keep the lock held into Task 7 (same board). + +- [ ] **Step 4: Snapshot as work-in-progress commit** (will be amended/squashed through Task 8) + +```bash +cd ~/app/openocd && git add -A && git commit -m "WIP: wch port (squash into single downstream commit before push)" +``` +**Do not push** until Task 8 squashes. + +--- + +### Task 7: WCH part 2 — target spike: mainline `riscv` over wlink DMI + +**The hypothesis (from the handoff, sharpened by code reading):** WCH-LinkE's `sdi` op *is* a raw DMI transfer, and mainline's riscv-013 target is just a DMI client. If mainline's riscv target can be fed by wlink DMI transfers, we skip porting `wch_riscv.c`/`wch_riscv-013.c` (~3.5k lines that `#include <target/riscv/...>` 0.11-era internals — the worst possible port surface). + +**Files:** +- Modify: `src/jtag/drivers/wlinke.c` (add the DTM bridge), possibly `src/target/riscv/riscv-013.c` shim hooks — decided by Step 1's reading. + +**Interfaces:** +- Consumes: Task 6's working adapter (lock on nanoch32v203 still held). +- Produces: a `target create ... riscv` (or, on fallback, `wch_riscv`) config shape that Task 8's flash/RTT/HIL work builds on. Records the decision in the WIP commit message. + +- [ ] **Step 1: Read mainline's DMI plumbing before writing anything** + +Read `src/target/riscv/riscv-013.c` (the `dmi_op`/`riscv_batch` layer) and `src/target/riscv/riscv.c`'s `riscv dmi_read`/`dmi_write` command handlers (they exist — mainline's `tcl/target/esp32c6.cfg` calls them). Determine the narrowest insertion point, in order of preference: +1. an existing DTM/DMI abstraction the adapter can implement directly (best); +2. a jtag-DTM emulation inside `wlinke.c`: expose `jtag_ops` whose queue executor decodes IR=DTMCS/DMI DR scans into `sdi` transfers (the esp_usb_jtag-style approach, one level up); +3. nothing viable → fallback (Step 4). + +- [ ] **Step 2: Implement the chosen bridge; build** + +Same build command as Task 6 Step 2. + +- [ ] **Step 3: Hypothesis test on nanoch32v203** (write the test cfg to the scratchpad, not the repo) + +```tcl +# wch-mainline-riscv-test.cfg +adapter driver wlinke +adapter speed 6000 +transport select sdi ;# or jtag, if Step 1 chose the jtag-DTM emulation +wlink_set_address 0x00000000 +sdi newtap ch32 cpu -irlen 5 -expected-id 0x00001 +target create ch32.cpu riscv -chain-position ch32.cpu +ch32.cpu configure -work-area-phys 0x20000000 -work-area-size 0x2800 -work-area-backup 1 +init +``` + +Evidence criteria — **all four must hold** to call the hypothesis confirmed: +``` +halt → "Target halted" with a sane pc +riscv dmi_read 0x11 → plausible dmstatus (nonzero, version field = 2 or 3) +mdw 0x20000000 4 → reads SRAM without error +resume → target runs again (LED blink / CDC re-enumerates) +``` + +- [ ] **Step 4: Decision checkpoint — STOP if the hypothesis fails** + +If any criterion fails for reasons that look architectural (wlink protocol can't express raw DMI reads, QingKe deviates from the RISC-V debug spec in ways mainline won't tolerate), **stop and report to the user** with the evidence. The two fallback options, costed: +- (a) Port the fork's full WCH target stack: `src/target/wch_riscv.c` (3033 ln) + `wch_riscv-013.c` + `wch_riscv.h`, plus the fork's core patches (all findable via `grep -rn 'riscvchip\|wlink_' src/` in the fork: `src/flash/nor/tcl.c` 5 hits, `src/target/target.c` 5, `src/server/gdb_server.c` 2). Hard: these files include 0.11-era `target/riscv/*` headers that clash with mainline's current riscv internals. +- (b) Ship the unified fork **without** WCH C support and keep `openocd_wch` as the rig's CH32 flasher indefinitely. +Do not silently pick (a). + +--- + +### Task 8: WCH part 3 — flash drivers, RTT, 4-board HIL green, squash to downstream commit #3 + +**Files:** +- Create: `src/flash/nor/wchriscv.c` (324 ln, copy), `src/flash/nor/wcharm.c` (897 ln, copy — CH32F ARM parts; self-contained memory-mapped driver, zero wlink deps), `src/jtag/drivers/wlinke.h` (new — prototypes for the `wlink_*` exports; the fork relied on implicit declarations) +- Modify: `src/flash/nor/drivers.c` (extern + table entries, fork pattern at its lines 93-94/170-171), `src/flash/nor/Makefile.am` (fork pattern at lines 78-79) +- Modify (TinyUSB repo, separate branch): `test/hil/hil_test.py` WCH cfg template (~line 381) — only if Task 7 landed on the mainline-riscv target shape + +**Interfaces:** +- Consumes: Task 7's confirmed target shape + `wlink_*` exports from Task 6. +- Produces: downstream commit #3 (single squashed commit, pushed); `~/.local/bin/openocd_wch` repointed at the fork; hil_test.py template branch `claude/hil-openocd-unified` in the TinyUSB repo (unpushed — user pushes; "hold pushes" applies to the TinyUSB repo). + +- [ ] **Step 1: Copy the flash drivers, add `wlinke.h`, wire `drivers.c`/`Makefile.am`; build** + +Keep the flash driver's registered name **`wch_riscv`** — the rig's generated per-probe cfg does `flash bank ... wch_riscv ...` and Task 8 Step 4's template keeps working. +Fork quirk to *not* copy: the fork patched `src/flash/nor/tcl.c` (`handle_flash_protect_check_command`, its line ~414) to call `wlink_softreset()`/`wlnik_protect_check()` for WCH banks. Implement that inside `wchriscv.c`'s own `protect_check` op instead — no core-file patch. +Check the fork's `src/server/gdb_server.c` 2 `wlink_` hits (`grep -n 'riscvchip\|wlink_' ~/app/riscv-openocd-wch/src/server/gdb_server.c`) — port the behavior into the driver/target layer if it matters for our flow (flash + RTT, no gdb needed on the rig for WCH), else document-and-skip in the commit message. + +- [ ] **Step 2: Flash test on nanoch32v203** (lock held; cfg = Task 7's test cfg + flash bank line) + +```tcl +set _FLASHNAME ch32.flash +flash bank $_FLASHNAME wch_riscv 0x00000000 0 0 0 ch32.cpu +``` +```bash +~/app/openocd_tinyusb/bin/openocd -c "adapter serial EBCA8F0670AF" \ + -f wch-mainline-riscv-test.cfg \ + -c "program /home/hathach/code/tinyusb/examples/cmake-build-nanoch32v203-usbfs/device/cdc_msc/cdc_msc.elf verify reset exit" +``` +Expected: `** Verified OK **`; board re-enumerates as CDC (`lsusb | grep -i cafe` or dmesg). + +- [ ] **Step 3: RTT test on nanoch32v203** (rig rule: `rtt polling_interval 1`, **never `reset run`**) + +RTT server start → capture a few seconds → nonzero events. The `target-debug` skill documents the WCH RTT route. + +- [ ] **Step 4: Update the rig's WCH flow** + +If Task 7 confirmed the mainline-riscv shape, the generated cfg template in `test/hil/hil_test.py` (~line 381: `adapter driver wlinke` … `target create $_TARGETNAME.0 wch_riscv …`) must switch to the Task 7 cfg shape. Do this on a TinyUSB branch: +```bash +cd ~/code/tinyusb && git worktree add .worktrees/claude/hil-openocd-unified -b claude/hil-openocd-unified +# edit test/hil/hil_test.py template in the worktree; commit there; DO NOT push +``` +Then repoint the rig's WCH binary (symlink, so scripts resolve): +```bash +ln -sf ~/app/openocd_tinyusb/bin/openocd ~/.local/bin/openocd_wch +``` +(Old target `~/app/openocd_wch_new/bin/…` and `~/.local/bin/openocd_wch.bak-20260727` stay as rollback.) + +- [ ] **Step 5: HIL green on all four WCH boards** (run from the worktree so the new template is used) + +```bash +cd ~/code/tinyusb/.worktrees/claude/hil-openocd-unified +python3 test/hil/hil_test.py test/hil/tinyusb.json \ + -b nanoch32v203 -b ch32v103r_r1_1v0 -b ch32v307v_r1_1v0 -b ch582m_evt +``` +Expected: 4/4 PASS. Firmware for missing `cmake-build-<board>` sets: build first (nanoch32v203 sets exist; ch32v103/307/ch582m may need `tools/get_deps.py -b <board>` + the examples build). Known flake: ch32v103r throughput is ~40% flaky historically — retry before blaming the port. If ch582m misbehaves specifically, note it and check `wlinke.c`'s riscvchip dispatch for CH58x. + +- [ ] **Step 6: Squash Tasks 6–8 into downstream commit #3 and push** + +```bash +cd ~/app/openocd +git reset --soft $(git log --grep='WIP: wch port' --format=%H | tail -1)^ +git commit -m "jtag, flash: add WCH-LinkE adapter, sdi transport and CH32 flash drivers + +Ported from hathach/riscv-openocd-wch @ ccb04d7 (originally +dragonlock2/miscboards WCH SDK, base openocd 0.11.0): +- src/jtag/drivers/wlinke.c: WCH-Link/LinkE USB adapter (GCC-14 fixes included) +- src/jtag/sdi.c: WCH single-wire debug transport, re-worked onto + mainline's id-based transport API (TRANSPORT_SDI) +- src/flash/nor/wchriscv.c, wcharm.c: CH32V/CH5xx (wlink protocol) and + CH32F (memory-mapped) flash drivers +CH32 cores are driven by mainline's riscv target over wlink DMI +transfers; the fork's wch_riscv target stack is not needed. +The fork's core patches (flash/nor/tcl.c protect-check hack) moved into +the wch_riscv flash driver's protect_check op. + +Verified on ci rig: nanoch32v203, ch32v103r_r1_1v0, ch32v307v_r1_1v0, +ch582m_evt - flash + verify + HIL suite + RTT (nanoch32v203)." +git push +``` +(Amend the target-stack paragraph if the fallback path was taken instead.) +Release the nanoch32v203 lock if still held. + +--- + +### Task 9: Espressif — ESP32-P4 configs, S3 attach verification, downstream commit #4 + +Mainline already has: `src/target/espressif/` (esp32/s2/s3 xtensa targets + apptrace/semihosting), the `esp_usb_jtag` adapter driver, and builtin cfgs for c2/c3/c6/h2/s3. Missing vs the rig: anything ESP32-P4. Flash stays esptool (rig flashes ESP via `idf.py`/esptool; the espressif fork's flash-stub stack is explicitly out of scope). + +**Files:** +- Create: `~/app/openocd/tcl/target/esp32p4.cfg`, `~/app/openocd/tcl/board/esp32p4-builtin.cfg` + +**Interfaces:** +- Consumes: install prefix; espressif fork cfgs fetched from GitHub. +- Produces: downstream commit #4; P4 + S3 debug-attach evidence. + +- [ ] **Step 1: Verify S3 attach with pure mainline inheritance** (no new files; proves the "espressif support" baseline) + +```bash +python3 ~/code/tinyusb/test/hil/board_lock.py hold espressif_s3_devkitm --reason "openocd-unified esp verify" +~/app/openocd_tinyusb/bin/openocd -f board/esp32s3-builtin.cfg -c "init; halt" +``` +Expected: both xtensa cores detected over USB-Serial-JTAG (303a:1001), `Target halted`. Then `resume; shutdown`, release lock. Gotchas live in the `esp-target-debug` skill (S3's debug port can be occupied when TinyUSB firmware owns the USB peripheral — use the same recovery steps as that skill). + +- [ ] **Step 2: Fetch and adapt the P4 configs (write both files)** + +```bash +curl -fsSL https://raw.githubusercontent.com/espressif/openocd-esp32/master/tcl/target/esp32p4.cfg -o /tmp/claude-1000/-home-hathach-code-tinyusb/7dee5f9e-874b-4680-bb09-01a5d13fbd37/scratchpad/esp32p4-espressif.cfg +curl -fsSL https://raw.githubusercontent.com/espressif/openocd-esp32/master/tcl/board/esp32p4-builtin.cfg -o /tmp/claude-1000/-home-hathach-code-tinyusb/7dee5f9e-874b-4680-bb09-01a5d13fbd37/scratchpad/esp32p4-builtin-espressif.cfg +``` +Espressif's cfg creates an `esp32p4`-type target (their `esp_riscv` C stack — not in mainline). Rewrite `tcl/target/esp32p4.cfg` following **mainline's own ESP RISC-V pattern** — `tcl/target/esp32c6.cfg` + `esp_common.cfg` (generic `riscv` target create, chip quirks via `riscv dmi_write` with the `_RISCV_*` register constants from `esp_common.cfg`) — carrying over from Espressif's file: `_CPUTAPID`, memory map/workarea, the dual-core SMP topology (P4 is 2× RV32 — model on how mainline handles SMP, and on Espressif's `_ESP_SMP_TARGET`), and the `_ESP_EFUSE_MAC_ADDR_REG` value. `tcl/board/esp32p4-builtin.cfg` = `esp_usb_jtag` adapter + `transport select jtag` + source the target cfg (mirror `board/esp32c6-builtin.cfg`, adjusting `ESP_USB_JTAG_*` ids to Espressif's P4 values). +Also check `src/jtag/drivers/esp_usb_jtag.c` accepts the P4 (VID/PID 303a:1001 is shared; verify any chip-id gating). + +- [ ] **Step 3: P4 attach test** + +```bash +python3 ~/code/tinyusb/test/hil/board_lock.py hold espressif_p4_function_ev --reason "openocd-unified esp verify" +cd ~/app/openocd && make install +~/app/openocd_tinyusb/bin/openocd -f board/esp32p4-builtin.cfg -c "init; halt" +``` +Evidence criteria: both HP cores halt, `mdw 0x4ff00000 4` (P4 HP TCM/SRAM — cross-check the address against Espressif's cfg memory map before running) reads, `resume` works. Known nuance from prior sessions: P4 attach can need the reset-into-attach dance — the `esp-target-debug` skill documents it; an attach that only works with that dance still counts as pass (note it in the commit). +**Decision checkpoint:** if the generic-riscv shape cannot attach P4 for architectural reasons (needs Espressif's C-level `esp_riscv` assist), stop and report — options are cherry-picking their `esp_riscv` stack (large) vs shipping P4 as esptool-flash-only with debug via ESP-IDF's openocd as today. Do not silently pick either. + +- [ ] **Step 4: Commit (downstream commit #4) and push** + +```bash +cd ~/app/openocd +git add tcl/target/esp32p4.cfg tcl/board/esp32p4-builtin.cfg +git commit -m "tcl: add ESP32-P4 target/board configs adapted from espressif/openocd-esp32 + +Adapted from espressif/openocd-esp32 master onto mainline's generic +RISC-V ESP pattern (tcl/target/esp32c6.cfg + esp_common.cfg): generic +riscv targets over esp_usb_jtag instead of the fork's esp_riscv C +stack. Flash programming stays with esptool, matching how the rig +flashes all Espressif boards. ESP32/S2/S3/C3/C6/H2 were already +supported by mainline. + +Verified on ci rig: espressif_p4_function_ev and espressif_s3_devkitm +attach/halt/resume over built-in USB-Serial-JTAG." +git push +``` +Release both ESP board locks. + +--- + +### Task 10: Final sweep, README truth-up, rig config flip + +**Files:** +- Modify: `~/app/openocd/README.md` (only if scope shifted in Tasks 7–9) +- Modify (TinyUSB worktree from Task 8): `test/hil/tinyusb.json` — max32666fthr flasher `openocd_adi` → `openocd` with args `-f interface/cmsis-dap.cfg -f target/max32665.cfg` (plain openocd now serves it) + +**Interfaces:** +- Consumes: everything green from Tasks 4–9. +- Produces: the finished fork; TinyUSB branch `claude/hil-openocd-unified` with hil_test.py + tinyusb.json changes, committed, **unpushed** (user pushes per standing instruction). + +- [ ] **Step 1: Full HIL regression across every openocd-family board** + +```bash +cd ~/code/tinyusb/.worktrees/claude/hil-openocd-unified +python3 test/hil/hil_test.py test/hil/tinyusb.json \ + -b raspberry_pi_pico -b raspberry_pi_pico_w -b raspberry_pi_pico2 \ + -b adafruit_fruit_jam -b stm32h743nucleo -b stm32g0b1nucleo -b max32666fthr \ + -b nanoch32v203 -b ch32v103r_r1_1v0 -b ch32v307v_r1_1v0 -b ch582m_evt +``` +Expected: 11/11 PASS (ch32v103r throughput may need its usual retries). + +- [ ] **Step 2: README truth-up** + +Re-read `README.md` against what actually landed (WCH target route, P4 outcome). Fix any row that no longer matches; amend into the README commit or add +`git commit -m "README: reflect verified scope"`. Push. + +- [ ] **Step 3: Verify the one-commit-per-fork shape** + +Run: `git -C ~/app/openocd log --oneline upstream/master..tinyusb` +Expected: exactly 5 commits (or 6 with a README truth-up): README, RPi configs, ADI config, WCH port, ESP32-P4 configs. If not, interactive-free cleanup: `git rebase --onto` / `reset --soft` re-squash, then `git push --force-with-lease` (fork branch, ours alone — safe). + +- [ ] **Step 4: Commit the TinyUSB-side changes in the worktree (do not push)** + +```bash +cd ~/code/tinyusb/.worktrees/claude/hil-openocd-unified +git add test/hil/hil_test.py test/hil/tinyusb.json +git commit -m "test(hil): drive WCH boards and max32666fthr through the unified openocd" +``` +Leave for the user to push/PR. + +- [ ] **Step 5: Leftovers report** (no deletions now) + +Write a short status into `OPENOCD_UNIFIED_FORK_HANDOFF.md` (append a "2026-07-XX outcome" section): what was repointed, rollback paths (`/usr/local/bin/openocd.rpi-backup-20260727`, `~/.local/bin/openocd_wch.bak-20260727`), and that `~/app/openocd_rpi`, `~/app/openocd_adi`, `~/app/openocd-mainline`, `~/app/openocd_mainline`, `~/app/openocd_wch_new`, `~/app/riscv-openocd-wch` can be retired **after a week of green CI** — not now. diff --git a/docs/superpowers/plans/2026-07-28-hil-test-split.md b/docs/superpowers/plans/2026-07-28-hil-test-split.md new file mode 100644 index 000000000..6b8528973 --- /dev/null +++ b/docs/superpowers/plans/2026-07-28-hil-test-split.md @@ -0,0 +1,355 @@ +# hil_test.py Split Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Split `test/hil/hil_test.py` (2370 ln) into a test-focused core plus `hil_lock.py` (board locks + controller permits + operator CLI, superseding `board_lock.py`) and `hil_flash.py` (run_cmd + flash backends + firmware/serial lookup), with no behavior change. + +**Architecture:** Pure code motion per `docs/superpowers/specs/2026-07-28-hil-test-refactor-design.md`. Import graph: `hil_test` → {`hil_lock`, `hil_flash`}; helpers import nothing local. Call sites use module-qualified names (`hil_lock.flash_permit(...)`), never wildcard mirroring. + +**Tech Stack:** Python 3.11+ (existing `TypedDict`/`NotRequired` usage), stdlib only in the helpers (fcntl, json, glob, multiprocessing objects passed in). + +## Global Constraints + +- Work in worktree `.claude/worktrees/hil-test-split` (branch `claude/hil-test-split`); never touch the primary checkout. +- Behavior-preserving: `hil_test.py` CLI args, log lines, report format, lock/permit semantics, flash behavior all byte-identical. The ONLY user-visible change is the CLI filename `board_lock.py` → `hil_lock.py`. +- Moved functions are moved **verbatim** — no reformatting, no comment editing, no "improvements". A diff of a moved function's body against its old self must be empty. +- Commit messages: imperative, scoped, no Co-Authored-By/Claude-Session trailers. +- Every commit leaves the tree working: `python3 -m py_compile` clean on all touched modules, and `python3 .claude/skills/hil/pool_check.py --scan-only` exits 0 (safe on the rig: scan-only takes no locks, flashes nothing). +- Hardware steps (Task 4) run on the `ci` rig only, from this worktree, and rely on the tools' own board flocks — never pre-hold boards you are about to run `hil_test.py`/`pool_check.py` on. + +--- + +### Task 1: Create hil_flash.py; repoint hil_test + pool_check flash call sites + +**Files:** +- Create: `test/hil/hil_flash.py` +- Modify: `test/hil/hil_test.py` (delete moved code; add import; qualify call sites) +- Modify: `.claude/skills/hil/pool_check.py` (flash-related imports) +- Modify: `test/hil/hil_ci.sh` (scp list) + +**Interfaces:** +- Produces (used by Tasks 2-4): module `hil_flash` with `CMD_TIMEOUT`, `run_cmd(cmd, cwd=None, timeout=CMD_TIMEOUT)`, `cmd_stdout_text(out)`, `OPENCOD_ADI_PATH`, `TINYUSB_ROOT`, `flash_jlink/reset_jlink`, `flash_stlink/reset_stlink`, `flash_stflash/reset_stflash`, `flash_openocd/reset_openocd`, `flash_openocd_wch/reset_openocd_wch`, `flash_openocd_adi/reset_openocd_adi`, `flash_wlink_rs/reset_wlink_rs`, `flash_esptool/reset_esptool`, `flash_uniflash/reset_uniflash`, `flash_lm4flash/reset_lm4flash`, `find_firmware(variant, example)`, `get_serial_dev(id, vendor_str, product_str, ifnum)`, module globals `build_dir = 'cmake-build'`, `verbose = False`. + +- [ ] **Step 1: Create `test/hil/hil_flash.py`** + +Header (new code), then the moved blocks verbatim: + +```python +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +# Firmware flashing for the TinyUSB HIL rig: run_cmd, one flash_*/reset_* pair per +# flasher type (dispatched by config name via getattr), find_firmware, and the +# fixture serial-port resolver get_serial_dev (here, not hil_test: flash_esptool +# needs it and helpers must not import hil_test). +# Callers set module globals `build_dir` and `verbose` (hil_test.main from argparse, +# pool_check directly) exactly as they set hil_test's globals today. + +import glob +import json +import os +import signal +import subprocess +import sys +from pathlib import Path + +verbose = False +build_dir = 'cmake-build' +``` + +Then MOVE (cut from `hil_test.py`, paste unchanged, in this order): +1. `CMD_TIMEOUT = int(os.getenv('HIL_CMD_TIMEOUT', '180'))` (from the constants block; leave `POOL_TIMEOUT`/`SERIAL_*_TIMEOUT` in hil_test) +2. `def cmd_stdout_text(out)` +3. `OPENCOD_ADI_PATH = Path.home() / 'app' / 'openocd_adi'` and `TINYUSB_ROOT = Path(__file__).resolve().parents[2]` +4. `def get_serial_dev(id, vendor_str, product_str, ifnum)` +5. `def run_cmd(cmd, cwd=None, timeout=CMD_TIMEOUT)` +6. All ten `flash_*`/`reset_*` pairs listed in Interfaces, in current file order +7. `def find_firmware(variant, example)` + +- [ ] **Step 2: Delete the moved code from `hil_test.py` and qualify call sites** + +In `hil_test.py`: add `import hil_flash` under the existing imports; delete the moved definitions and the `build_dir = 'cmake-build'` global (line ~165) plus `global build_dir` in `main`. Repoint every use, all module-qualified: +- `globals()[f'flash_{...}']` → `getattr(hil_flash, f'flash_{...}')` (1 site, in `test_example`) +- `globals()[f'reset_{...}']` → `getattr(hil_flash, f'reset_{...}')` (3 sites: `test_host_device_info`, `test_host_cdc_msc_hid`, `test_host_msc_file_explorer`) +- bare `run_cmd(` → `hil_flash.run_cmd(` ; `cmd_stdout_text(` → `hil_flash.cmd_stdout_text(` ; `find_firmware(` → `hil_flash.find_firmware(` ; `get_serial_dev(` → `hil_flash.get_serial_dev(` ; `TINYUSB_ROOT` → `hil_flash.TINYUSB_ROOT` (in `build_board`, `CONTROLLER_CACHE` stays hil_test-local) +- In `main()`: `build_dir = args.build_dir` → `hil_flash.build_dir = args.build_dir`; where `verbose` is set, add `hil_flash.verbose = args.verbose` (hil_test keeps its own `verbose` for test-side prints) +- `run_cmd`'s `elif verbose:` branch now reads `hil_flash.verbose` (it moved with the function — verify it references the module-local name, not hil_test's) + +Find every remaining call site mechanically: + +Run: `grep -nE 'run_cmd|cmd_stdout_text|find_firmware|get_serial_dev|flash_[a-z]|reset_[a-z]|TINYUSB_ROOT|OPENCOD' test/hil/hil_test.py | grep -v hil_flash` +Expected: only hits inside comments/strings and the `reset_{flasher}` dispatch f-strings already qualified. + +- [ ] **Step 3: Repoint pool_check's flash imports** + +In `.claude/skills/hil/pool_check.py`: add `import hil_flash` next to `import hil_test`; replace `hil_test.find_firmware` → `hil_flash.find_firmware` (3 sites), `hil_test.cmd_stdout_text` → `hil_flash.cmd_stdout_text`, `hil_test.get_serial_dev` → `hil_flash.get_serial_dev`, `hil_test.TINYUSB_ROOT` → `hil_flash.TINYUSB_ROOT`, `hil_test.build_dir` → `hil_flash.build_dir` (2 sites incl. `main`'s assignment), `hil_test.verbose = args.verbose` → `hil_flash.verbose = args.verbose`, `getattr(hil_test, f'flash_...')`/`getattr(hil_test, f'reset_...')` → `getattr(hil_flash, ...)` (4 sites). Keep `import hil_test` and the pymtp shim for now (locks still live there; removed in Task 2). + +- [ ] **Step 4: Add hil_flash.py to the hil_ci.sh scp list** + +```bash +scp -q "$ROOT_DIR/test/hil/hil_test.py" \ + "$ROOT_DIR/test/hil/hil_flash.py" \ + "$ROOT_DIR/test/hil/pymtp.py" \ + "$CONFIG" \ + "$REMOTE:$REMOTE_DIR/test/hil/" +``` + +- [ ] **Step 5: Verify** + +Run: `python3 -m py_compile test/hil/hil_flash.py test/hil/hil_test.py .claude/skills/hil/pool_check.py && python3 test/hil/hil_test.py --help >/dev/null && python3 .claude/skills/hil/pool_check.py --scan-only` +Expected: compiles; help prints nothing to stderr; scan-only prints the table and exits 0. + +- [ ] **Step 6: Commit** + +```bash +git add test/hil/hil_flash.py test/hil/hil_test.py test/hil/hil_ci.sh .claude/skills/hil/pool_check.py +git commit -m "hil: extract flashing into hil_flash.py" +``` + +--- + +### Task 2: Create hil_lock.py core (flock protocol + controller permits); repoint hil_test + pool_check + +**Files:** +- Create: `test/hil/hil_lock.py` +- Modify: `test/hil/hil_test.py` +- Modify: `.claude/skills/hil/pool_check.py` +- Modify: `test/hil/hil_ci.sh` + +**Interfaces:** +- Produces: module `hil_lock` with `BOARD_LOCK_DIR`, `CI_REASON = 'hil_test.py'`, `lock_path(board)`, `flock_nb(board)`, `write_record(fh, reason)`, `clear_record(fh)`, `read_record(board)`, `acquire_board_lock(board, reason=CI_REASON)`, `FLASH_PARALLEL`, `USBTEST_PARALLEL`, `CONTROLLER_SLOTS`, `controller_of(uid)`, `controller_slot(pci)`, `controller_permit`, `flash_permit(uid)`, `usbtest_permit(uid)`, `init_scheduling(b_sems, f_sems, cmap, cmeta, hints, log_fn=None)`. + +- [ ] **Step 1: Create `test/hil/hil_lock.py` with the flock core** + +New code (the protocol, factored from today's three copies — `board_lock.py` `cmd_hold`/`read_info`, `hil_test.acquire_board_lock`, pool_check `lock_board`; behavior identical to `hil_test.acquire_board_lock` for the acquire path): + +```python +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +"""Board locks + controller permits for the TinyUSB HIL rig. + +Board locks are kernel flocks in BOARD_LOCK_DIR arbitrating hardware access +between dev sessions and CI's hil_test.py (never stop the actions-runner). +Controller permits are in-process semaphores budgeting flashes and usbtest +batteries per host controller; they have no CLI meaning. The CLI below +(hold/release/status) manages board locks only; it supersedes board_lock.py. +""" +import argparse +import fcntl +import glob +import json +import os +import re +import select +import signal +import sys +import time + +BOARD_LOCK_DIR = '/tmp/tinyusb-hil-locks' +CI_REASON = 'hil_test.py' # release-protected holder tag (release refuses to kill it) +PROFILE = os.environ.get('HIL_PROFILE') == '1' + + +def lock_path(board: str) -> str: + return os.path.join(BOARD_LOCK_DIR, f'{board}.lock') + + +def flock_nb(board: str): + """Open-or-create the lock file WITHOUT truncating (a losing racer must not + wipe the winner's record) and take LOCK_EX|LOCK_NB. Returns the open handle; + raises OSError when the flock is held elsewhere (handle already closed).""" + fd = os.open(lock_path(board), os.O_RDWR | os.O_CREAT, 0o666) + fh = os.fdopen(fd, 'r+') + try: + fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB) + except OSError: + fh.close() + raise + return fh + + +def write_record(fh, reason: str) -> None: + """Best-effort holder record; the flock itself is already held.""" + try: + fh.truncate(0) + fh.seek(0) + json.dump({'pid': os.getpid(), 'reason': reason, + 'since': time.strftime('%Y-%m-%dT%H:%M:%S%z')}, fh) + fh.flush() + except OSError: + pass + + +def clear_record(fh) -> None: + """Clear our record before dropping the flock so records stay truthful.""" + try: + fh.truncate(0) + except OSError: + pass + + +def read_record(board: str): + try: + with open(lock_path(board)) as f: + return json.load(f) + except (OSError, ValueError): + return None +``` + +Then MOVE `acquire_board_lock` from `hil_test.py` verbatim, with exactly two mechanical edits: signature becomes `def acquire_board_lock(board_name, reason=CI_REASON):` and the record-write dict's `'reason': 'hil_test.py'` becomes `'reason': reason`. Do NOT rewrite its body in terms of `flock_nb` — on conflict it reads holder info from the still-open handle before closing, which `flock_nb` (closes on conflict) cannot provide; the fail-open warning text and RuntimeError message must survive character-for-character. + +- [ ] **Step 2: Move the controller-permit block into `hil_lock.py`** + +MOVE verbatim from `hil_test.py`: the scheduling comment block + `FLASH_PARALLEL`, `USBTEST_PARALLEL`, `CONTROLLER_SLOTS`, the five module globals (`usbtest_sems`, `flash_sems`, `controller_map`, `controller_meta`, `controller_hints`), `controller_of`, `controller_slot`, `controller_permit`, `flash_permit`, `usbtest_permit`. Two mechanical adaptations: +- add at module scope `log = print` and a setter, replacing the two `log_line(...)` calls inside `controller_of`/`controller_permit` with `log(...)`: + +```python +log = print # hil_test.init_worker points this at log_line via init_scheduling + + +def init_scheduling(b_sems, f_sems, cmap, cmeta, hints, log_fn=None): + """Install per-worker scheduling state (called from hil_test.init_worker).""" + global usbtest_sems, flash_sems, controller_map, controller_meta, controller_hints, log + usbtest_sems, flash_sems = b_sems, f_sems + controller_map, controller_meta, controller_hints = cmap, cmeta, hints + if log_fn is not None: + log = log_fn +``` + +- `PROFILE` inside `controller_permit` now resolves to hil_lock's own module constant (defined in Step 1). + +- [ ] **Step 3: Repoint `hil_test.py`** + +Add `import hil_lock`. Delete the moved lock + permit code and the five globals. `init_worker` keeps its exact signature and initargs; its body sets the hil_test globals it still owns (`print_lock`, `shuffle_seed`) and forwards the rest: + +```python +def init_worker(lock, seed, b_mutexes, f_sems, cmap, cmeta, hints_by_uid): + global print_lock, shuffle_seed + print_lock = lock + shuffle_seed = seed + hil_lock.init_scheduling(b_mutexes, f_sems, cmap, cmeta, hints_by_uid, log_fn=log_line) +``` + +Qualify remaining uses: `acquire_board_lock(name)` → `hil_lock.acquire_board_lock(name)` (in `test_board`), `flash_permit(` → `hil_lock.flash_permit(`, `usbtest_permit(` → `hil_lock.usbtest_permit(`, and `main()`'s startup log line + Semaphore construction read `hil_lock.FLASH_PARALLEL`/`hil_lock.USBTEST_PARALLEL`/`hil_lock.CONTROLLER_SLOTS`. `controller_map` reads in the hint-persistence block of `main` use the Manager dict it already holds locally (`cmap`) — no hil_lock global access there; verify. + +- [ ] **Step 4: Repoint pool_check to hil_lock and drop its private copies + hil_test import** + +In `pool_check.py`: replace `lock_board`/`unlock_board` bodies with the shared core — + +```python +import hil_lock + +def lock_board(name: str): + try: + fh = hil_lock.flock_nb(name) + except OSError: + info = hil_lock.read_record(name) + return json.dumps(info) if info else 'unknown holder' + hil_lock.write_record(fh, 'pool_check') + return fh + + +def unlock_board(fh) -> None: + hil_lock.clear_record(fh) + fh.close() +``` + +(Behavior note: `lock_board` currently returns the raw record text; JSON-dumping the parsed record is equivalent for display. `hil_lock.BOARD_LOCK_DIR` replaces `hil_test.BOARD_LOCK_DIR`; `os.makedirs(...)` call stays, now on `hil_lock.BOARD_LOCK_DIR`.) Then delete `import hil_test` and the pymtp stub block (`try: import pymtp ... sys.modules['pymtp'] = ...`) — pool_check now imports only `hil_lock` + `hil_flash`. + +Run: `grep -n 'hil_test' .claude/skills/hil/pool_check.py` +Expected: only the docstring mention of the protocol/history, no code references (update the docstring's "imports test/hil/hil_test.py" line to name hil_lock/hil_flash). + +- [ ] **Step 5: Add hil_lock.py to the hil_ci.sh scp list** (same block as Task 1 Step 4, one more line: `"$ROOT_DIR/test/hil/hil_lock.py" \`) + +- [ ] **Step 6: Verify** + +Run: `python3 -m py_compile test/hil/hil_lock.py test/hil/hil_test.py .claude/skills/hil/pool_check.py && python3 test/hil/hil_test.py --help >/dev/null && python3 .claude/skills/hil/pool_check.py --scan-only` +Expected: clean compile, working scan table, exit 0. + +- [ ] **Step 7: Commit** + +```bash +git add test/hil/hil_lock.py test/hil/hil_test.py test/hil/hil_ci.sh .claude/skills/hil/pool_check.py +git commit -m "hil: extract board locks and controller permits into hil_lock.py" +``` + +--- + +### Task 3: Absorb board_lock.py CLI into hil_lock.py; delete board_lock.py; rename in docs + +**Files:** +- Modify: `test/hil/hil_lock.py` (append CLI) +- Delete: `test/hil/board_lock.py` +- Modify: `.claude/skills/hil/SKILL.md`, `.claude/agents/hil-operator.md`, `.claude/agents/target-debugger.md`, `.claude/skills/etm-trace/SKILL.md`, `.claude/skills/usb-kernel-recover/SKILL.md`, `.claude/skills/target-debug/SKILL.md` + +**Interfaces:** +- Produces: `python3 test/hil/hil_lock.py hold|release|status` — identical subcommands, flags, output, and exit codes to today's `board_lock.py`. + +- [ ] **Step 1: Move the CLI from `board_lock.py` into `hil_lock.py`** + +MOVE verbatim to the end of `hil_lock.py`: `boards_from_config`, `is_locked`, `cmd_hold`, `cmd_release`, `cmd_status`, `main()`, and the `if __name__ == '__main__':` guard. Mechanical adaptations only: +- `LOCK_DIR` → `BOARD_LOCK_DIR` (all sites), `lock_path` already exists (delete the duplicate), `read_info` → `read_record` (all sites; delete the duplicate definition) +- `cmd_hold`'s holder loop body (the open/flock/json.dump block) becomes `fh = flock_nb(b)` + `write_record(fh, reason)` inside the existing try/except OSError +- `_bow_out`'s per-handle truncate loop becomes `clear_record(h)` per handle +- `cmd_release`'s probe uses `flock_nb(b)` in a try/except OSError (held → existing record/victim logic, with the literal `'hil_test.py'` comparison becoming `CI_REASON`); the free-path truncate becomes `clear_record(fh)` +- `main()`'s module docstring reference for `--help` text: keep the usage lines, updating the tool name to `hil_lock.py` + +Then delete `test/hil/board_lock.py` (`git rm test/hil/board_lock.py`). + +- [ ] **Step 2: Rename `board_lock.py` → `hil_lock.py` in the six live docs** + +Run: `cd <worktree> && sed -i 's/board_lock\.py/hil_lock.py/g' .claude/skills/hil/SKILL.md .claude/agents/hil-operator.md .claude/agents/target-debugger.md .claude/skills/etm-trace/SKILL.md .claude/skills/usb-kernel-recover/SKILL.md .claude/skills/target-debug/SKILL.md` +Then: `grep -rn 'board_lock' .claude/ test/ --include='*.md' --include='*.py' --include='*.sh'` +Expected: zero hits outside `docs/superpowers/` history (which stays untouched). + +- [ ] **Step 3: Verify CLI behavior end-to-end** + +```bash +python3 test/hil/hil_lock.py status # expect: no locks (or current holders) +python3 test/hil/hil_lock.py hold stm32f072disco --reason "split test" & +sleep 1 +python3 test/hil/hil_lock.py status # expect: stm32f072disco: {... 'reason': 'split test' ...} +python3 test/hil/hil_lock.py hold stm32f072disco --reason "rival" || echo "conflict OK" # expect: ERROR ... locked + conflict OK +python3 test/hil/hil_lock.py release stm32f072disco # expect: released holder pid NNN +python3 test/hil/hil_lock.py status # expect: no locks +``` + +Also verify CI-holder protection: create a fake record `echo '{"pid": 1, "reason": "hil_test.py"}' > /tmp/tinyusb-hil-locks/faketest.lock` — since pid 1 holds no flock, `release faketest` must clear the stale record without printing the mid-test error; then `rm -f /tmp/tinyusb-hil-locks/faketest.lock`. + +- [ ] **Step 4: Commit** + +```bash +git add -A test/hil .claude +git commit -m "hil: fold board_lock CLI into hil_lock.py, retire board_lock.py" +``` + +--- + +### Task 4: Rig verification + pre-commit + +**Files:** none new (fixes only if verification fails) + +- [ ] **Step 1: pool_check flash path on one board** + +Run: `python3 .claude/skills/hil/pool_check.py -b stm32f407disco` +Expected: `✅ dfu_runtime ✅ cafe:...`, exit 0. + +- [ ] **Step 2: Capture a pre-refactor baseline report** + +Run: `cd /home/hathach/code/tinyusb && python3 test/hil/hil_test.py -b stm32f407disco -B examples test/hil/tinyusb.json && cp hil_report.md /tmp/claude-1000/-home-hathach-code-tinyusb/*/scratchpad/hil_report_master.md` +(Primary checkout = pre-refactor code but same rig/config; its working tree already carries the new probe uids.) + +- [ ] **Step 3: Run the same board from the worktree and diff the report shape** + +Run: `cd .claude/worktrees/hil-test-split && python3 test/hil/hil_test.py -b stm32f407disco -B /home/hathach/code/tinyusb/examples test/hil/tinyusb.json && diff <(sed 's/[0-9.]*s//g;s/[0-9.]* [kMG]B\/s//g' hil_report.md) <(sed 's/[0-9.]*s//g;s/[0-9.]* [kMG]B\/s//g' /tmp/claude-1000/-home-hathach-code-tinyusb/*/scratchpad/hil_report_master.md)` +Expected: empty diff after stripping timings/speeds. Note: `-B` accepts the absolute path so the worktree run reuses the primary checkout's built firmware; `find_firmware` resolves `TINYUSB_ROOT/<build_dir>` and an absolute `-B` overrides relative rooting — if it does not (Path join semantics), instead symlink `ln -s /home/hathach/code/tinyusb/examples/cmake-build-stm32f407disco examples/cmake-build-stm32f407disco` in the worktree and use `-B examples`. + +- [ ] **Step 4: pre-commit + final grep hygiene** + +Run: `pre-commit run --files test/hil/hil_test.py test/hil/hil_lock.py test/hil/hil_flash.py test/hil/hil_ci.sh .claude/skills/hil/pool_check.py $(git diff --name-only HEAD~3 -- '*.md')` +Expected: all hooks pass. + +- [ ] **Step 5: Commit any verification fixes** + +```bash +git add -A && git commit -m "hil: post-split verification fixes" # only if Steps 1-4 required changes +``` diff --git a/docs/superpowers/plans/2026-07-29-hil-select.md b/docs/superpowers/plans/2026-07-29-hil-select.md new file mode 100644 index 000000000..a8abf9887 --- /dev/null +++ b/docs/superpowers/plans/2026-07-29-hil-select.md @@ -0,0 +1,856 @@ +# PR-Scoped HIL Selection Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** A diff→(boards, tests) selector (`test/hil/hil_select.py`) that scopes CI's HIL build+test jobs on pull requests and is reusable locally, per `docs/superpowers/specs/2026-07-29-hil-pr-scoped-selection-design.md`. + +**Architecture:** Pure-stdlib classification engine (changed files → per-board test selection, fail-open to full) + thin CLI emitting JSON with per-rig `hil_test.py` arg strings; consumed by `hil_ci_set_matrix.py --select` (prunes hil-build) and shell steps in the three HIL jobs (prunes rig runs). Test lists shared via new `hil_examples.py`. + +**Tech Stack:** Python 3.11 stdlib only (`re`, `json`, `glob`, `subprocess` for git), `unittest` for tests, GitHub Actions YAML. + +## Global Constraints + +- Work in worktree `.claude/worktrees/hil-select` (branch `claude/hil-select`); never touch the primary checkout. +- `hil_select.py`, `hil_examples.py`, `test_hil_select.py` import NOTHING outside the stdlib and each other — in particular never `hil_test`/`hil_flash`/`hil_lock` (GitHub's bare runner has no pyserial/pymtp). +- Fail-open: any changed file matching no classification rule ⇒ `full: true`. Scoping applies to `pull_request` events only; push/scheduled runs stay full. +- Behavior-preserving for existing tools: `hil_test.py` runtime behavior unchanged (only its test-list constants move to `hil_examples.py`); `hil_ci_set_matrix.py` without `--select` emits byte-identical output to today. +- The selector only ever emits board names present in the given roster (`config['boards']`); `boards-skip` is invisible to it. +- Commit messages: imperative, scoped, NO Co-Authored-By/Claude-Session trailers. +- Every commit: `python3 -m py_compile` clean on touched python files, `python3 test/hil/test_hil_select.py` green (once it exists), `pre-commit run --files <touched>` clean. + +--- + +### Task 1: hil_examples.py + selection engine with unit tests + +**Files:** +- Create: `test/hil/hil_examples.py` +- Create: `test/hil/hil_select.py` (engine only; CLI comes in Task 2) +- Create: `test/hil/test_hil_select.py` +- Modify: `test/hil/hil_test.py` (import test lists from hil_examples) +- Modify: `test/hil/hil_ci.sh` (scp list gains `hil_examples.py`) + +**Interfaces:** +- Produces `hil_examples.py`: `device_tests: list[str]`, `dual_tests: list[str]`, `host_test: list[str]` — the three lists moved VERBATIM (incl. comments) from `hil_test.py`. +- Produces `hil_select.py` engine API used by Task 2: + - `classify(changed_files: list[str], repo_root: str, rosters: list[tuple[str, list[dict]]]) -> dict` + returning `{'full': bool, 'boards': {board_name: 'all' | sorted list[str]}, 'reasons': list[str]}` + where `rosters` = `[(config_path, config['boards']), ...]`. + - `board_roles(board: dict) -> set[str]` — subset of `{'device', 'host'}` from the roster + entry's `tests` flags (`device`/`host`/`dual` booleans; an `only` list contributes the + roles of its entries' path prefixes; `dual` implies both roles). + - `board_family(board_name: str, repo_root: str) -> str | None` — the `<family>` for which + `hw/bsp/<family>/boards/<board_name>` exists. + - `port_families(port_dir: str, repo_root: str) -> set[str]` — directories of + `hw/bsp/*/family.cmake` and `hw/bsp/*/family.mk` whose text contains `port_dir` + (e.g. `raspberrypi/rp2040`). + - `class_examples(class_dir: str, role: str, repo_root: str) -> set[str]` — tests from + `hil_examples` lists whose example `tusb_config.h` enables the class for that role (regex + `#define\s+CFG_TUD_<C>\s+\(?\s*0*[1-9]` / `CFG_TUH_<C>`; exceptions per spec: + `dfu_rt_device.*`→`CFG_TUD_DFU_RUNTIME`, `dfu_device.*`→`CFG_TUD_DFU`, class dir `net` + → `CFG_TUD_ECM_RNDIS|CFG_TUD_NCM`). Test path `device/x` ⇒ config at + `examples/device/x/src/tusb_config.h`; same pattern for `host/` and `dual/`. + +- [ ] **Step 1: Move the test lists into `hil_examples.py`** + +Create `test/hil/hil_examples.py`: + +```python +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +# HIL example test lists, shared by hil_test.py (runner) and hil_select.py +# (PR-diff selector). Stdlib-only: hil_select runs on bare CI runners. +``` + +then MOVE the `device_tests`, `dual_tests`, `host_test` list definitions (and their preceding +comment block "The per-board run order is shuffled...") VERBATIM from `hil_test.py` into it. +In `hil_test.py`, add `from hil_examples import device_tests, dual_tests, host_test` where the +lists were (a `from`-import of data constants is fine here — they are read-only lists used by +name throughout `test_board`). Add `"$ROOT_DIR/test/hil/hil_examples.py" \` to the +`hil_ci.sh` scp list after the `hil_lock.py` line. + +- [ ] **Step 2: Verify the move broke nothing** + +Run: `cd /home/hathach/code/tinyusb/.claude/worktrees/hil-select && python3 -m py_compile test/hil/hil_examples.py test/hil/hil_test.py && python3 test/hil/hil_test.py --help >/dev/null && echo ok` +Expected: `ok` + +- [ ] **Step 3: Write the failing unit tests (spec acceptance cases)** + +Create `test/hil/test_hil_select.py`. ROSTER is a trimmed but real-shaped fixture; tests call +the engine API directly (no git, no CLI): + +```python +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +# Unit tests for hil_select.py — pure logic, no hardware, no git. Run directly: +# python3 test/hil/test_hil_select.py +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +import hil_select +from hil_examples import device_tests, dual_tests, host_test + +REPO = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + +ROSTER = [ + # device-only, rp2040 family + {'name': 'raspberry_pi_pico', 'uid': 'u1', + 'tests': {'device': True, 'host': True, 'dual': True}}, + # device-only, stm32f4 family + {'name': 'stm32f407disco', 'uid': 'u2', + 'tests': {'device': True, 'host': False, 'dual': False}}, + # host-only board + {'name': 'raspberry_pi_pico2', 'uid': 'u3', + 'tests': {'device': False, 'host': True, 'dual': False}}, + # only-list board (espressif-style) + {'name': 'espressif_s3_devkitm', 'uid': 'u4', + 'tests': {'only': ['device/cdc_msc_freertos', 'host/device_info']}}, +] +ROSTERS = [('test/hil/tinyusb.json', ROSTER)] + + +def sel(files): + return hil_select.classify(files, REPO, ROSTERS) + + +class TestPortRule(unittest.TestCase): + def test_dcd_rp2040_selects_pico_family_only(self): + s = sel(['src/portable/raspberrypi/rp2040/dcd_rp2040.c']) + self.assertFalse(s['full']) + self.assertIn('raspberry_pi_pico', s['boards']) + self.assertNotIn('stm32f407disco', s['boards']) + self.assertNotIn('espressif_s3_devkitm', s['boards']) + # device role: no host tests in pico's list + self.assertTrue(all(not t.startswith('host/') for t in s['boards']['raspberry_pi_pico'])) + # host-only boards drop out entirely on a device-role change + self.assertNotIn('raspberry_pi_pico2', s['boards']) + + def test_shared_port_file_is_both_roles(self): + s = sel(['src/portable/synopsys/dwc2/dwc2_common.c']) + self.assertFalse(s['full']) + self.assertNotIn('raspberry_pi_pico', s['boards']) # rp2040 is not a dwc2 family + self.assertIn('stm32f407disco', s['boards']) # stm32f4 is + + +class TestCoreRoleRule(unittest.TestCase): + def test_usbd_selects_all_device_tests_everywhere(self): + s = sel(['src/device/usbd.c']) + self.assertFalse(s['full']) + self.assertNotIn('raspberry_pi_pico2', s['boards']) # host-only board dropped + pico = s['boards']['raspberry_pi_pico'] + self.assertTrue(set(device_tests).issubset(set(pico))) + self.assertTrue(set(dual_tests).issubset(set(pico))) # dual survives device role + self.assertTrue(all(not t.startswith('host/') for t in pico)) + # only-list board: selection intersects its only-list + esp = s['boards']['espressif_s3_devkitm'] + self.assertEqual(esp, ['device/cdc_msc_freertos']) + + def test_host_change_drops_device(self): + s = sel(['src/host/usbh.c']) + self.assertFalse(s['full']) + self.assertIn('raspberry_pi_pico2', s['boards']) + self.assertNotIn('stm32f407disco', s['boards']) # device-only board dropped + + +class TestClassRule(unittest.TestCase): + def test_cdc_device_selects_cdc_examples_only(self): + s = sel(['src/class/cdc/cdc_device.c']) + self.assertFalse(s['full']) + pico = s['boards']['raspberry_pi_pico'] + self.assertIn('device/cdc_msc', pico) + self.assertIn('device/cdc_dual_ports', pico) + self.assertNotIn('device/msc_dual_lun', pico) # CFG_TUD_CDC 0 there + self.assertNotIn('device/usbtest', pico) # CFG_TUD_CDC 0 there + self.assertTrue(all(not t.startswith('host/') for t in pico)) + + def test_msc_host_selects_host_side(self): + s = sel(['src/class/msc/msc_host.c']) + self.assertFalse(s['full']) + self.assertNotIn('stm32f407disco', s['boards']) # device-only board + pico2 = s['boards']['raspberry_pi_pico2'] + self.assertIn('host/msc_file_explorer', pico2) + self.assertTrue(all(not t.startswith('device/') for t in pico2)) + + +class TestFallbackRules(unittest.TestCase): + def test_unknown_tool_is_full(self): + s = sel(['tools/random_new_script.py']) + self.assertTrue(s['full']) + + def test_docs_only_is_empty_not_full(self): + s = sel(['docs/info/contributing.rst', 'README.rst']) + self.assertFalse(s['full']) + self.assertEqual(s['boards'], {}) + + def test_bsp_family_selects_family_boards(self): + s = sel(['hw/bsp/rp2040/family.cmake']) + self.assertFalse(s['full']) + self.assertIn('raspberry_pi_pico', s['boards']) + self.assertEqual(s['boards']['raspberry_pi_pico'], 'all') + self.assertNotIn('stm32f407disco', s['boards']) + + def test_bsp_board_narrows_to_board(self): + s = sel(['hw/bsp/rp2040/boards/raspberry_pi_pico/board.h']) + self.assertFalse(s['full']) + self.assertEqual(list(s['boards'].keys()), ['raspberry_pi_pico']) + + def test_example_change_selects_that_example(self): + s = sel(['examples/device/cdc_msc/src/main.c']) + self.assertFalse(s['full']) + self.assertEqual(s['boards']['raspberry_pi_pico'], ['device/cdc_msc']) + + def test_core_common_is_full(self): + for f in ['src/tusb.c', 'src/common/tusb_fifo.c', 'src/osal/osal_freertos.h']: + self.assertTrue(sel([f])['full'], f) + + def test_harness_is_full(self): + for f in ['test/hil/hil_test.py', '.github/workflows/build.yml', 'hw/mcu/nxp/x.c', 'lib/foo/x.c']: + self.assertTrue(sel([f])['full'], f) + + def test_mixed_roles_no_pruning(self): + s = sel(['src/device/usbd.c', 'src/host/usbh.c']) + self.assertFalse(s['full']) + self.assertIn('raspberry_pi_pico2', s['boards']) + self.assertIn('stm32f407disco', s['boards']) + + +if __name__ == '__main__': + unittest.main(verbosity=1) +``` + +- [ ] **Step 4: Run tests to verify they fail** + +Run: `python3 test/hil/test_hil_select.py 2>&1 | tail -2` +Expected: `ModuleNotFoundError: No module named 'hil_select'` (or import error). + +- [ ] **Step 5: Implement the engine** + +Create `test/hil/hil_select.py`: + +```python +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +"""PR-diff -> HIL selection: which rig boards and which tests a change can affect. + +Stdlib-only (runs on bare CI runners; never imports hil_test/hil_flash/hil_lock). +Fail-open: any file no rule classifies forces the full matrix. See +docs/superpowers/specs/2026-07-29-hil-pr-scoped-selection-design.md. +""" +import argparse +import glob +import json +import os +import re +import subprocess +import sys + +from hil_examples import device_tests, dual_tests, host_test + +ALL_TESTS = {'device': device_tests, 'dual': dual_tests, 'host': host_test} + +# class dir -> config macro suffix exceptions (rule 3); dfu is per-file, handled inline +NET_MACROS = ('ECM_RNDIS', 'NCM') + +_NONCODE_RE = re.compile( + r'^(docs/|\.claude/|.*\.(md|rst|txt)$|LICENSE)') +_FULL_RE = re.compile( + r'^(src/common/|src/osal/|src/tusb\.c$|src/tusb\.h$|src/tusb_option\.h$|' + r'test/hil/|\.github/workflows/build.*\.yml$|\.github/actions/|' + r'tools/build\.py$|tools/get_deps\.py$|tools/cmake/|hw/mcu/|lib/|' + r'hw/bsp/(family_support\.cmake|board_api\.h|board\.c|ansi_escape\.h)$|' + r'examples/build_system/|examples/CMakeLists\.txt$)') + + +def test_role(test: str) -> str: + return test.split('/', 1)[0] # 'device' | 'dual' | 'host' + + +def board_roles(board: dict) -> set: + t = board.get('tests', {}) + roles = set() + if t.get('device'): + roles.add('device') + if t.get('host'): + roles.add('host') + if t.get('dual'): + roles.update(('device', 'host')) + for only in t.get('only', []): + r = test_role(only) + roles.update(('device', 'host') if r == 'dual' else (r,)) + return roles + + +def board_tests(board: dict) -> list: + """Every test this board would run today (mirrors hil_test.test_board's default).""" + t = board.get('tests', {}) + if 'only' in t: + run = list(t['only']) + else: + run = [] + if t.get('device'): + run += device_tests + if t.get('dual'): + run += dual_tests + if t.get('host'): + run += host_test + return [x for x in run if x not in t.get('skip', [])] + + +def board_family(board_name: str, repo_root: str): + hits = glob.glob(os.path.join(repo_root, 'hw/bsp/*/boards', board_name)) + return os.path.basename(os.path.dirname(os.path.dirname(hits[0]))) if hits else None + + +def port_families(port_dir: str, repo_root: str) -> set: + fams = set() + for f in glob.glob(os.path.join(repo_root, 'hw/bsp/*/family.cmake')) + \ + glob.glob(os.path.join(repo_root, 'hw/bsp/*/family.mk')): + try: + if port_dir in open(f).read(): + fams.add(os.path.basename(os.path.dirname(f))) + except OSError: + pass + return fams + + +def _config_enables(cfg_path: str, macros) -> bool: + try: + text = open(cfg_path).read() + except OSError: + return False + return any(re.search(rf'#define\s+{m}\s+\(?\s*0*[1-9]', text) for m in macros) + + +def class_examples(macros, role: str, repo_root: str) -> set: + """Tests (from role's + dual lists) whose example config enables any macro.""" + pools = {'device': device_tests + dual_tests, 'host': host_test + dual_tests} + out = set() + for test in pools[role]: + cfg = os.path.join(repo_root, 'examples', test, 'src', 'tusb_config.h') + if _config_enables(cfg, macros): + out.add(test) + return out + + +class _Sel: + """Accumulates contributions. board->set(tests) plus 'all-board' markers.""" + def __init__(self): + self.full = False + self.by_board = {} # name -> set of tests, or 'all' + self.roles = set() # roles touched by any contribution + self.reasons = [] + + def add(self, boards, tests, reason): + """tests: 'all' or iterable of test paths.""" + self.reasons.append(reason) + for b in boards: + cur = self.by_board.get(b) + if tests == 'all' or cur == 'all': + self.by_board[b] = 'all' + else: + self.by_board[b] = (cur or set()) | set(tests) + + def force_full(self, reason): + self.full = True + self.reasons.append(reason) + + +def _classify_one(path, repo_root, roster_boards, s: _Sel): + base = os.path.basename(path) + if _NONCODE_RE.match(path): + s.reasons.append(f'{path}: non-code, no contribution') + return + if _FULL_RE.match(path): + s.force_full(f'{path}: core/infra -> full matrix') + return + + m = re.match(r'src/portable/((?:[^/]+/)?[^/]+)/', path) + if m: + port = m.group(1) + if re.match(r'(dcd_|.*_device)', base): + roles = {'device'} + elif re.match(r'(hcd_|.*_host)', base): + roles = {'host'} + else: + roles = {'device', 'host'} + fams = port_families(port, repo_root) + boards = [b['name'] for b in roster_boards + if board_family(b['name'], repo_root) in fams and (board_roles(b) & roles)] + tests = [t for r in roles for t in ALL_TESTS[r]] + dual_tests + s.roles.update(roles) + s.add(boards, tests, f'{path}: port {port} -> families {sorted(fams)} -> boards {boards} ({"/".join(sorted(roles))})') + return + + m = re.match(r'src/class/([^/]+)/', path) + if m: + cls = m.group(1) + if re.search(r'_device\.[ch]$', base): + roles = {'device'} + elif re.search(r'_host\.[ch]$', base): + roles = {'host'} + else: + roles = {'device', 'host'} + # macro names per role + def macros(prefix): + if cls == 'net': + return [f'CFG_{prefix}_{m2}' for m2 in NET_MACROS] + if cls == 'dfu': + if base.startswith('dfu_rt'): + return [f'CFG_{prefix}_DFU_RUNTIME'] + if base.startswith('dfu_device') or base.startswith('dfu_host'): + return [f'CFG_{prefix}_DFU'] + return [f'CFG_{prefix}_DFU', f'CFG_{prefix}_DFU_RUNTIME'] + return [f'CFG_{prefix}_{cls.upper()}'] + tests = set() + if 'device' in roles: + tests |= class_examples(macros('TUD'), 'device', repo_root) + if 'host' in roles: + tests |= class_examples(macros('TUH'), 'host', repo_root) + boards = [b['name'] for b in roster_boards if board_roles(b) & roles] + s.roles.update(roles) + s.add(boards, tests, f'{path}: class {cls} -> {sorted(tests)} ({"/".join(sorted(roles))})') + return + + m = re.match(r'src/(device|host)/', path) + if m: + role = m.group(1) + boards = [b['name'] for b in roster_boards if role in board_roles(b)] + s.roles.add(role) + s.add(boards, ALL_TESTS[role] + dual_tests, f'{path}: core {role} stack -> all {role} tests') + return + + m = re.match(r'hw/bsp/([^/]+)/(?:boards/([^/]+)/)?', path) + if m: + fam, brd = m.group(1), m.group(2) + if brd: + boards = [b['name'] for b in roster_boards if b['name'] == brd] + why = f'{path}: bsp board {brd}' + else: + boards = [b['name'] for b in roster_boards + if board_family(b['name'], repo_root) == fam] + why = f'{path}: bsp family {fam}' + s.roles.update(('device', 'host')) + s.add(boards, 'all', f'{why} -> boards {boards}') + return + + m = re.match(r'examples/(device|host|dual)/([^/]+)/', path) + if m: + test = f'{m.group(1)}/{m.group(2)}' + known = any(test in pool for pool in ALL_TESTS.values()) + if known: + boards = [b['name'] for b in roster_boards] + role = test_role(test) + s.roles.update(('device', 'host') if role == 'dual' else (role,)) + s.add(boards, [test], f'{path}: example -> {test} on all boards') + else: + s.reasons.append(f'{path}: example not in HIL lists, no contribution') + return + + s.force_full(f'{path}: unclassified -> full matrix') + + +def classify(changed_files, repo_root, rosters): + all_boards = [] + seen = set() + for _, boards in rosters: + for b in boards: + if b['name'] not in seen: + seen.add(b['name']) + all_boards.append(b) + + s = _Sel() + for path in changed_files: + _classify_one(path, repo_root, all_boards, s) + if s.full: + break + + if s.full: + return {'full': True, 'boards': {b['name']: 'all' for b in all_boards}, + 'reasons': s.reasons} + + # role pruning: single-role selections drop the other role's tests and boards + by_name = {b['name']: b for b in all_boards} + out = {} + for name, tests in s.by_board.items(): + allowed = board_tests(by_name[name]) + if tests == 'all': + kept = list(allowed) + else: + kept = [t for t in allowed if t in tests] + if s.roles and s.roles != {'device', 'host'}: + role = next(iter(s.roles)) + kept = [t for t in kept if test_role(t) in (role, 'dual')] + if kept: + out[name] = 'all' if set(kept) == set(allowed) else sorted(kept) + return {'full': False, 'boards': out, 'reasons': s.reasons} +``` + +- [ ] **Step 6: Run tests to verify they pass** + +Run: `python3 test/hil/test_hil_select.py` +Expected: all tests PASS (OK line). Iterate on the engine (not the tests) until green; if a +test premise contradicts the repo (e.g. a family name), verify against the tree and fix the +test only with evidence noted in your report. + +- [ ] **Step 7: Commit** + +```bash +git add test/hil/hil_examples.py test/hil/hil_select.py test/hil/test_hil_select.py test/hil/hil_test.py test/hil/hil_ci.sh +git commit -m "hil: add PR-diff selection engine (hil_select) with shared example lists" +``` + +--- + +### Task 2: CLI + args emission + +**Files:** +- Modify: `test/hil/hil_select.py` (add `selection_args`, `main`) +- Modify: `test/hil/test_hil_select.py` (add CLI/args tests) + +**Interfaces:** +- Consumes: Task 1's `classify` and roster shapes. +- Produces: + - `selection_args(sel: dict, rosters) -> dict` mapping each config path's basename to the + `hil_test.py` argument string for that rig: for each selected board ON that roster, + `-b <name>`, plus `-bt <name>:<t1>,<t2>` when the board's entry is a list (not 'all'). + Empty string when no selected board is on that roster. When `sel['full']`, every roster + board gets bare `-b`? NO — full means "today's behavior": `selection_args` returns `''` + for every config (no filtering args at all). + - CLI: `python3 test/hil/hil_select.py [--base REF | --diff-file PATH] CONFIG...` printing + the JSON `{'full', 'boards', 'args', 'reasons'}` to stdout, reasons also to stderr + (one line each, prefixed `hil_select: `). Non-zero exit only on operational errors + (bad ref, unreadable config) — never on an empty selection. + +- [ ] **Step 1: Add failing CLI/args tests to `test_hil_select.py`** + +```python +class TestArgsEmission(unittest.TestCase): + def test_args_for_scoped_selection(self): + s = sel(['src/portable/raspberrypi/rp2040/dcd_rp2040.c']) + args = hil_select.selection_args(s, ROSTERS) + a = args['tinyusb.json'] + self.assertIn('-b raspberry_pi_pico', a) + self.assertNotIn('stm32f407disco', a) + self.assertIn('-bt raspberry_pi_pico:', a) # device-only subset of a device+host board + + def test_args_full_is_empty(self): + s = sel(['tools/random_new_script.py']) + self.assertEqual(hil_select.selection_args(s, ROSTERS), {'tinyusb.json': ''}) + + def test_args_all_board_gets_bare_b(self): + s = sel(['hw/bsp/rp2040/boards/raspberry_pi_pico/board.h']) + a = hil_select.selection_args(s, ROSTERS)['tinyusb.json'] + self.assertIn('-b raspberry_pi_pico', a) + self.assertNotIn('-bt', a) + + def test_cli_diff_file(self): + import subprocess, tempfile, json as j + with tempfile.NamedTemporaryFile('w', suffix='.txt', delete=False) as f: + f.write('src/class/cdc/cdc_device.c\n') + path = f.name + r = subprocess.run([sys.executable, os.path.join(REPO, 'test/hil/hil_select.py'), + '--diff-file', path, os.path.join(REPO, 'test/hil/tinyusb.json')], + capture_output=True, text=True) + self.assertEqual(r.returncode, 0, r.stderr) + out = j.loads(r.stdout) + self.assertFalse(out['full']) + self.assertIn('tinyusb.json', out['args']) + self.assertTrue(any('cdc_device' in line for line in out['reasons'])) + os.unlink(path) +``` + +- [ ] **Step 2: Run to verify the new tests fail** + +Run: `python3 test/hil/test_hil_select.py 2>&1 | tail -3` +Expected: failures/errors mentioning `selection_args`. + +- [ ] **Step 3: Implement `selection_args` and `main`** + +Append to `hil_select.py`: + +```python +def selection_args(sel, rosters): + args = {} + for cfg_path, boards in rosters: + key = os.path.basename(cfg_path) + if sel['full']: + args[key] = '' + continue + parts = [] + for b in boards: + chosen = sel['boards'].get(b['name']) + if chosen is None: + continue + parts.append(f'-b {b["name"]}') + if chosen != 'all': + parts.append(f'-bt {b["name"]}:{",".join(chosen)}') + args[key] = ' '.join(parts) + return args + + +def changed_files_from_git(base, repo_root): + mb = subprocess.run(['git', 'merge-base', 'HEAD', base], cwd=repo_root, + capture_output=True, text=True, check=True).stdout.strip() + diff = subprocess.run(['git', 'diff', '--name-only', f'{mb}..HEAD'], cwd=repo_root, + capture_output=True, text=True, check=True).stdout + return [l for l in diff.splitlines() if l.strip()] + + +def main(): + ap = argparse.ArgumentParser(description=__doc__) + g = ap.add_mutually_exclusive_group(required=True) + g.add_argument('--base', help='git ref to diff against (merge-base..HEAD)') + g.add_argument('--diff-file', help='newline-separated changed-file list') + ap.add_argument('configs', nargs='+', help='rig roster JSON file(s)') + a = ap.parse_args() + + repo_root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + rosters = [] + for c in a.configs: + with open(c) as f: + rosters.append((c, json.load(f)['boards'])) + + files = (open(a.diff_file).read().splitlines() if a.diff_file + else changed_files_from_git(a.base, repo_root)) + files = [f for f in files if f.strip()] + + s = classify(files, repo_root, rosters) + s['args'] = selection_args(s, rosters) + for r in s['reasons']: + print(f'hil_select: {r}', file=sys.stderr) + print(json.dumps(s)) + + +if __name__ == '__main__': + main() +``` + +(The `parts.append f'...'` line above is pseudo-highlighted; write valid Python: +`parts.append(f'-bt {b["name"]}:{",".join(chosen)}')`.) + +- [ ] **Step 4: Run the full suite** + +Run: `python3 test/hil/test_hil_select.py && chmod +x test/hil/hil_select.py` +Expected: OK. + +- [ ] **Step 5: Smoke against the real repo state** + +Run: `python3 test/hil/hil_select.py --base HEAD test/hil/tinyusb.json test/hil/hfp.json` +Expected: empty diff ⇒ `{"full": false, "boards": {}, "args": {"tinyusb.json": "", "hfp.json": ""}, ...}` exit 0. +Then: `printf 'src/portable/wch/dcd_ch32_usbfs.c\n' > /tmp/d.txt && python3 test/hil/hil_select.py --diff-file /tmp/d.txt test/hil/tinyusb.json | python3 -m json.tool | head -20` +Expected: only WCH-family boards (nanoch32v203, ch32v103r_r1_1v0, ch32v307v_r1_1v0 — whichever reference that port) with device tests. + +- [ ] **Step 6: Commit** + +```bash +git add test/hil/hil_select.py test/hil/test_hil_select.py +git commit -m "hil: hil_select CLI with per-rig hil_test argument emission" +``` + +--- + +### Task 3: hil_ci_set_matrix --select + build.yml wiring + +**Files:** +- Modify: `test/hil/hil_ci_set_matrix.py` +- Modify: `.github/workflows/build.yml` (set-matrix job; hil-build consumers unchanged; hil-tinyusb + hil-tinyusb-esp steps) + +**Interfaces:** +- Consumes: Task 2's CLI JSON (`full`, `boards`, `args`). +- Produces: + - `hil_ci_set_matrix.py [--select JSON_STRING] CONFIG...`: with `--select` and + `full == false`, boards not in `select['boards']` are skipped when building the toolchain + buckets; otherwise identical behavior. Buckets stay present (possibly `[]`) so + `fromJSON(...)[toolchain]` keeps resolving. + - set-matrix outputs: `hil_select_json` (compact selection), `hil_args_tinyusb`, + `hil_args_hfp`, `hil_run_tinyusb`, `hil_run_hfp` (string 'true'/'false'). + +- [ ] **Step 1: Add `--select` to `hil_ci_set_matrix.py`** + +In `main()` add: + +```python + parser.add_argument('--select', help='hil_select.py JSON; scopes boards when full=false') +``` + +and after parsing: + +```python + selected = None + sel = json.loads(args.select) if args.select else None + if sel and not sel.get('full'): + selected = set(sel.get('boards', {})) +``` + +then inside the per-board loop, first line: + +```python + if selected is not None and board['name'] not in selected: + continue +``` + +- [ ] **Step 2: Verify byte-identical without --select and scoped with it** + +Run: `python3 test/hil/hil_ci_set_matrix.py test/hil/tinyusb.json test/hil/hfp.json > /tmp/m1.json && git stash -q && python3 test/hil/hil_ci_set_matrix.py test/hil/tinyusb.json test/hil/hfp.json > /tmp/m0.json && git stash pop -q && diff /tmp/m0.json /tmp/m1.json && echo identical` +Expected: `identical`. +Then: `python3 test/hil/hil_ci_set_matrix.py --select '{"full": false, "boards": {"raspberry_pi_pico": "all"}}' test/hil/tinyusb.json test/hil/hfp.json` +Expected: JSON whose `arm-gcc` list contains only the raspberry_pi_pico entry, `riscv-gcc`/`esp-idf` = []. + +- [ ] **Step 3: Wire set-matrix in `.github/workflows/build.yml`** + +In the `set-matrix` job: give the checkout full history and add the selection step between +checkout and matrix generation; make the HIL matrix use it: + +```yaml + - name: Checkout TinyUSB + uses: actions/checkout@v6 + with: + fetch-depth: 0 + + - name: HIL selection (PR only) + id: hil-select + if: github.event_name == 'pull_request' + run: | + python3 test/hil/test_hil_select.py + SELECT_JSON=$(python3 test/hil/hil_select.py --base "origin/${{ github.base_ref }}" test/hil/tinyusb.json test/hil/hfp.json) + echo "select=$SELECT_JSON" >> $GITHUB_OUTPUT + python3 - "$SELECT_JSON" >> $GITHUB_OUTPUT <<'EOF' + import json, sys + s = json.loads(sys.argv[1]) + args = s.get('args', {}) + for cfg, key in (('tinyusb.json', 'tinyusb'), ('hfp.json', 'hfp')): + a = args.get(cfg, '') + run = 'true' if (s['full'] or a) else 'false' + print(f'args_{key}={a}') + print(f'run_{key}={run}') + EOF +``` + +and in the existing "Generate matrix json" step, change the HIL line to: + +```yaml + # HIL matrix (merged from tinyusb + hifiphile configs), scoped on PRs + SELECT='${{ steps.hil-select.outputs.select }}' + HIL_MATRIX_JSON=$(python test/hil/hil_ci_set_matrix.py ${SELECT:+--select "$SELECT"} test/hil/tinyusb.json test/hil/hfp.json) +``` + +Add to the job's `outputs:` block: + +```yaml + hil_args_tinyusb: ${{ steps.hil-select.outputs.args_tinyusb }} + hil_args_hfp: ${{ steps.hil-select.outputs.args_hfp }} + hil_run_tinyusb: ${{ steps.hil-select.outputs.run_tinyusb }} + hil_run_hfp: ${{ steps.hil-select.outputs.run_hfp }} +``` + +(On non-PR events the step is skipped: outputs are empty strings — the consumers below treat +empty `run_*` as 'true' and empty args as no filtering, i.e. today's behavior.) + +- [ ] **Step 4: Wire the rig jobs** + +In the `hil-tinyusb` job (the matrixed one covering both rigs), find the step that runs +`hil_test.py --retry 1 ${{ matrix.test_args }} ${{ env.HIL_JSON }} $RERUN_ARGS` (~line 360) +and change the step's `run:` to select per-rig args and honor the skip flag: + +```yaml + run: | + case "$HIL_JSON" in + *tinyusb.json) SEL_ARGS='${{ needs.set-matrix.outputs.hil_args_tinyusb }}'; SEL_RUN='${{ needs.set-matrix.outputs.hil_run_tinyusb }}' ;; + *hfp.json) SEL_ARGS='${{ needs.set-matrix.outputs.hil_args_hfp }}'; SEL_RUN='${{ needs.set-matrix.outputs.hil_run_hfp }}' ;; + esac + if [ "$SEL_RUN" = "false" ]; then echo "HIL skipped by PR selection (no affected boards on this rig)"; exit 0; fi + python3 test/hil/hil_test.py --retry 1 ${{ matrix.test_args }} $SEL_ARGS ${{ env.HIL_JSON }} $RERUN_ARGS +``` + +Apply the same pattern to the second `hil_test.py` invocation at ~line 423 (`hil-tinyusb-esp`, +which is tinyusb-rig only: use the `hil_args_tinyusb`/`hil_run_tinyusb` outputs directly, no +case needed) and to the hfp job's direct `python3 test/hil/hil_test.py hfp.json` call at +~line 487 (use `hil_args_hfp`/`hil_run_hfp`). Preserve each step's existing surrounding lines +(report-dir env, RERUN_ARGS logic) — only inject the SEL_ARGS/SEL_RUN mechanics. + +- [ ] **Step 5: Validate the YAML and the exact shell locally** + +Run: `pre-commit run check-yaml --files .github/workflows/build.yml && python3 -c "import yaml; yaml.safe_load(open('.github/workflows/build.yml')); print('yaml ok')"` +Expected: `yaml ok` (pyyaml is available; if not, `pip install --user pyyaml` first). +Also simulate the selection step's python inline script: +`SELECT_JSON=$(python3 test/hil/hil_select.py --diff-file /tmp/d.txt test/hil/tinyusb.json test/hil/hfp.json) && python3 -c "import json,sys; s=json.loads(sys.argv[1]); print(s['args'])" "$SELECT_JSON"` +Expected: the args dict prints. + +- [ ] **Step 6: Commit** + +```bash +git add test/hil/hil_ci_set_matrix.py .github/workflows/build.yml +git commit -m "ci: scope HIL build+test matrix by PR diff via hil_select" +``` + +--- + +### Task 4: pre-pr + hil skill docs, final validation + +**Files:** +- Modify: `.claude/skills/pre-pr/SKILL.md` (mapping section delegates to the selector) +- Modify: `.claude/skills/hil/SKILL.md` (document the selector for manual runs) + +**Interfaces:** +- Consumes: Task 2's CLI. + +- [ ] **Step 1: Rewrite pre-pr's "2. Map changes to boards" section** + +Replace the section's grep heuristics (keep its numbered-section structure and the roster/cap +policy) with: + +```markdown +## 2. Map changes to boards + +- `python3 test/hil/hil_select.py --base $BASE test/hil/tinyusb.json` → JSON with the affected + rig boards (`boards`) and per-file `reasons`. `full: true` means a broad/infra change. +- Build-board sampling: from the selection's boards (or, when `full`, the representative set + `stm32f407disco` + `raspberry_pi_pico`), pick ONE board per family, preferring rig-roster + boards; cap at 4 and tell the user which families the cap dropped. The boards list must + NEVER end up empty — final fallback is `[stm32f407disco]`. +- A `full: true` selection or an empty one (docs-only) keeps today's behavior: minimal + software-only gate for docs-only, representative set otherwise. +``` + +- [ ] **Step 2: Add a short "PR-scoped selection" note to the hil skill** + +Append to `.claude/skills/hil/SKILL.md` after the pool-check section: + +```markdown +## PR-scoped selection + +`test/hil/hil_select.py` maps a diff to affected boards/tests (used by CI on PRs; fail-open +to the full matrix). Manual use: + +```bash +ARGS=$(python3 test/hil/hil_select.py --base master test/hil/tinyusb.json | python3 -c "import json,sys; print(json.load(sys.stdin)['args']['tinyusb.json'])") +python3 test/hil/hil_test.py -B examples $ARGS test/hil/tinyusb.json +``` + +Unit suite: `python3 test/hil/test_hil_select.py` (no hardware). +``` + +- [ ] **Step 3: Full validation sweep** + +Run: `python3 test/hil/test_hil_select.py && python3 -m py_compile test/hil/hil_select.py test/hil/hil_examples.py test/hil/hil_ci_set_matrix.py test/hil/hil_test.py && python3 test/hil/hil_test.py --help >/dev/null && pre-commit run --files $(git diff --name-only claude/hil-pool-check..HEAD) && echo ALL-GREEN` +Expected: `ALL-GREEN`. + +- [ ] **Step 4: Real-diff spot checks (acceptance)** + +Run each and eyeball the JSON (record outputs in your report): +```bash +for f in 'src/portable/raspberrypi/rp2040/dcd_rp2040.c' 'src/device/usbd.c' 'src/class/cdc/cdc_device.c' 'src/host/usbh.c'; do + printf '%s\n' "$f" > /tmp/d.txt + echo "=== $f"; python3 test/hil/hil_select.py --diff-file /tmp/d.txt test/hil/tinyusb.json test/hil/hfp.json 2>/dev/null | python3 -m json.tool | sed -n '1,25p' +done +``` +Expected: matches the spec's acceptance examples (pico-family only / all-device / CDC examples +only / host side only, with hfp.json args populated only where hfp boards qualify). + +- [ ] **Step 5: Commit** + +```bash +git add .claude/skills/pre-pr/SKILL.md .claude/skills/hil/SKILL.md +git commit -m "docs: pre-pr and hil skill use hil_select for PR-scoped boards" +``` diff --git a/docs/superpowers/plans/2026-08-15-ci-hs-reset-edges.md b/docs/superpowers/plans/2026-08-15-ci-hs-reset-edges.md new file mode 100644 index 000000000..ec0834e55 --- /dev/null +++ b/docs/superpowers/plans/2026-08-15-ci-hs-reset-edges.md @@ -0,0 +1,782 @@ +# Bus-Reset Edge Events + Review Fix Wave Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Give the device stack a "bus reset started" event so ci_hs can tell usbd to stand down at the URI interrupt instead of up to 50 ms later, and clear the ten findings agreed from the max review. + +**Architecture:** `DCD_EVENT_BUS_RESET` splits into `DCD_EVENT_BUS_RESET_START` / `_END` with a compatibility alias, so every other port stays byte-identical. `dcd_ci_hs.c`'s `bus_reset()` splits along the register/software line — registers at URI (`_START`), software structures at the port-change ending the reset (`_END`) — which eliminates the window where usbd believes it is configured over zeroed queue heads. A single bounded-flush helper absorbs the five flush sites. Seven mechanical fixes follow. + +**Tech Stack:** C99, TinyUSB device stack (`src/device/`), ChipIdea HS DCD (`src/portable/chipidea/ci_hs/`), NXP IP3511 DCD (`src/portable/nxp/lpc_ip3511/`), CMake+Ninja and Make builds, J-Link flashing, `test/hil/` HIL harness. + +## Global Constraints + +- Branch `fix-ci-hs` in worktree `/home/hathach/.herdr/worktrees/tinyusb/fix-ci-hs`. Do NOT push; the user pushes. +- C99, 2-space indent, no tabs. Match each file's surrounding style (`dcd_lpc_ip3511.c` mixes styles — follow the immediate neighbourhood). +- Commit messages: imperative mood, no `Co-Authored-By:` or `Claude-Session:` trailers (repo rule: hathach is sole author). +- The repo pre-commit hook (trailing-whitespace, end-of-file-fixer, codespell, unique-PIDs, ceedling unit tests) must pass. If it rewrites a file, re-stage and retry the commit once. +- Comments: short, only the non-obvious "why". Cite manuals as `UM10503 25.10.3` / `Errata LPC546xx USB.13` style — never `ES_` prefixes. +- Never edit anything under `hw/mcu/` or `lib/` (vendor code). +- Build commands used throughout (each ~30-60 s): + `cmake --build examples/cmake-build-<board>` for `mimxrt1064_evk`, `lpcxpresso18s37`, `lpcxpresso11u37`, `lpcxpresso55s28`. +- Design source of truth: `docs/superpowers/specs/2026-08-15-ci-hs-reset-edges-design.md`. + +## File Structure + +| File | Responsibility in this plan | +|---|---| +| `src/device/dcd.h` | Event enum + compatibility alias + contract comment | +| `src/device/usbd.c` | Handle both reset edges; log strings; stop breakpointing on DCD refusal | +| `src/portable/chipidea/ci_hs/dcd_ci_hs.c` | Flush helper; `bus_reset()` split; setup-flush wait; `dcd_set_address`; RESUME guard | +| `src/portable/nxp/lpc_ip3511/dcd_lpc_ip3511.c` | Torn-setup delivery; USB.13 TODO token | +| `hw/bsp/lpc55/boards/lpcxpresso55s28/board.cmake` | Delete dead RHPORT block | +| `hw/bsp/lpc11/boards/lpcxpresso11u37/lpc11u37.ld` | Correct stale comment; relabel ASSERT | + +Tasks 1-3 are ordered (each builds on the previous); Tasks 4-6 are independent of each other. + +--- + +### Task 1: Split the bus-reset event into START/END edges + +**Files:** +- Modify: `src/device/dcd.h` (enum at lines 23-34; contract comment above it) +- Modify: `src/device/usbd.c` (`_usbd_event_str[]` at line 457; the `DCD_EVENT_BUS_RESET` case at line 700) + +**Interfaces:** +- Produces: `DCD_EVENT_BUS_RESET_START` and `DCD_EVENT_BUS_RESET_END` enum members; `#define DCD_EVENT_BUS_RESET DCD_EVENT_BUS_RESET_END`. Task 2 emits `_START` via the existing `dcd_event_bus_signal(uint8_t rhport, dcd_eventid_t eid, bool in_isr)` and `_END` via the existing `dcd_event_bus_reset(uint8_t rhport, tusb_speed_t speed, bool in_isr)`. + +- [ ] **Step 1: Replace the enum member in `src/device/dcd.h`** + +Replace: + +```c +typedef enum { + DCD_EVENT_INVALID = 0, // 0 + DCD_EVENT_BUS_RESET, // 1 + DCD_EVENT_UNPLUGGED, // 2 + DCD_EVENT_SOF, // 3 + DCD_EVENT_SUSPEND, // 4 TODO LPM Sleep L1 support + DCD_EVENT_RESUME, // 5 + DCD_EVENT_SETUP_RECEIVED, // 6 + DCD_EVENT_XFER_COMPLETE, // 7 + USBD_EVENT_FUNC_CALL, // 8 Not an DCD event, just a convenient way to defer ISR function + DCD_EVENT_COUNT +} dcd_eventid_t; +``` + +with: + +```c +// Bus reset is reported as two edges. BUS_RESET_START is optional: a controller that +// cannot tell the edges apart emits only BUS_RESET_END, which stays self-sufficient (it +// performs the full teardown with or without a preceding START). Emit START when reset +// signaling is detected - the link is unusable and the speed is not negotiated yet - so +// the stack stops using endpoints immediately instead of at the end of the reset. +typedef enum { + DCD_EVENT_INVALID = 0, // 0 + DCD_EVENT_BUS_RESET_START, // 1 + DCD_EVENT_BUS_RESET_END, // 2 with negotiated speed + DCD_EVENT_UNPLUGGED, // 3 + DCD_EVENT_SOF, // 4 + DCD_EVENT_SUSPEND, // 5 TODO LPM Sleep L1 support + DCD_EVENT_RESUME, // 6 + DCD_EVENT_SETUP_RECEIVED, // 7 + DCD_EVENT_XFER_COMPLETE, // 8 + USBD_EVENT_FUNC_CALL, // 9 Not an DCD event, just a convenient way to defer ISR function + DCD_EVENT_COUNT +} dcd_eventid_t; + +#define DCD_EVENT_BUS_RESET DCD_EVENT_BUS_RESET_END // backward compatibility +``` + +- [ ] **Step 2: Update the log-string table in `src/device/usbd.c`** + +At line 457 the table is indexed by event id and MUST stay in enum order. Replace the +`"Bus Reset",` entry (line 459) with two entries: + +```c + "Bus Reset Start", + "Bus Reset End", +``` + +- [ ] **Step 3: Handle both edges in the usbd task loop** + +Replace the case at `src/device/usbd.c:700`: + +```c + case DCD_EVENT_BUS_RESET: + TU_LOG_USBD(": %s Speed\r\n", tu_str_speed[event.bus_reset.speed]); + usbd_reset(event.rhport); + _usbd_dev.speed = event.bus_reset.speed; + break; +``` + +with: + +```c + case DCD_EVENT_BUS_RESET_START: + TU_LOG_USBD("\r\n"); + usbd_reset(event.rhport); + break; + + case DCD_EVENT_BUS_RESET_END: + TU_LOG_USBD(": %s Speed\r\n", tu_str_speed[event.bus_reset.speed]); + // TODO a DCD that reports both edges pays for two teardowns: track a per-rhport + // "start seen" flag and skip this reset, keeping it for the single-event DCDs. + usbd_reset(event.rhport); + _usbd_dev.speed = event.bus_reset.speed; + break; +``` + +- [ ] **Step 4: Verify legacy ports still build (the alias must carry them)** + +Run: + +```bash +cd examples && cmake -B cmake-build-stm32f407disco -DBOARD=stm32f407disco -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel . && cmake --build cmake-build-stm32f407disco +``` + +Expected: builds clean. This board's DCD (dwc2) still calls `dcd_event_bus_reset()`, which +now resolves to `_END` through the unchanged helper — proving the alias works. + +- [ ] **Step 5: Verify the unit tests still build and pass** + +Run: `cd test/unit-test && ceedling test:all` +Expected: all tests pass (they reference `DCD_EVENT_BUS_RESET` via the alias). + +- [ ] **Step 6: Commit** + +```bash +git add src/device/dcd.h src/device/usbd.c +git commit -m "usbd: split bus reset into start/end edge events + +A DCD that can see reset signaling begin has no way to say so: the only +event carries the negotiated speed, which does not exist until the reset +ends. On ChipIdea that leaves the stack believing it is configured for the +whole reset window (3 ms minimum, tens of ms in practice) while the +controller has already torn its endpoints down. + +Add DCD_EVENT_BUS_RESET_START for the leading edge and rename the existing +event to DCD_EVENT_BUS_RESET_END, keeping DCD_EVENT_BUS_RESET as an alias +so every other port and the unit tests are untouched. START is optional and +END stays self-sufficient, so single-event drivers keep working unchanged." +``` + +--- + +### Task 2: Split ci_hs `bus_reset()` across the two edges, behind one flush helper + +**Files:** +- Modify: `src/portable/chipidea/ci_hs/dcd_ci_hs.c` (`bus_reset()`; `dcd_deinit()`; `dcd_edpt_iso_activate()`; the `INTR_RESET` and `INTR_PORT_CHANGE` branches of `dcd_int_handler()`) + +**Interfaces:** +- Consumes: `DCD_EVENT_BUS_RESET_START` (Task 1), `dcd_event_bus_signal()`, `dcd_event_bus_reset()`. +- Produces: `static bool flush_endpoints(ci_hs_regs_t *dcd_reg, uint32_t mask)` — writes `ENDPTFLUSH = mask`, spins bounded by `CI_HS_BUSY_SPIN`, returns `true` if the bits cleared. Used by Task 3. + +- [ ] **Step 1: Add the flush helper next to `bus_reset()`** + +Insert above `bus_reset()`: + +```c +// Flush endpoint buffers and wait for the controller to acknowledge. Callers proceed +// regardless of the result; the bound only prevents an ISR-context hang on dead hardware. +static bool flush_endpoints(ci_hs_regs_t *dcd_reg, uint32_t mask) { + dcd_reg->ENDPTFLUSH = mask; + uint32_t guard = CI_HS_BUSY_SPIN; + while (dcd_reg->ENDPTFLUSH & mask) { + if (!guard--) { + return false; + } + } + return true; +} +``` + +- [ ] **Step 2: Split `bus_reset()` into begin/complete** + +Replace the whole `bus_reset()` function with these two. `bus_reset_begin()` keeps only +register work; `bus_reset_complete()` owns everything that touches `_dcd_data`: + +```c +/// Register-side reset handling, must run inside the reset window (UM10503 25.10.3) +static void bus_reset_begin(uint8_t rhport) { + ci_hs_regs_t *dcd_reg = CI_HS_REG(rhport); + + // The reset value for all endpoint types is the control endpoint. If one endpoint + // direction is enabled and the paired endpoint of opposite direction is disabled, then the + // endpoint type of the unused direction must be changed from the control type to any other + // type (e.g. bulk). Leaving an un-configured endpoint control will cause undefined behavior + // for the data PID tracking on the active endpoint. + const uint8_t ep_count = ci_ep_count(dcd_reg); + for (uint8_t i = 1; i < ep_count; i++) { + dcd_reg->ENDPTCTRL[i] = ENDPTCTRL_RESET_MASK; + } + + //------------- Clear All Registers -------------// + dcd_reg->ENDPTNAK = dcd_reg->ENDPTNAK; + dcd_reg->ENDPTNAKEN = 0; + dcd_reg->ENDPTSETUPSTAT = dcd_reg->ENDPTSETUPSTAT; + dcd_reg->ENDPTCOMPLETE = dcd_reg->ENDPTCOMPLETE; + + uint32_t guard = CI_HS_BUSY_SPIN; + while (dcd_reg->ENDPTPRIME && guard--) {} + flush_endpoints(dcd_reg, 0xFFFFFFFF); +} + +/// Software-side reset handling, deferred to the port change ending the reset so the queue +/// heads stay coherent until the stack is told - and so a prime issued by a task that had +/// not yet seen BUS_RESET_START is flushed here rather than surviving re-enumeration. +static void bus_reset_complete(uint8_t rhport) { + ci_hs_regs_t *dcd_reg = CI_HS_REG(rhport); + flush_endpoints(dcd_reg, 0xFFFFFFFF); + + //------------- Queue Head & Queue TD -------------// + tu_memclr(&_dcd_data, sizeof(dcd_data_t)); + + //------------- Set up Control Endpoints (0 OUT, 1 IN) -------------// + _dcd_data.qhd[0][0].zero_length_termination = _dcd_data.qhd[0][1].zero_length_termination = 1; + _dcd_data.qhd[0][0].max_packet_size = _dcd_data.qhd[0][1].max_packet_size = CFG_TUD_ENDPOINT0_SIZE; + _dcd_data.qhd[0][0].qtd_overlay.next = _dcd_data.qhd[0][1].qtd_overlay.next = QTD_NEXT_INVALID; + + _dcd_data.qhd[0][0].int_on_setup = 1; // OUT only + + dcd_dcache_clean_invalidate(&_dcd_data, sizeof(dcd_data_t)); +} +``` + +- [ ] **Step 3: Route the two ISR branches to the new functions** + +In `dcd_int_handler()`, the `INTR_RESET` branch becomes: + +```c + if (int_status & INTR_RESET) { + bus_reset_begin(rhport); + _port_change_reason[rhport] = PORT_CHANGE_REASON_RESET; + dcd_event_bus_signal(rhport, DCD_EVENT_BUS_RESET_START, true); + } +``` + +and inside the `INTR_PORT_CHANGE` branch, the reset arm (the `else` of the resume test) +becomes: + +```c + } else { + bus_reset_complete(rhport); + // PSPD: 0 full, 1 low, 2 high, 3 undefined (treated as full) + const uint32_t pspd = (dcd_reg->PORTSC1 & PORTSC1_PORT_SPEED) >> PORTSC1_PORT_SPEED_POS; + const tusb_speed_t speed = (pspd == 1) ? TUSB_SPEED_LOW : (pspd == 2) ? TUSB_SPEED_HIGH : TUSB_SPEED_FULL; + dcd_event_bus_reset(rhport, speed, true); + } +``` + +Delete the now-unused EP0 `ENDPTFLUSH` line that previously sat at the top of that arm — +`bus_reset_complete()` flushes all endpoints. + +- [ ] **Step 4: Route the remaining flush sites through the helper** + +In `dcd_deinit()`, replace the flush block with: + +```c + // flush all endpoints + uint32_t guard = CI_HS_BUSY_SPIN; + while (dcd_reg->ENDPTPRIME && guard--) {} + flush_endpoints(dcd_reg, 0xFFFFFFFF); +``` + +In `dcd_edpt_iso_activate()`, replace the flush + spin with: + +```c + // Flush EP + flush_endpoints(dcd_reg, TU_BIT(epnum + (dir ? 16 : 0))); +``` + +- [ ] **Step 5: Build both ci_hs board families** + +Run: + +```bash +cmake --build examples/cmake-build-mimxrt1064_evk && cmake --build examples/cmake-build-lpcxpresso18s37 +``` + +Expected: both succeed with no new warnings. + +- [ ] **Step 6: Commit** + +```bash +git add src/portable/chipidea/ci_hs/dcd_ci_hs.c +git commit -m "dcd(ci_hs): report bus reset start at URI, finish at port change + +The RM wants the reset cleanup inside the reset window, but the negotiated +speed only exists once the port reaches its operational state, so the stack +was told nothing for the whole window - it kept believing it was configured +while the queue heads had been zeroed under it, and a transfer a class +driver started in that gap stayed primed across re-enumeration. + +Split the work along the register/software line: bus_reset_begin() does the +register cleanup at URI and signals BUS_RESET_START, bus_reset_complete() +re-flushes, resets the queue heads and reports BUS_RESET_END with the final +speed at the port change. Zeroing the queue heads now happens in the same +breath as telling the stack, and the second flush retires anything primed +in between. + +Fold the five hand-rolled endpoint flushes into one bounded helper while +the reset path is open." +``` + +--- + +### Task 3: Make the setup-time EP0 flush wait, and stop dropping the SET_ADDRESS status prime + +**Files:** +- Modify: `src/portable/chipidea/ci_hs/dcd_ci_hs.c` (`dcd_set_address()`; the `ENDPTSETUPSTAT` branch inside `dcd_int_handler()`) + +**Interfaces:** +- Consumes: `flush_endpoints()` (Task 2); `qhd_start_xfer()` returning `bool`, already propagated by `dcd_edpt_xfer()`. + +- [ ] **Step 1: Wait for the setup-time flush to complete** + +In the ISR's setup branch, replace the fire-and-forget flush line + +```c + dcd_reg->ENDPTFLUSH = TU_BIT(0) | TU_BIT(16); +``` + +with + +```c + // Wait it out: the flush retires a status/handshake phase left primed by the previous + // control sequence (UM10503 25.10.8.1.1), and an unfinished flush would otherwise + // still be asserted when the task primes the response to this setup and would retire + // that instead. A flush waits for any packet already in progress - microseconds at + // high speed - and the guard caps wedged hardware. + flush_endpoints(dcd_reg, TU_BIT(0) | TU_BIT(16)); +``` + +- [ ] **Step 2: Honour the status-prime result in `dcd_set_address`** + +Replace the body of `dcd_set_address()`: + +```c +void dcd_set_address(uint8_t rhport, uint8_t dev_addr) { + // Response with status first before changing device address. A refused prime means a new + // setup superseded this transfer; staging an address whose ACK will never arrive would + // leave the device answering on it, so only arm the address when the status went out. + if (dcd_edpt_xfer(rhport, tu_edpt_addr(0, TUSB_DIR_IN), NULL, 0, false)) { + ci_hs_regs_t *dcd_reg = CI_HS_REG(rhport); + dcd_reg->DEVICEADDR = (dev_addr << 25) | TU_BIT(24); + } +} +``` + +- [ ] **Step 3: Build and commit** + +Run: `cmake --build examples/cmake-build-mimxrt1064_evk && cmake --build examples/cmake-build-lpcxpresso18s37` +Expected: both succeed. + +```bash +git add src/portable/chipidea/ci_hs/dcd_ci_hs.c +git commit -m "dcd(ci_hs): wait out the setup flush, honour the set-address prime + +The flush issued on every new setup was fire-and-forget. A flush waits for +a packet already in progress, so it could still be asserted when the task +primed the response to that setup and retire the fresh prime instead - +leaving EP0 silent until the host gave up. + +dcd_set_address() also armed DEVICEADDR unconditionally, but the status +prime can now be refused when a newer setup supersedes the transfer; the +address was then staged behind an ACK that never came and the device sat at +address 0. Only arm it when the status transfer actually started." +``` + +--- + +### Task 4: Emit RESUME only when the port really left suspend + +**Files:** +- Modify: `src/portable/chipidea/ci_hs/dcd_ci_hs.c` (the resume arm of the `INTR_PORT_CHANGE` branch in `dcd_int_handler()`) + +**Interfaces:** none consumed or produced. + +- [ ] **Step 1: Restore the hardware guard** + +In the `INTR_PORT_CHANGE` branch, the resume arm currently reads: + +```c + if (pci_reason == PORT_CHANGE_REASON_SUSPEND) { + dcd_event_bus_signal(rhport, DCD_EVENT_RESUME, true); + } else { +``` + +Replace that condition with one that also consults live hardware: + +```c + if (pci_reason == PORT_CHANGE_REASON_SUSPEND) { + // Only when the port actually left suspend: a starved snapshot can hold the resume's + // port change together with a second suspend, and reporting a resume there would + // leave the stack awake on a sleeping bus with no further event to correct it. + if (!(dcd_reg->PORTSC1 & PORTSC1_SUSPEND)) { + dcd_event_bus_signal(rhport, DCD_EVENT_RESUME, true); + } + } else { +``` + +- [ ] **Step 2: Build and commit** + +Run: `cmake --build examples/cmake-build-mimxrt1064_evk && cmake --build examples/cmake-build-lpcxpresso18s37` +Expected: both succeed. + +```bash +git add src/portable/chipidea/ci_hs/dcd_ci_hs.c +git commit -m "dcd(ci_hs): only report resume when the port left suspend + +A suspend, resume and second suspend collapsed into one interrupt pass +queued suspend then resume from the recorded cause alone, so the stack +ended up awake while the bus was still suspended and nothing arrived to +correct it. Consult PORTSC1 before reporting the resume." +``` + +--- + +### Task 5: ip3511 — never deliver a knowingly-torn setup packet + +**Files:** +- Modify: `src/portable/nxp/lpc_ip3511/dcd_lpc_ip3511.c` (setup branch of `dcd_int_handler()`; the `dcd_edpt_clear_stall()` comment) + +**Interfaces:** none consumed or produced. + +- [ ] **Step 1: Deliver only when the copy is known good** + +Replace: + +```c + // a SETUP that raced in after the acks (its bit0 consumed above, this copy possibly torn): + // its latch is visible again - re-raise the endpoint interrupt so the next pass redelivers + // the newer payload + if (dcd_reg->DEVCMDSTAT & DEVCMDSTAT_SETUP_RECEIVED_MASK) { + dcd_reg->INTSETSTAT = TU_BIT(0); + } + + dcd_event_setup_received(rhport, setup_copy, true); +``` + +with: + +```c + // a SETUP that raced in after the acks (its bit0 consumed above) makes this copy suspect: + // its latch is visible again, so re-raise the endpoint interrupt and let the next pass + // deliver the newer payload rather than passing up bytes that may be torn between the two + if (dcd_reg->DEVCMDSTAT & DEVCMDSTAT_SETUP_RECEIVED_MASK) { + dcd_reg->INTSETSTAT = TU_BIT(0); + } else { + dcd_event_setup_received(rhport, setup_copy, true); + } +``` + +- [ ] **Step 2: Add the TODO token to the USB.13 deferral** + +In `dcd_edpt_clear_stall()`, change the caveat's opening line from + +```c + // Known caveat (Errata LPC546xx USB.13, same semantics in UM11126): with RF/TV preserved at 1, TR +``` + +to + +```c + // TODO implement the Errata LPC546xx USB.13 work-around (same semantics in UM11126): with RF/TV preserved at 1, TR +``` + +- [ ] **Step 3: Build and commit** + +Run: `cmake --build examples/cmake-build-lpcxpresso11u37 && cmake --build examples/cmake-build-lpcxpresso55s28` +Expected: both succeed. + +```bash +git add src/portable/nxp/lpc_ip3511/dcd_lpc_ip3511.c +git commit -m "dcd(ip3511): drop a setup packet the hardware may have overwritten + +The handler already notices when a new setup landed while it was copying +the previous one, and re-raises the endpoint interrupt so the newer payload +is delivered next pass - but it then passed the suspect copy up anyway. +Usually harmless, since the redelivery supersedes it, but if that second +event cannot be queued the torn bytes are processed as a real request. +Deliver the copy only when no newer setup is pending." +``` + +--- + +### Task 6: BSP cleanups — dead RHPORT block and the stale linker comment + +**Files:** +- Modify: `hw/bsp/lpc55/boards/lpcxpresso55s28/board.cmake` +- Modify: `hw/bsp/lpc11/boards/lpcxpresso11u37/lpc11u37.ld` + +**Interfaces:** none consumed or produced. + +- [ ] **Step 1: Delete the redundant RHPORT block** + +`hw/bsp/lpc55/family.cmake` already applies the identical guarded defaults (`RHPORT_DEVICE 1`, +`RHPORT_HOST 0`) after including the board file, so remove these lines from +`board.cmake` entirely: + +```cmake +# device highspeed, host fullspeed; guarded so a -D override on the cmake command line wins +if (NOT DEFINED RHPORT_DEVICE) + set(RHPORT_DEVICE 1) +endif () +if (NOT DEFINED RHPORT_HOST) + set(RHPORT_HOST 0) +endif () +``` + +Leave `board.mk`'s `RHPORT_DEVICE ?= 1` / `RHPORT_HOST ?= 0` alone — `?=` is the idiomatic +Make form and matches sibling boards. + +- [ ] **Step 2: Prove the defaults and the override still work** + +Run: + +```bash +cd examples && rm -rf /tmp/rh-default /tmp/rh-override +cmake -B /tmp/rh-default -DBOARD=lpcxpresso55s28 -G Ninja . > /tmp/rh-default.log 2>&1 +grep -m1 "RHPORT_DEVICE" /tmp/rh-default.log || cmake -B /tmp/rh-default -DBOARD=lpcxpresso55s28 -G Ninja -LA . | grep -E "^RHPORT_(DEVICE|HOST)" +cmake -B /tmp/rh-override -DBOARD=lpcxpresso55s28 -DRHPORT_DEVICE=0 -DRHPORT_HOST=1 -G Ninja -LA . | grep -E "^RHPORT_(DEVICE|HOST)" +``` + +Expected: the default configure yields device 1 / host 0; the override configure yields +device 0 / host 1. Then rebuild the real tree: `cmake --build cmake-build-lpcxpresso55s28`. + +- [ ] **Step 3: Correct the linker-script comment and relabel the ASSERT** + +In `lpc11u37.ld`, replace the comment block above `__user_stack_top` and the ASSERT with: + +```text + /* Main (MSP/ISR) stack lives at the top of the USB SRAM bank: the 8K main bank is packed so + tight that only ~280 B remained above .bss, and ISR frames overflowed into the topmost task + stack (cdc_msc_freertos hard fault). Nothing else is placed in this bank in either build + system, so the stack owns all 2 KB; the ASSERT is future-proofing in case USB buffers are + ever mapped here again. */ + __user_stack_top = ORIGIN(RamUsb2) + LENGTH(RamUsb2); + ASSERT(__user_stack_top - (ADDR(.noinit_RAM2) + SIZEOF(.noinit_RAM2)) >= 0x200, + "main stack headroom in RamUsb2 below 512 bytes") +``` + +- [ ] **Step 4: Build both build systems for lpc11u37** + +Run: + +```bash +cmake --build examples/cmake-build-lpcxpresso11u37 +cd examples/device/cdc_msc_freertos && make -j8 BOARD=lpcxpresso11u37 all && cd ../../.. +``` + +Expected: both succeed. + +- [ ] **Step 5: Commit** + +```bash +git add hw/bsp/lpc55/boards/lpcxpresso55s28/board.cmake hw/bsp/lpc11/boards/lpcxpresso11u37/lpc11u37.ld +git commit -m "bsp: drop duplicated lpc55s28 rhport defaults, fix lpc11u37 comment + +hw/bsp/lpc55/family.cmake already applies the same guarded rhport defaults +after including the board file, so the board-level copy only added a second +place to keep in sync. + +The lpc11u37 linker comment still described USB buffers living in RamUsb2, +a placement the same branch removed; nothing lands there now, so say so and +label the headroom assert as future-proofing." +``` + +--- + +### Task 7: Stop halting the target when a DCD legitimately refuses a transfer + +**Files:** +- Modify: `src/device/usbd.c` (`usbd_edpt_xfer()` failure arm) + +**Interfaces:** none consumed or produced. + +- [ ] **Step 1: Remove the breakpoint from the DCD-refusal path** + +Replace the failure arm of `usbd_edpt_xfer()`: + +```c + } else { + // DCD error, mark endpoint as ready to allow next transfer + _usbd_dev.ep_status[epnum][dir] &= (uint8_t) ~(TU_EDPT_STATE_BUSY | TU_EDPT_STATE_CLAIMED); + TU_LOG_USBD("FAILED\r\n"); + TU_BREAKPOINT(); + return false; + } +``` + +with: + +```c + } else { + // Driver refused the transfer, mark endpoint as ready to allow next transfer. This is a + // recoverable condition (e.g. a new setup superseding a control response), not a bug, so + // do not break into the debugger - TU_BREAKPOINT() halts the CPU whenever a probe is + // attached, which on a test rig is always. + _usbd_dev.ep_status[epnum][dir] &= (uint8_t) ~(TU_EDPT_STATE_BUSY | TU_EDPT_STATE_CLAIMED); + TU_LOG_USBD("FAILED\r\n"); + return false; + } +``` + +- [ ] **Step 2: Confirm no other stack path relies on that breakpoint** + +Run: `grep -n "TU_BREAKPOINT" src/device/*.c src/device/*.h` +Expected: no remaining hits inside `usbd_edpt_xfer`; other occurrences (if any) are in +unrelated assert macros and stay as they are. + +- [ ] **Step 3: Build and run unit tests** + +Run: + +```bash +cmake --build examples/cmake-build-mimxrt1064_evk +cd test/unit-test && ceedling test:all && cd ../.. +``` + +Expected: build succeeds, all unit tests pass. + +- [ ] **Step 4: Commit** + +```bash +git add src/device/usbd.c +git commit -m "usbd: do not breakpoint when a driver refuses a transfer + +TU_BREAKPOINT() is not gated on CFG_TUSB_DEBUG - it halts the CPU whenever +a debugger is attached, which on a test rig is always. A driver declining a +transfer is recoverable (a new setup superseding a control response, for +one) and the endpoint is already released for the retry, so a halted target +turns a self-healing case into a dead board." +``` + +--- + +### Task 8: Full validation on hardware + +**Files:** none modified — this task produces the evidence for the PR description. + +**Interfaces:** consumes the firmware built by Tasks 1-7. + +- [ ] **Step 1: Software gate** + +Run: + +```bash +pre-commit run --all-files +cd examples +for b in mimxrt1064_evk lpcxpresso18s37 lpcxpresso11u37 lpcxpresso55s28; do + rm -rf cmake-build-$b && cmake -B cmake-build-$b -DBOARD=$b -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel . && cmake --build cmake-build-$b || echo "FAILED $b" +done +cd .. +``` + +Expected: pre-commit all green; all four boards build every example. + +- [ ] **Step 2: Make-build regression checks** + +Run: + +```bash +cd examples/host/cdc_msc_hid && make -j8 BOARD=lpcxpresso55s28 all && cd ../../.. +cd examples/device/cdc_msc_throughput && make -j8 BOARD=lpcxpresso11u37 all && cd ../../.. +``` + +Expected: both link (these two were broken earlier in the branch and are the regression +canaries for the BSP changes). + +- [ ] **Step 3: Flash with verification (mandatory)** + +The mimxrt1064_evk has twice accepted a flash that silently did not take, so every load in +this task uses `verifyfile`. For each board, write a J-Link script of this shape and run it: + +``` +r +h +loadfile examples/cmake-build-<board>/device/usbtest/usbtest.elf +verifyfile examples/cmake-build-<board>/device/usbtest/usbtest.elf +r +g +qc +``` + +Probes and devices: `mimxrt1064_evk` = `-USB 000725299165 -device MIMXRT1064xxx6A`, +`lpcxpresso55s28` = `-USB 000727031389 -device LPC55S28`, +`lpcxpresso11u37` = `-USB 000724441579 -device LPC11U37/401`. +Invoke as `JLinkExe <probe/device args> -if swd -speed 4000 -autoconnect 1 -NoGui 1 -CommandFile <script>`. +Expected: `Verify` reports O.K. and the board re-enumerates as `cafe:4010` with its own +serial before any test runs. + +- [ ] **Step 4: HIL batteries and stress** + +Hold each board's lock for its own leg (`python3 test/hil/hil_lock.py hold <board> --reason "reset-edge validation"`, +release after), never run two batteries at once, and abort if CI is active +(`pgrep -f "hil_test.py [-]-retry"`). + +```bash +# per board: full battery +timeout 700 python3 test/hil/usbtest.py --serial <serial> --json --keep-binding --timeout 60 + +# mimxrt1064_evk only: queued-control stress and the unlink storm +for i in $(seq 1 50); do timeout 200 python3 test/hil/usbtest.py --serial BAE96FB95AFA6DBB8F00005002001200 --tests 9,10 --json --keep-binding --timeout 60 > /dev/null || break; done +for i in $(seq 1 10); do timeout 300 python3 test/hil/usbtest.py --serial BAE96FB95AFA6DBB8F00005002001200 --tests 11,12,24 --json --keep-binding --timeout 60 > /dev/null || break; done +``` + +Serials: 1064 `BAE96FB95AFA6DBB8F00005002001200`, 55s28 `2BF1839A7D51F553A15AB03FD08F70AB`, +11u37 `17121919`. +Expected: 30/30 on all three boards, 50/50 and 10/10 loops, and +`ps -eo stat,comm | awk '$1 ~ /^D/'` empty after each leg. + +- [ ] **Step 5: Reset-path evidence with logging** + +Build and flash `device/cdc_msc` for `mimxrt1064_evk` with `-DLOG=2 -DLOGGER=rtt`, capture +RTT during one unplug/replug cycle (`timeout 20s JLinkRTTClient > /tmp/reset.log`), then: + +```bash +grep -cE "Bus Reset Start" /tmp/reset.log +grep -cE "Bus Reset End" /tmp/reset.log +grep -c "Resume" /tmp/reset.log +``` + +Expected: equal non-zero counts for start and end (one pair per enumeration) and no +`Resume` lines during a plain plug-in. + +- [ ] **Step 6: Suspend/resume pairing** + +With the same RTT build attached, suspend the port from the host and resume it: + +```bash +# find the 1064's busport, then: +echo auto | sudo tee /sys/bus/usb/devices/<busport>/power/control +sleep 5 +echo on | sudo tee /sys/bus/usb/devices/<busport>/power/control +``` + +Expected in the log: one `Suspend` followed by one `Resume`, and no `Bus Reset` of either +edge from the suspend cycle alone. + +- [ ] **Step 7: Record the evidence** + +Append the numbers from Steps 1-6 to the PR description draft. No commit. + +## Self-Review + +**Spec coverage:** §1 event split → Task 1. §2 ci_hs bus_reset split → Task 2. §3 flush +helper → Task 2 (Steps 1, 4). §4 mechanical: setup-flush wait and `dcd_set_address` → Task 3; +RESUME guard → Task 4; ip3511 torn setup and USB.13 TODO → Task 5; usbd breakpoint → Task 7; +BSP pair → Task 6. Verification matrix → Task 8 (legacy-DCD build guard is Task 1 Step 4). +Deferred items are deliberately absent from every task. No gaps. + +**Placeholder scan:** no TBD/TODO-as-placeholder; the two literal `TODO` strings are +deliverable code comments (Task 1 Step 3, Task 5 Step 2). Every code step carries the exact +text to write; every run step carries the command and expected result. + +**Type consistency:** `flush_endpoints(ci_hs_regs_t *dcd_reg, uint32_t mask) -> bool` is +defined in Task 2 Step 1 and used with that exact signature in Task 2 Steps 2/4 and Task 3 +Step 1. `DCD_EVENT_BUS_RESET_START` / `_END` are defined in Task 1 and used in Task 2 Step 3 +via `dcd_event_bus_signal()` / `dcd_event_bus_reset()`, whose signatures are quoted in Task 1's +Interfaces block. `bus_reset_begin()` / `bus_reset_complete()` are defined and called with +matching names in Task 2. diff --git a/docs/superpowers/plans/2026-08-16-drop-ep0-prime-verify.md b/docs/superpowers/plans/2026-08-16-drop-ep0-prime-verify.md new file mode 100644 index 000000000..aa999c9e3 --- /dev/null +++ b/docs/superpowers/plans/2026-08-16-drop-ep0-prime-verify.md @@ -0,0 +1,314 @@ +# Drop the EP0 Post-Prime Verify Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Remove the EP0 post-prime verification that was built on a theory the RT106x endpoint-conflict errata has superseded, and prove on hardware that nothing depended on it. + +**Architecture:** One deletion in `qhd_start_xfer()`, then a rebase onto current master, then an A/B validation whose "with it" arm is already banked (10x 30/30 batteries plus 40 targeted loops on 2026-08-16). No interfaces change: the pre-prime setup-lockout guard keeps `qhd_start_xfer()` returning `bool`, so `dcd_set_address()`'s gating and usbd's failure path stay exactly as they are. + +**Tech Stack:** C99, TinyUSB ChipIdea HS DCD (`src/portable/chipidea/ci_hs/`), CMake+Ninja and Make builds, J-Link (JLinkExe V9.66), `test/hil/usbtest.py` driving the Linux testusb battery. + +## Global Constraints + +- Branch `fix-ci-hs` in worktree `/home/hathach/.herdr/worktrees/tinyusb/fix-ci-hs`. Do NOT push; the user pushes. +- C99, 2-space indent. Commit messages imperative, no `Co-Authored-By:` or `Claude-Session:` trailers (repo rule: hathach is sole author). +- Pre-commit hook (trailing-whitespace, end-of-file-fixer, codespell, unique-PIDs, ceedling) must pass; if it rewrites a file, re-stage and retry the commit once. +- Never edit anything under `hw/mcu/` or `lib/` (vendor code). +- Rig etiquette: hold the board lock for hardware work (`python3 test/hil/hil_lock.py hold <board> --reason "..."`, release after); abort if CI is active (`pgrep -f "hil_test.py [-]-retry"`); NEVER use `uhubctl`, `pci-reset` or `pci-rebind`; never touch the actions-runner. +- JLinkExe on this rig is **V9.66 and has no `verifyfile` command** — use `loadfile` (built-in Program & Verify) plus a mandatory enumeration check. +- Board facts: `mimxrt1064_evk`, serial `BAE96FB95AFA6DBB8F00005002001200`, J-Link probe `000725299165`, device `MIMXRT1064xxx6A`, expected `cafe:4010`. +- Design source of truth: `docs/superpowers/specs/2026-08-16-drop-ep0-prime-verify-design.md`. + +## File Structure + +| File | Responsibility in this plan | +|---|---| +| `src/portable/chipidea/ci_hs/dcd_ci_hs.c` | The only code change: delete the post-prime block in `qhd_start_xfer()` | + +Tasks 2 and 3 change no files; they rebase and validate. + +--- + +### Task 1: Delete the EP0 post-prime verify + +**Files:** +- Modify: `src/portable/chipidea/ci_hs/dcd_ci_hs.c` (the tail of `qhd_start_xfer()`) + +**Interfaces:** +- Produces: `qhd_start_xfer()` keeps its existing signature `static bool qhd_start_xfer(uint8_t rhport, uint8_t epnum, uint8_t dir)` and still returns `false` from the pre-prime setup-lockout guard. No caller changes. + +- [ ] **Step 1: Apply the deletion** + +In `qhd_start_xfer()`, replace this (everything from the prime write to the closing `return true;`): + +```c + // start transfer + const uint32_t prime_bit = TU_BIT(epnum + (dir ? 16 : 0)); + dcd_reg->ENDPTPRIME = prime_bit; + + if (epnum == 0) { + // RM (RT1050 RM Executing a Transfer / UM10503 25.10.8): after priming EP0 the DCD must + // verify the prime completed - ENDPTPRIME bit clear AND the buffer reported ready in + // ENDPTSTAT - because the controller silently cancels an EP0 prime when a SETUP arrives + // during the prime operation. An undetected drop NAK-parks the endpoint forever: usbd never + // re-primes a busy endpoint. A very fast transfer may already have completed and retired the + // ENDPTSTAT bit, so ENDPTCOMPLETE also counts as the prime having taken. + uint32_t guard = CI_HS_BUSY_SPIN; + while (dcd_reg->ENDPTPRIME & prime_bit) { + if (!guard--) { + dcd_reg->ENDPTFLUSH = prime_bit; // never leave a wedged prime armed over a freed buffer + return false; + } + } + // Fail only when the cancel-cause is visibly pending: a completed transfer can have both + // status bits already retired by the ISR, and a cancel whose SETUP the ISR consumed is + // re-driven by that queued SETUP event anyway. + if (!((dcd_reg->ENDPTSTAT | dcd_reg->ENDPTCOMPLETE) & prime_bit) && + (dcd_reg->ENDPTSETUPSTAT & TU_BIT(0))) { + return false; // prime cancelled (setup mid-prime): the pending SETUP re-drives EP0 + } + } + return true; +``` + +with: + +```c + // start transfer + dcd_reg->ENDPTPRIME = TU_BIT(epnum + (dir ? 16 : 0)); + return true; +``` + +Leave the `if (epnum == 0)` setup-lockout block ABOVE the prime write completely untouched — +that one spins on `ENDPTSETUPSTAT` before priming and is required by UM10503 25.10.8.1.1 +step 4. + +- [ ] **Step 2: Confirm nothing else referenced the removed code** + +Run: + +```bash +grep -n "ENDPTSTAT\|ENDPTCOMPLETE\|prime_bit" src/portable/chipidea/ci_hs/dcd_ci_hs.c +``` + +Expected: no `prime_bit` hits at all; `ENDPTCOMPLETE` hits only in `bus_reset_begin()` and the +`INTR_USB` branch of `dcd_int_handler()`; `ENDPTSTAT` hits only in `ci_hs_type.h`-style register +declarations if any appear — none inside `qhd_start_xfer()`. + +- [ ] **Step 3: Build both ci_hs board families** + +Run: + +```bash +cmake --build examples/cmake-build-mimxrt1064_evk && cmake --build examples/cmake-build-lpcxpresso18s37 +``` + +Expected: both succeed, no new warnings (in particular no "unused variable" for anything the +deletion orphaned). + +- [ ] **Step 4: Commit** + +```bash +git add src/portable/chipidea/ci_hs/dcd_ci_hs.c +git commit -m "dcd(ci_hs): drop the EP0 post-prime verify + +The verify came from a theory that a setup arriving mid-prime silently +cancels an EP0 prime, which was how the recurring wedge on the test rig +looked at the time. The wedge turned out to be Errata i.MX RT1064_A +ERR050101: with an isochronous IN endpoint active, an IN token to that +endpoint number on another device sharing the host unprimes one of our OUT +endpoints, undetectably and with no interrupt. Moving the usbtest iso IN +endpoint clear of the conflict fixed it - 340 runs where the board used to +wedge within hours. + +The capture that motivated the verify (EP0 status stage armed but unprimed, +device a control transfer ahead of the host) is explained by that errata +just as well, because it covers control OUT endpoints and a control status +stage is one. So the verify has no independent evidence behind it, while it +does cost two register spins on every EP0 transfer and can misread a +transfer the interrupt handler already completed as a cancelled prime. + +The setup-lockout check before priming stays - that one is in the manual." +``` + +--- + +### Task 2: Rebase onto current master and re-run the software gates + +**Files:** none modified by hand. + +**Interfaces:** none. + +- [ ] **Step 1: Rebase** + +Master has advanced (midi2/usbtmc/video changes) since this branch last rebased. Validating a +tree that is not the one being merged would be a false pass. + +```bash +git fetch origin master +git rebase origin/master +``` + +Expected: clean rebase. If a conflict appears in `src/portable/chipidea/ci_hs/dcd_ci_hs.c` or +`src/device/usbd.c`, resolve it hunk-by-hunk keeping BOTH sides' intent (never `git checkout +--theirs/--ours` on a whole file), then `git rebase --continue`. + +- [ ] **Step 2: Rebuild everything from scratch** + +```bash +cd examples +for b in mimxrt1064_evk lpcxpresso18s37 lpcxpresso11u37 lpcxpresso55s28; do + rm -rf cmake-build-$b + cmake -B cmake-build-$b -DBOARD=$b -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel . && cmake --build cmake-build-$b || echo "FAILED $b" +done +cd .. +``` + +Expected: all four boards build every example, no "FAILED" line. + +- [ ] **Step 3: Make link canaries** + +```bash +cd examples/host/cdc_msc_hid && make -j8 BOARD=lpcxpresso55s28 all && cd ../../.. +cd examples/device/cdc_msc_throughput && make -j8 BOARD=lpcxpresso11u37 all && cd ../../.. +``` + +Expected: both link. These two were broken earlier in the branch's life and are the regression +canaries for the BSP changes. + +- [ ] **Step 4: Unit tests and pre-commit** + +```bash +cd test/unit-test && ceedling test:all && cd ../.. +pre-commit run --all-files +``` + +Expected: all unit tests pass; every pre-commit hook passes. + +- [ ] **Step 5: No commit** + +This task produces no commit of its own — the rebase rewrites existing commits and the builds +are throwaway. Record the resulting HEAD hash in the report for Task 3 to reference. + +--- + +### Task 3: Hardware A/B on mimxrt1064_evk + +**Files:** none modified — this task produces the evidence. + +**Interfaces:** consumes the firmware built in Task 2 at +`examples/cmake-build-mimxrt1064_evk/device/usbtest/usbtest.elf`. + +Only this board is tested: it is the sole ci_hs board on the rig. The lpcxpresso55s28 and +lpcxpresso11u37 run the ip3511 driver, which this change does not touch. + +- [ ] **Step 1: Preconditions** + +```bash +pgrep -f "hil_test.py [-]-retry" && echo "CI ACTIVE - wait" || echo "CI idle" +ps -eo stat,pid,etimes,comm | awk '$1 ~ /^D/' +python3 test/hil/hil_lock.py hold mimxrt1064_evk --reason "prime-verify removal A/B" +``` + +Expected: CI idle, no pre-existing D-state processes, lock acquired. If CI is active, wait for +it to drain rather than running concurrently. + +- [ ] **Step 2: Flash with verification** + +```bash +cat > /tmp/pv.jlink <<'EOF' +r +h +loadfile examples/cmake-build-mimxrt1064_evk/device/usbtest/usbtest.elf +r +g +qc +EOF +JLinkExe -device MIMXRT1064xxx6A -if SWD -speed 4000 -SelectEmuBySN 000725299165 \ + -autoconnect 1 -nogui 1 -CommandFile /tmp/pv.jlink +``` + +Expected: `Program & Verify` reports O.K. + +- [ ] **Step 3: Confirm the right image is actually running** + +```bash +sleep 5 +grep -l BAE96FB95AFA6DBB8F00005002001200 /sys/bus/usb/devices/*/serial +sudo lsusb -v -d cafe:4010 2>/dev/null | grep -A3 "Isochronous" | grep bEndpointAddress +``` + +Expected: the board is present, and the iso IN endpoint reads **0x87**. If it reads 0x83 the +flash did not take (this board has silently no-op'd a flash twice) — reflash and re-check +before running anything. + +- [ ] **Step 4: 5x full battery** + +```bash +for i in $(seq 1 5); do + timeout 700 python3 test/hil/usbtest.py --serial BAE96FB95AFA6DBB8F00005002001200 \ + --json --keep-binding --timeout 60 2>/dev/null | python3 -c " +import json,sys +d=json.load(sys.stdin) +bad=[str(c['num']) for c in d['cases'] if c['status']!='PASS'] +print(f\"run: {d['passed']}/30 speed={d['speed']}\" + (' FAILED:'+','.join(bad) if bad else '')) +" + ps -eo stat,pid,etimes,comm | awk '$1 ~ /^D/ && $4=="testusb"' +done +``` + +Expected: five lines each reading `30/30 speed=480`, and no testusb D-state line between runs. + +- [ ] **Step 5: 15x control-focused loop** + +These are the paths the removed verify actually protected — queued control, the ch9 subset, and +both ctrl_out cases. A full battery samples each only once per run. + +```bash +PASS=0 +for i in $(seq 1 15); do + timeout 300 python3 test/hil/usbtest.py --serial BAE96FB95AFA6DBB8F00005002001200 \ + --tests 9,10,14,21 --json --keep-binding --timeout 60 >/dev/null 2>&1 && PASS=$((PASS+1)) || { echo "FAILED at iteration $i"; break; } + D=$(ps -eo stat,comm | awk '$1 ~ /^D/ && $2=="testusb"' | wc -l) + [ "$D" != "0" ] && { echo "D-STATE at iteration $i"; break; } +done +echo "control loops: $PASS/15" +``` + +Expected: `control loops: 15/15`, no FAILED or D-STATE line. + +- [ ] **Step 6: Release the lock and record** + +```bash +python3 test/hil/hil_lock.py release mimxrt1064_evk +ps -eo stat,pid,etimes,comm | awk '$1 ~ /^D/' +``` + +Expected: lock released, no leftover D-state. + +**Acceptance:** 5/5 batteries at 30/30, 15/15 control loops, no `testusb` D-state outliving its +case runtime. + +**Rollback trigger:** any control-case failure (errno 110 or 71 on cases 9, 10, 14, 21) or a +lingering D-state means the verify was load-bearing after all. In that case: `git revert` the +Task 1 commit, re-run Steps 4-5 to confirm the failure disappears, and record the result — that +is a finding worth keeping, not a setback to hide. + +--- + +## Self-Review + +**Spec coverage:** the spec's change section → Task 1; "rebase first, then rebuild" → Task 2 +Steps 1-2; software gates → Task 2 Steps 3-4; hardware preconditions, verified flash and the +0x87 descriptor check → Task 3 Steps 1-3; 5x battery and 15x control loop → Task 3 Steps 4-5; +acceptance and rollback trigger → Task 3's closing block. The spec's "deliberately kept" list is +enforced negatively by Task 1 Step 1's instruction to leave the setup-lockout block untouched +and by Task 1 Step 2's grep. No gaps. + +**Placeholder scan:** no TBD/TODO/"handle edge cases"; every step carries its exact command or +code and its expected result. + +**Type consistency:** `qhd_start_xfer(uint8_t rhport, uint8_t epnum, uint8_t dir) -> bool` is +unchanged by this plan and no caller is touched, so there are no cross-task signatures to +reconcile. The only removed identifier, `prime_bit`, is local to the deleted block and Task 1 +Step 2 greps to confirm it has no remaining references. diff --git a/docs/superpowers/specs/2026-07-09-claude-agents-workflows-design.md b/docs/superpowers/specs/2026-07-09-claude-agents-workflows-design.md index 63788720c..3035723c4 100644 --- a/docs/superpowers/specs/2026-07-09-claude-agents-workflows-design.md +++ b/docs/superpowers/specs/2026-07-09-claude-agents-workflows-design.md @@ -29,10 +29,12 @@ Layered: **agents** (who does the work, with baked-in domain knowledge) × ### Worker agents — `.claude/agents/*.md` -Tiered models (owner revision 2026-07-09; originally all-opus): `port-dev` -and `driver-reviewer` on **opus** at **xhigh**; `hil-operator`, `pr-monitor` -and `static-analyzer` on **sonnet**; `builder` on **haiku** (mechanical, -log-heavy). +Tiered models (owner revision 2026-07-09; originally all-opus): `port-dev`, +`driver-reviewer` and `target-debugger` on **opus** at **xhigh**; +`hil-operator`, `pr-monitor` and `static-analyzer` on **sonnet**; `builder` +on **haiku** (mechanical, log-heavy). The registry has no effort field — +xhigh is requested per `agent()` call by whichever workflow or session spawns +the agent. | Agent | Effort | Role | |---|---|---| @@ -40,6 +42,7 @@ log-heavy). | `port-dev` | xhigh | Implement one well-scoped change in one port / file set. Follows repo rules: C99, 2-space indent, snake_case, `TU_ASSERT`, no dynamic allocation, ISR work deferred to task context. Runs `clang-format` (repo `.clang-format`) on touched files before finishing. Cross-checks the MCU datasheet in `$HOME/Documents/calibre-library` when changing dcd/hcd register logic. Verifies with a targeted build of one board using the port. Returns `{item, diffstat, buildOk, notes}`. | | `driver-reviewer` | xhigh | Review one dcd/hcd directory against dimensions: correctness, ISR safety, register use vs. datasheet AND MCU errata (calibre library; missing erratum workarounds are findings), style. Returns structured findings `{file, line, snippet, why, severity, confidence}` — coverage-first (report everything; filtering happens downstream). | | `hil-operator` | default | All rig interaction — the actions-runner service is NEVER stopped; per-board flock locks arbitrate with concurrent CI. `hil_test.py` runs rely on its per-board self-locking; manual hardware work (JLink/GDB, usbtest, serial) is wrapped in `test/hil/board_lock.py hold/release`; rig-wide ops (uhubctl, pci-rebind) require `hold --all`; on wedge `usb_recover.sh` + dmesg. Used strictly serially — never two instances concurrently. | +| `target-debugger` | xhigh | Root-cause one USB misbehavior on one board by instrumenting the device side (TU_LOG/RTT, RAM ring-buffer trace, GDB autopsy, J-Link PC-sampling) with dual-side host+target capture, per `.claude/skills/usb-target-debug/SKILL.md`, plus wire-level capture via the ataradov hardware tap (`.claude/skills/usb-sniffer/SKILL.md`) when the host side can't see or is disputed. Deliberately serial loop under one held board lock (released around `hil_test.py` runs, which self-lock); strictly one instance. Diagnosis standard: evidence shows the mechanism, or a fix flips the ORIGINAL failing case on hardware; stops after two evidence-free cycles with a partial report. Hard rule "fix stays, probe goes, re-verify clean": instrumentation reverted, candidate fix left uncommitted and re-verified on a clean build, pristine firmware reflashed before lock release. Returns `{board, bug, diagnosis, confirmed, ruledOut[], evidence[], fixDiffstat, fixVerified, instrumentationReverted, lockReleased, notes}`. | | `pr-monitor` | default | Triage one GitHub PR via `gh`: check CI status (`gh pr checks`), read failing run logs and classify each failure infra/flake vs real; re-run infra failures (`gh run rerun --failed`); harvest automated review comments (Codex/Copilot/Claude bots — knows their signals: Codex posts a "Didn't find any major issues" issue comment when clean; Copilot drops out of `requested_reviewers` when done; bot logins differ across APIs); adversarially validate each finding against the actual code. Returns structured triage `{ci: {status, infraRerun[], realFailures[]}, findings: [{source, file, line, claim, verdict, fixHint}]}`. Read/triage/re-run/reply only — never edits code. | | `static-analyzer` | low | Run PVS-Studio (SAST + MISRA C:2023/C++:2008) for one board: build with exported `compile_commands.json` (via `run_pvs.sh` solo, or a dedicated `cmake-build-pvs` dir when parallel builders run), analyze against `.PVS-Studio/.pvsconfig`, gate on diagnostics in files changed vs a base ref. Returns `{pass, ga1, ga2, changedFindings[], detail}`; `pass=false` only on GA:1 in changed files or tool failure. Read-only. | @@ -120,8 +123,8 @@ carries the judgment; JS carries the orchestration. ## Model & effort policy -- Tiered worker models: `port-dev`/`driver-reviewer` **opus** `xhigh`; - `hil-operator`/`pr-monitor` **sonnet**; `builder` **haiku**. +- Tiered worker models: `port-dev`/`driver-reviewer`/`target-debugger` **opus** + `xhigh`; `hil-operator`/`pr-monitor` **sonnet**; `builder` **haiku**. - Inline workflow stages: unit/size **haiku**; pvs **sonnet** (low effort); pr-babysit push/replies **sonnet**. - Agent frontmatter `model:` is canonical for `agentType` calls; it is read diff --git a/docs/superpowers/specs/2026-07-23-esp-target-debug-design.md b/docs/superpowers/specs/2026-07-23-esp-target-debug-design.md new file mode 100644 index 000000000..491466992 --- /dev/null +++ b/docs/superpowers/specs/2026-07-23-esp-target-debug-design.md @@ -0,0 +1,102 @@ +# esp-target-debug Skill Design + +Backend skill for debugging TinyUSB firmware on Espressif targets (rig: +`espressif_p4_function_ev`, `espressif_s3_devkitm`) via the chips' **built-in +USB-Serial-JTAG**, with external JTAG documented as a TODO until the rig has +an adapter. Companion to `target-debug`, which keeps the architecture-neutral +methodology (intrusiveness ladder, board locks, dual-side capture, diagnosis +standards) — this skill is the Espressif toolchain/probe backend, the same +boundary that makes `usb-kernel-debug` its own skill. + +## Goals + +- An agent can attach, halt, backtrace, set breakpoints/watchpoints, list + FreeRTOS threads, and capture logs on the rig's P4 **while TinyUSB device + traffic is live** — every recipe hardware-verified before landing unmarked + (the `target-debug` ethos). +- The S3's USB-SJ/OTG PHY conflict is mapped precisely, not hand-waved: + verified working via `board_test` (TinyUSB off — PHY free), verified failure + mode with a USB device example, external-JTAG escape hatch documented as + TODO. + +## Non-goals (deferred) + +- External JTAG bring-up (no adapter on the rig) — TODO section with S3 JTAG + pin notes (GPIO39-42) and openocd-esp32 adapter support pointers. +- Xtensa/S3 full parity under live USB traffic (needs external JTAG). +- ETM-class instruction trace; SystemView tooling beyond an apptrace spike. + +## Architecture + +New skill `.claude/skills/esp-target-debug/SKILL.md`; two integration edits: + +- `target-debug` gains a 2-line pointer under the probe-mapping bullets: + Espressif boards use a different toolchain, probe model, and trace story — + read `esp-target-debug`. +- `target-debugger` agent table gains an `esp-target-debug` row (name-only, + aligned columns, per the established conventions). + +Skill content (order): + +1. **Role + defer line** — methodology lives in `target-debug`; this file is + the Espressif backend. Built-in USB-SJ now; external JTAG TODO. +2. **PHY-conflict map** — + - S3: USB-SJ and OTG share one PHY (GPIO19/20). TinyUSB claiming the PHY + drops JTAG-over-USB mid-session: JTAG works for non-USB examples + (`board_test`), dies for USB device examples (verified boundary, exact + symptom recorded). External JTAG = the future escape hatch (TODO). + - P4: OTG-HS has a dedicated HS PHY; USB-SJ is separate — JTAG and the + TinyUSB DUT port coexist (verified). USB-SJ doubles as a live log + console during device traffic — the TU_LOG-equivalent channel. +3. **Toolchain & attach** — `. $HOME/code/esp-idf/export.sh` provides + `openocd-esp32` + `riscv32-esp-elf-gdb` (P4) / `xtensa-esp32s3-elf-gdb` + (S3). Rig path is raw openocd (HIL firmware isn't an idf project on disk): + `openocd -f board/esp32p4-builtin.cfg` with `adapter serial <uid>` (USB-SJ + is VID 303A:1001; uid = the `flasher.uid` already in `tinyusb.json`), gdb + on :3333. `idf.py openocd` / `idf.py gdb` noted for idf-project work. +4. **Technique mapping table** (aligned) — ARM technique → Espressif + equivalent: + + | target-debug technique | Espressif backend | + |---|---| + | GDB autopsy, bp/wp | same flow; RISC-V trigger module (P4) / Xtensa 2 bp + 2 wp (S3); budget read verified on P4 | + | Vector catch | none — breakpoint the panic handler; decode `mcause`/`mepc`/`mtval` (P4) | + | SWO / DWT data trace | none — apptrace over JTAG is the analog (gated spike; lands `(untested)` if it fails) | + | RTT / TU_LOG | USB-SJ console — on P4 it coexists with DUT traffic | + | FreeRTOS threads | native in openocd-esp32 — `info threads` out of the box | + | verifybin | `esptool.py verify_flash` | + +5. **Rig discipline deltas** — same `board_lock.py` protocol; flasher is + esptool (serial-port-by-uid); reflash pristine before release; one client + per USB-SJ device. +6. **External JTAG — TODO** — S3 JTAG pins, adapter classes openocd-esp32 + supports, and the efuse caveat (JTAG pin selection), unverified. + +## Verification gates (execution order) + +All under board locks, serial, evidence in commit messages: + +1. **P4 coexistence (headline)**: flash a device example, confirm enumeration + + traffic on the DUT port, then attach openocd+gdb over USB-SJ → + halt, `bt`, resume — device stays functional after resume. +2. **P4 budget**: read trigger/watchpoint counts via openocd/gdb; set a + hardware watchpoint on a TinyUSB variable, confirm hit. +3. **P4 threads**: `info threads` lists ESP-IDF tasks (usbd task visible). +4. **P4 console**: capture USB-SJ console log output during device traffic. +5. **P4 apptrace spike (gated)**: bounded attempt; verified recipe or + `(untested)` tag. +6. **S3 boundary**: `board_test` flashed → attach works (halt+bt); then a USB + device example → record the exact JTAG failure symptom when the PHY + switches. No further S3 work (external JTAG TODO). + +## Constraints + +- Worktree `claude/improve-debug-skill-agent`; commit per gate; pre-commit + before each; no Co-Authored-By trailers. +- Formatting conventions already established: aligned table columns, + skill-name-only cross references, bullets over run-on paragraphs. +- Espressif builds need `export.sh` first (CLAUDE.md); P4/S3 examples build + via idf.py — reuse existing HIL-built firmware where possible instead of + rebuilding. +- Hardware-verify-before-landing: unverified content ships tagged + `(untested)` or not at all. diff --git a/docs/superpowers/specs/2026-07-28-hil-test-refactor-design.md b/docs/superpowers/specs/2026-07-28-hil-test-refactor-design.md new file mode 100644 index 000000000..3cd202d95 --- /dev/null +++ b/docs/superpowers/specs/2026-07-28-hil-test-refactor-design.md @@ -0,0 +1,141 @@ +# hil_test.py refactor: test core + infra helpers + +**Date:** 2026-07-28 +**Branch:** `claude/hil-test-split` (based on `claude/hil-pool-check`, which adds `pool_check.py`) + +## Motivation + +`test/hil/hil_test.py` is 2370 lines mixing five concerns: board-lock protocol, per-controller +scheduling permits, flash/reset backends, the actual per-example tests, and orchestration/report/CLI. +The lock protocol additionally exists in three copies (`hil_test.py`, `board_lock.py`, +`.claude/skills/hil/pool_check.py`), which has already produced drift (pool_check's copy lacks +hil_test's fail-open and error guards). Splitting the infrastructure out makes `hil_test.py` +test-focused and gives external tools (pool_check) one canonical import for locks, permits, and +flashing. + +## Goal / non-goals + +**Goal:** behavior-preserving code motion. `hil_test.py`'s CLI, arguments, output, report format, +and runtime behavior stay byte-identical. One deliberate user-visible change: the operator lock CLI +moves from `board_lock.py` to `hil_lock.py` (same subcommands, same behavior); `board_lock.py` is +deleted. + +**Non-goals (explicit follow-ups, not this change):** +- The 15 pool_check findings from the 2026-07-28 code review (exception isolation, park-on-failure, + espressif coverage, probe-recovery criterion, etc.). +- pool_check adopting `flash_permit` controller budgeting (enabled by this split). +- Any change to lock semantics, permit widths, flash behavior, or test logic. + +## Resulting layout (`test/hil/`) + +| File | ~Lines | Role | +|---|---|---| +| `hil_test.py` | 1600 | tests + orchestration + report + CLI (unchanged interface) | +| `hil_lock.py` (new) | 420 | board-lock protocol + controller permits + operator CLI | +| `hil_flash.py` (new) | 250 | `run_cmd` + flash/reset backends + `find_firmware` | +| `board_lock.py` | deleted | superseded by `hil_lock.py` | + +Import graph: `hil_test` → {`hil_lock`, `hil_flash`}; the helpers import nothing local (no cycles). +`pool_check.py` imports all three. + +## hil_lock.py + +Docstring states the scope: board locks + controller flash/battery permits; the CLI manages board +locks only (permits are in-process semaphores with no CLI meaning). + +**Flock core** (protocol defined once; moved from `board_lock.py`/`hil_test.py`): +- `BOARD_LOCK_DIR = '/tmp/tinyusb-hil-locks'`, `lock_path(board)` +- `CI_REASON = 'hil_test.py'` — the release-protected holder tag (release refuses to kill it) +- `flock_nb(board) -> fh` — `os.open(O_RDWR|O_CREAT, 0o666)` **without O_TRUNC** (a losing racer + must not wipe the winner's record), `fdopen('r+')`, `LOCK_EX|LOCK_NB`; raises `OSError` when held +- `write_record(fh, reason)` — truncate+seek+`json.dump({pid, reason, since})`+flush +- `clear_record(fh)` — truncate(0), swallow OSError (records stay truthful on release) +- `read_record(board) -> dict | None` — today's `board_lock.read_info` +- `acquire_board_lock(board, reason=CI_REASON) -> fh | None` — today's `hil_test.acquire_board_lock` + with a `reason` parameter: `HIL_NO_BOARD_LOCK=1` bypass, fail-open with warning on lock-dir + OSError, `RuntimeError` carrying holder info on conflict + +**Controller permits** (moved verbatim from `hil_test.py`): +- `FLASH_PARALLEL`, `USBTEST_PARALLEL`, `CONTROLLER_SLOTS` (env-overridable as today) +- `controller_of(uid)`, `controller_slot(pci)`, `controller_permit`, `flash_permit(uid)`, + `usbtest_permit(uid)` +- Per-worker globals (`usbtest_sems`, `flash_sems`, `controller_map`, `controller_meta`, + `controller_hints`) set by a new `init_scheduling(sems, fsems, cmap, cmeta, hints)` hook that + `hil_test.init_worker` calls from the Pool initializer. `controller_permit`'s PROFILE logging + calls back through a module-level `log = print`-style hook that `hil_test` points at `log_line` + during `init_scheduling` (keeps helpers free of hil_test imports). The `PROFILE` env flag + (`HIL_PROFILE=1`) is read independently in `hil_lock` at import, same derivation as today. + +**Operator CLI** (moved verbatim from `board_lock.py`): `hold`/`release`/`status` subcommands with +the daemon-holder machinery (double-fork, setsid, stdio detach, success pipe, SIGTERM bow-out), +release policy (probe the flock; protect `CI_REASON` holders; SIGTERM other recorded pids), +`is_locked` pid-liveness, `--all`/`--config` roster handling. The hold/release/status internals +switch to the flock-core helpers above; observable behavior unchanged. + +## hil_flash.py + +Moved verbatim from `hil_test.py`: +- `CMD_TIMEOUT` (env-overridable), `run_cmd(cmd, cwd, timeout)`, `cmd_stdout_text(out)` +- `OPENCOD_ADI_PATH`, `TINYUSB_ROOT` +- All backends: `flash_jlink`/`reset_jlink`, `flash_stlink`/`reset_stlink`, + `flash_stflash`/`reset_stflash`, `flash_openocd`/`reset_openocd`, + `flash_openocd_wch`/`reset_openocd_wch`, `flash_openocd_adi`/`reset_openocd_adi`, + `flash_wlink_rs`/`reset_wlink_rs`, `flash_esptool`/`reset_esptool`, + `flash_uniflash`/`reset_uniflash`, `flash_lm4flash`/`reset_lm4flash` +- `find_firmware(variant, example)` +- `get_serial_dev(id, vendor_str, product_str, ifnum)` — moves here (not hil_test) because + `flash_esptool` calls it; keeping it test-side would create a helper→hil_test import cycle. + Tests call `hil_flash.get_serial_dev`. +- Module globals `build_dir = 'cmake-build'` and `verbose = False`, set by callers exactly as the + `hil_test` globals are today (`hil_test.main` sets them from argparse; pool_check sets them + directly). `run_cmd`'s verbose echo reads `hil_flash.verbose`. + +Dispatch in callers stays string-based: `getattr(hil_flash, f'flash_{flasher["name"].lower()}')`. + +## hil_test.py (what remains) + +Config TypedDicts (`Board`, `FlasherCfg`, …), device-node lookup except `get_serial_dev` +(`get_disk_dev`, `get_hid_dev`, `get_alsa_capture_dev`, `open_serial_dev`, `serial_write_all`, +`read_disk_file`, `open_mtp_dev`, `get_printer_dev`/`open_printer_dev`), enum-timeout globals + +`wait_until`, +`log_line`/print-lock, `compact_output`, all `test_*` functions, test lists, `test_example`, +`build_board`, `test_board`, report rendering/accumulation, `main`. Call sites use explicit +module-qualified names (`hil_lock.flash_permit(...)`, `hil_flash.run_cmd(...)`) so provenance is +greppable; no `from … import *`-style mirroring. + +`init_worker` keeps its signature (Pool initargs unchanged) and forwards the scheduling state to +`hil_lock.init_scheduling(...)`. + +## Consumer updates (same commit) + +- **`.claude/skills/hil/pool_check.py`** — drop its private `lock_board`/`unlock_board` in favor of + `hil_lock.flock_nb` + `write_record(fh, 'pool_check')` (+ `clear_record` on release; deliberately NOT `acquire_board_lock`, whose HIL_NO_BOARD_LOCK bypass and fail-open behavior pool_check must not inherit); import + flashers/`find_firmware`/`get_serial_dev`/`cmd_stdout_text`/`TINYUSB_ROOT`/`build_dir` from + `hil_flash`; `BOARD_LOCK_DIR` references move to `hil_lock`. pool_check then imports **only** + `hil_lock` + `hil_flash` (no `hil_test`), so its `pymtp` stub shim is deleted — that shim existed + solely because importing `hil_test` pulls in libmtp. +- **`test/hil/hil_ci.sh`** — the scp list is currently `hil_test.py`, `pymtp.py`, `$CONFIG`; add + `hil_lock.py` and `hil_flash.py` (hil_test cannot even import without them). `board_lock.py` was + never in the list. +- **Docs rename `board_lock.py` → `hil_lock.py`** (live docs only): `.claude/skills/hil/SKILL.md`, + `.claude/agents/hil-operator.md`, `.claude/agents/target-debugger.md`, + `.claude/skills/etm-trace/SKILL.md`, `.claude/skills/usb-kernel-recover/SKILL.md`, + `.claude/skills/target-debug/SKILL.md`. Historical `docs/superpowers/{plans,specs}` stay as + records. +- **CI workflow** — untouched (invokes `hil_test.py` CLI only). + +## Verification + +1. `python3 -m py_compile` on all three modules + pool_check. +2. `hil_lock.py hold/status/release` interplay: hold, conflicting hold, status listing, release, + protection of a `CI_REASON` record, stale-record cleanup. +3. `pool_check.py --scan-only`, then a single flash board (e.g. `-b stm32f407disco`). +4. Full `hil_test.py -b stm32f407disco -B examples tinyusb.json` on the rig; compare the report + row and log shape against a pre-refactor run. +5. `pre-commit run` on all touched files. + +## Sequencing + +Lands on top of `claude/hil-pool-check`. After merge, fix the pool_check review findings as a +separate change on the new module boundaries, and update agent-memory references to +`board_lock.py`. diff --git a/docs/superpowers/specs/2026-07-29-hil-pr-scoped-selection-design.md b/docs/superpowers/specs/2026-07-29-hil-pr-scoped-selection-design.md new file mode 100644 index 000000000..898b3c8ab --- /dev/null +++ b/docs/superpowers/specs/2026-07-29-hil-pr-scoped-selection-design.md @@ -0,0 +1,179 @@ +# PR-scoped HIL selection: helper/hil_select.py + +**Date:** 2026-07-29 +**Branch:** `claude/hil-select` (based on `claude/hil-pool-check`, which carries the +hil_lock/hil_flash split and the current rig rosters) + +## Motivation + +Every PR currently builds and runs the full HIL matrix (both rigs, every roster board, every +test). Most PRs touch one port or one class: a `dcd_rp2040` change cannot affect an STM32 board, +a `cdc_device.c` change cannot affect an MSC-only example, and a device-stack change cannot +affect host tests. Scoping HIL to the affected boards/tests cuts CI wall time and rig wear +without losing relevant coverage. + +## Goal / non-goals + +**Goal:** a shared selector that maps a PR diff to (boards, per-board test lists), wired into +CI's `set-matrix` on `pull_request` events (pruning both `hil-build` and the rig jobs) and +callable locally (pre-pr, manual runs). Scoping may only shrink coverage when the mapping is +confident; every uncertainty widens to the full matrix. + +**Non-goals:** +- Variant-level selection (all variants of a selected board run). +- Scoping the non-HIL build jobs (cmake/CircleCI one-per-family builds are independent build + coverage and stay untouched). +- Scoping push/master/scheduled runs (always full). +- Changing hil_test.py behavior (the selector only *composes* existing `-b`/`-bt` args). + +## Component: `test/hil/helper/hil_select.py` + +Stdlib-only, importable and CLI. Lives beside the harness so `hil_ci.sh` copies are unaffected +(it runs on the GitHub runner / dev PC, not on the rig). It must NOT import `hil_test.py` +(which drags pyserial/pymtp onto the bare GitHub runner): the three test lists +(`device_tests`, `dual_tests`, `host_test`) move verbatim into the stdlib-only +`test/hil/helper/hil_util.py` that both `hil_test.py` and `hil_select.py` import (behavior +preserving; `hil_ci.sh` copies the whole `helper/` directory). + +``` +python3 test/hil/helper/hil_select.py --base <ref> [--diff-file <path>] CONFIG.json [CONFIG.json...] +``` + +- `--base REF`: changed files = `git diff --name-only $(git merge-base HEAD REF)..HEAD` + (mirrors pre-pr). `--diff-file`: newline-separated file list instead of git (unit tests, CI + reuse of a precomputed diff). +- Output (stdout, JSON): + +```json +{ + "full": false, + "boards": {"raspberry_pi_pico": "all", "stm32f407disco": ["device/cdc_msc", "device/cdc_dual_ports"]}, + "args": {"tinyusb.json": "-b raspberry_pi_pico -b stm32f407disco -bt stm32f407disco:device/cdc_msc,device/cdc_dual_ports", + "hfp.json": ""}, + "reasons": ["src/portable/raspberrypi/rp2040/dcd_rp2040.c: port rp2040 -> family rp2040 -> boards [raspberry_pi_pico, ...] (device role)"] +} +``` + +- `full: true` ⇒ `boards`/`args` cover the entire rosters (identical to today's behavior). +- `args` maps each input config file to the hil_test.py argument string for that rig: `-b` per + selected board on that roster, plus `-bt BOARD:t1,t2` for boards with a restricted test list + ("all" boards get bare `-b`). An empty string means: nothing on this rig is affected — the + rig job is skipped for this PR. +- Per-file reasoning lines (`file → rule → contribution`) go in `reasons` and to stderr, so the + CI log answers "why did/didn't HIL run X" without archaeology. + +## Classification rules + +Each changed file yields a contribution; the selection is the union. Any file matching no rule +sets `full: true` (fail-open). Rules, first match wins: + +1. **Non-code:** `docs/**`, `.claude/**` (except the workflows below via rule 8), `*.md`, + `*.rst`, `LICENSE*` → contributes nothing. +2. **Port:** `src/portable/<vendor>/<ip>/**` (or single-level `src/portable/<name>/**`). + Role from basename: `dcd_*`/`*_device*` → device; `hcd_*`/`*_host*` → host; anything else + (shared port files, e.g. `dwc2/dwc2_common.c`) → both. Families = directories of + `hw/bsp/*/family.cmake|family.mk` whose text references `<vendor>/<ip>` (pre-pr's grep), + boards = those families' entries on the input rosters. Tests = all tests of that role + (device_tests / host_test from hil_test.py's lists; dual_tests count as both roles). +3. **Class:** `src/class/<c>/*_device.*` → all device-capable roster boards; tests = the + device/dual examples in hil_test.py's lists whose `examples/<role>/<ex>/src/tusb_config.h` + defines `CFG_TUD_<C>` with a nonzero value (derived at runtime; `<C>` = upper-cased class + dir, with the map `musb→n/a`-style exceptions NOT needed — class dirs and config macros + share names: cdc, msc, hid, midi, audio, video, vendor, usbtmc, mtp, printer. Two + exceptions: in class dir `dfu`, `dfu_rt_device.*` maps to CFG_TUD_DFU_RUNTIME and + `dfu_device.*` to CFG_TUD_DFU; class dir `net` maps to CFG_TUD_ECM_RNDIS|CFG_TUD_NCM.) `*_host.*` analogously via `CFG_TUH_<C>`. Shared class files (e.g. `cdc.h`) → + both roles' matching examples. A class with zero matching examples contributes nothing + (known path, does not force full). +4. **Core role:** `src/device/**` → all device-capable boards, all device tests (+dual); + `src/host/**` → all host-capable boards, all host tests (+dual). +5. **Core common:** `src/common/**`, `src/osal/**`, `src/tusb.c`, `src/tusb.h`, + `src/tusb_option.h` → full. +6. **BSP:** `hw/bsp/<family>/**` → that family's roster boards, all their tests; + `hw/bsp/<family>/boards/<board>/**` narrows to that board if it is on a roster, and + contributes nothing when it is not (an off-rig board cannot be HIL-tested; known path, + does not force full). + Family-agnostic BSP files (`hw/bsp/board_api.h`, `hw/bsp/board.c`, ansi_escape.h) → full. +7. **Example:** `examples/<role>/<ex>/**` → all roster boards, tests = that example if present + in hil_test.py's lists, else contributes nothing. `examples/build_system/**`, top-level + `examples/CMakeLists.txt` → full. `examples/device/board_test/**` → full: it is the park + firmware hil_test.py flashes on every board (variant boundary + teardown), not a test. +8. **Harness/infra:** `test/hil/**`, `.github/workflows/build*.yml`, + `.github/actions/**`, `tools/build.py`, `tools/get_deps.py`, `tools/cmake/**`, + `hw/mcu/**`, `lib/**` → full. +9. **Everything else** (`test/unit-test/**`, `tools/**` not above, unknown paths) → full. + (Unit-test-only changes could safely skip HIL, but per the fail-open stance anything not + explicitly classified widens; narrowing rule 9 is a later refinement.) + +**Role pruning:** after the union, if only device-role contributions exist, host-only boards +drop out and host tests are stripped from mixed boards (vice versa for host-only changes). +Dual tests survive either role. Board capability (device/host) comes from the roster entry's +`tests` flags/only-list, same logic hil_test.py uses. + +**No-rig-coverage case:** a cleanly classified change whose boards intersect a roster to the +empty set yields an empty `args` string for that rig and a stderr line saying so — the rig job +is skipped, not widened (running unrelated boards would test nothing relevant). + +**Roster source:** `config['boards']` only (boards-skip stays parked). + +## CI wiring (`.github/workflows/build.yml`) + +- `set-matrix` (PR events only): after generating today's matrices, run + `helper/hil_select.py --base origin/${{ github.base_ref }} test/hil/tinyusb.json test/hil/hfp.json` + (checkout with enough history to reach the merge base: `fetch-depth: 0` on this one job, or + an explicit `git fetch origin $BASE_REF`). New job outputs: `hil_select_full`, + `hil_args_tinyusb`, `hil_args_hfp`, plus the selected-board list consumed by the matrix + generator. Non-PR events: skip the selector, outputs default to full/empty-args-means-all. +- `hil_ci_set_matrix.py` gains `--select '<json>'`: when given and `full` is false, it emits + build entries only for selected boards (per config). Untouched otherwise. +- `hil-tinyusb` job (one matrixed job covering both rigs, selected by `matrix.hil_json`): a + step picks the rig's selector args in shell (`case "$HIL_JSON" in ...`) from the set-matrix + outputs and either appends them to the `hil_test.py` invocation or exits the step early with + a "HIL skipped by selection" log line when that rig has nothing to run (`run` flag output + false). The separate `hil-tinyusb-esp` job (esptool split) gets the same treatment with the + tinyusb args. Non-PR events: outputs default to run=true with empty args (today's behavior). +- The `--flasher`/`--exclude-flasher` split in the existing matrix `test_args` composes fine + with `-b` (hil_test.py applies both filters). + +## Local use + +- pre-pr's "Map changes to boards" step delegates to + `python3 test/hil/helper/hil_select.py --base $BASE test/hil/tinyusb.json` and derives its + one-board-per-family sample from the selector's board set (its capping/sampling policy is + unchanged — the selector provides the affected set, pre-pr samples it). +- Manual: `python3 test/hil/hil_test.py -B examples $(python3 test/hil/helper/hil_select.py --base master test/hil/tinyusb.json | jq -r '.args["tinyusb.json"]') test/hil/tinyusb.json` + — documented in the hil skill. + +## Testing + +`test/hil/test/test_hil_select.py` — stdlib `unittest`, no hardware, injected diffs via +`--diff-file`/API. Cases (the acceptance examples): +1. `src/portable/raspberrypi/rp2040/dcd_rp2040.c` → only rp2040-family roster boards, device + tests only, host-only boards absent, `full` false. +2. `src/device/usbd.c` → every device-capable board on both rosters, all device tests + dual, + no host-only board, no host tests. +3. `src/class/cdc/cdc_device.c` → only examples with CFG_TUD_CDC enabled (must include + device/cdc_msc and device/cdc_dual_ports; must exclude device/msc_dual_lun and all + host tests). +4. `src/class/msc/msc_host.c` → host-capable boards only, host examples with CFG_TUH_MSC. +5. `tools/random_new_script.py` → `full: true`. +6. `docs/foo.rst` alone → contributes nothing ⇒ empty selection, `full` false, all `args` + empty (CI additionally has check-paths gating; the selector's answer is still honest). +7. `hw/bsp/rp2040/family.cmake` → rp2040-family boards, all their tests. +8. Mixed device+host diff → no pruning (both roles present). +The suite runs in `set-matrix` before the selector is used, and locally via +`python3 test/hil/test/test_hil_select.py`. + +## Safety properties + +- Fail-open: unknown/infra paths ⇒ full matrix; selector crash in CI ⇒ job fails visibly + (never silently skips HIL). +- Only `pull_request` events are scoped. +- The selection JSON + per-file reasons are printed in the job log for audit. +- hil_test.py errors on `-b` names not in the config — the selector only emits roster names, + and the unit suite locks that invariant. + +## Sequencing + +Lands on `claude/hil-select` on top of the pool-check/split stack. Follow-ups it does not +include: narrowing rule 9 for unit-test-only changes; variant-level selection; pre-pr skill +text update ships in the same change (its mapping section shrinks to a selector call). diff --git a/docs/superpowers/specs/2026-07-30-hil-usbtest-fleet-wedge-design.md b/docs/superpowers/specs/2026-07-30-hil-usbtest-fleet-wedge-design.md new file mode 100644 index 000000000..3ed0c1519 --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-hil-usbtest-fleet-wedge-design.md @@ -0,0 +1,236 @@ +# HIL fleet-wedge containment + +Date: 2026-07-30 +Status: implemented, then superseded in part — addendum last checked 2026-08-12 +against the shipped code; where they disagree the CODE and the usb-kernel-recover +skill win, never this document. + +- **Pool guard.** A single constant, not the flat 4200s below and not a derivation: + `POOL_TIMEOUT = pos_int_env('HIL_POOL_TIMEOUT', 3600)`. A per-controller model briefly + lived here and was removed -- it under-modelled the flash phase and could INVERT + (adding a usbtest board lowered the guard, because the derived value fell below the + baseline it was meant to raise). The guard's only job is to stop a wedged pool short + of the job ceiling so the report still gets written; predicting a healthy run's + duration is a different problem. `pos_int_env` warns only on a non-integer or a value + <= 0: there is NO upper clamp and no warning above any threshold, so a pin larger than + a job ceiling silently restores the inversion this work removed. +- **Job ceilings.** 90/90/120 min (build.yml), not 60/60/90 and not the 85/115 below. + They must clear the 3600s guard plus the pre-pool checkout/artifact merge and the + post-guard sweep and report upload. No job pins `HIL_POOL_TIMEOUT`. +- **Battery budgets.** `USBTEST_BATTERY_BUDGET` 260s, `USBTEST_RECOVERY_BUDGET` 250s. + The 200s-with-a-197s-floor derivation recorded here was never shipped; the floor + assertion was removed with it. +- **HUNG recovery.** Reflash of the DUT through its roster flasher + (`usbtest.py --recover-board/--recover-fw`), not the root-cycle-first recovery in + section 1d — replaced after the 2026-08-11 ppps measurement (uhubctl never cuts + VBUS; root-cycle is probe-only). Since 2026-08-12 the reflash is SKIPPED + when `hil_flash.convoy_safe(board['flasher'])` is false (usbtest.py:675): the flasher + would enumerate by opening usbfs nodes, block on the same convoy, and become a second + stray rather than clear the first. A holder that owns the device lock inside a driver + ioctl is terminal either way -- a reflash only produces a disconnect, and + `usb_disconnect()` needs that same lock -- and that state needs a reboot. + +Step 0 done — the host was rebooted 2026-07-30 14:11 and the rig +came back clean. The device that triggered this incident was removed from the rig, so +only the containment work remains relevant. +Rig: `ci.lan` (Proxmox guest on `pve.lan`) + +## Problem + +On 2026-07-29/30 every board in the `ci.lan` usbtest fleet failed, `openocd` processes +landed in uninterruptible sleep, and no subsequent HIL run could start. Two GitHub +Actions runs were stranded: `30484641269` sat `in_progress` for over eight hours +(past GitHub's own 360-minute default), and `30485082274` sat `queued` behind it from +2026-07-29 19:35 UTC onward. Both report directories were written empty. + +A reboot of the `ci` guest at 10:48 did not clear the condition: the same kernel state +re-formed at 10:52:23. + +## Root cause + +Five layers, each independently observable. + +### 1. A permanently wedged hub worker holds a root-hub device lock + +A device that repeatedly re-asserts connect while failing to enumerate keeps +`hub_event()` busy, and `hub_event()` holds `usb_lock_device(hdev)` on its hub for its +whole run (hub.c:5896/5989). The `usb_hub_wq` worker sits in `hub_port_reset`, so that +hub's `device_lock` is effectively never released: + +``` +kworker/14:6+usb_hub_wq (state D, 400+ s) + msleep+0x2b + hub_port_reset+0x1a4 [usbcore] + hub_event+0x727 [usbcore] +``` + +`usb usbN-portM: Cannot enable. Maybe the USB cable is bad?` is logged every four seconds +for as long as it lasts. + +Verified against hub.c v6.12.96 rather than inferred: the kernel does **not** retry +without bound, and root and downstream ports are bounded identically — +`hub_port_reset()` tries `PORT_RESET_TRIES` then logs that message (hub.c:3149), +`hub_port_connect()` wraps it in `PORT_INIT_TRIES` = 4 and disables the port on give-up +(hub.c:5455/5619). A count in the thousands is therefore that many separate connect +events, not one runaway loop, and it indicts the device rather than the port. + +### 2. A parked board storms the second controller + +`ra6m5_ek` (`test/hil/tinyusb.json`, uid `8419032D32363657364EF4622D294B4E`, at +`13-3.3`) runs dfu firmware (`cafe:400b`) and re-enumerates every 1-2 seconds +continuously, wrapping the entire bus-13 devnum space (`...120 -> 127 -> 4 -> 6 -> 10`). +This is standing `hub_event` and Address-Device pressure on controller `03:00.0`, +concurrent with parallel usbtest batteries on the same silicon. + +The board is already listed in `boards-skip`, which is precisely why it storms: +`boards-skip` stops testing a board but never parks it, so it keeps running whatever +firmware it last received. Park-flash only runs as teardown of a board that actually +executed tests. + +### 3. The kernel `usbtest` control-queue case waits without a timeout + +`test_ctrl_queue` blocks on an untimed `wait_for_completion()` while `usbdev_ioctl` +holds the DUT's `device_lock`: + +``` +wait_for_completion+0x8a <- no _timeout variant +test_ctrl_queue+0x4ab [usbtest] +usbtest_do_ioctl+0x501 [usbtest] +usbdev_ioctl+0x6b8 [usbcore] +``` + +`--timeout 60` in `test/hil/usbtest.py` is a subprocess timeout only. `SIGKILL` is not +delivered to a task in uninterruptible sleep. `usbtest.py` already recognises this and +reports `HUNG`, then calls `usb_recover.sh root-cycle`. + +### 4. openocd inherits the convoy and the whole fleet dies + +Once a device lock is stuck, `port_event()` takes a child device's lock to warm-reset +it and blocks while still holding its hub's lock. Any later +`open("/dev/bus/usb/BBB/DDD")` against such a device blocks uninterruptibly: + +``` +usbdev_open+0xdc [usbcore] -> __mutex_lock +chrdev_open -> do_sys_openat2 -> __x64_sys_openat +``` + +That is the state of the three `openocd` processes at 04:16:51 (pids 207921, 207987, +208034) — the flasher, unkillable. Because one controller carries two buses, a single +convoy takes out every board on both, which is why the failure presents as the entire +fleet. + +The existing `HUNG` recovery cannot help here. A root-port VBUS cycle frees a +*device-lock* holder; it cannot free a lock held by a stuck *hub worker*, and on this +rig the cycle lands on the controller that is already wedged. + +### 5. Nothing bounds the damage, so one bad run becomes a CI outage + +- `hil-tinyusb` and `hil-tinyusb-esp` in `.github/workflows/build.yml` carry no + `timeout-minutes`. Only `hil-hfp-iar` does. +- `ci.lan` runs a single runner service, so there is one job slot. +- `test/hil/hil_test.py` bounds the pool with `POOL_TIMEOUT` (4200 s), and that guard + fires correctly — but the recovery path does not survive a D-state worker: + +```python +with Pool(processes=os.cpu_count() or 1, initializer=init_worker, initargs=initargs) as pool: + async_ret = pool.map_async(test_board, config_boards) + try: + mret = async_ret.get(timeout=POOL_TIMEOUT) + except MpTimeoutError: + pool.terminate() + pool.join() # blocks forever: a D-state worker never reaps + raise RuntimeError(f'HIL worker pool timed out after {POOL_TIMEOUT}s') +``` + +`multiprocessing` joins workers unbounded, so both `pool.terminate()` and +`pool.join()` hang, as does the `with Pool(...)` exit on the success path. Normal +`hil-tinyusb (tinyusb.json)` runs take 10-20 minutes; one recent run took 71.3 +minutes, which is the 70-minute guard firing and succeeding. The eight-hour run is the +pathological case. + +## Design + +### Step 0 — recovery (manual prerequisite) + +Power-cycle the PVE **host**, not the `ci` guest. A guest reboot is not sufficient; +hubs latch up across the PCIe reset, which the 10:48 reboot demonstrated. Nothing +below can be verified until the rig is clean. + +### Section 1 — CI containment + +**1a. Two layered timers.** An inner guard inside `hil_test.py` (`POOL_TIMEOUT`, 70 min) +that fails gracefully -- it writes a report naming the timeout and the dispatched boards, +shuts the pool down and exits -- and an outer `timeout-minutes` per rig job (85 for the +hil-tinyusb jobs; 115 for hil-hfp-iar, which also builds four boards with IAR in the same +job) as the backstop for when even exiting cannot free the runner. The ceiling must stay +ABOVE the inner guard, or GitHub kills the job before the report is written. + +> **Corrected after measurement.** An earlier revision cut the guard to 30 min on the +> reading that real runs take 9-17 min and everything longer was the old guard firing. +> That was wrong. `hil_lock.py` records 22.2/14.3/12.5/10.8 min at usbtest width 1/2/3/4, +> and raising the per-battery budget to 380s made hung boards cost more again. The 30 min +> guard then fired on 5 of the last 8 HIL job executions across both rigs, and because +> `map_async` is all-or-nothing each of those runs published a banner instead of any +> per-board result. Restored to 4200s, the value whose original rationale -- usbtest +> batteries are serialized fleet-wide, lengthening the tail -- was correct. + +**1b. Bound the pool shutdown.** Add a helper to `test/hil/hil_test.py`: + +```python +def _shutdown_pool(pool, grace=30): + """terminate() a Pool without ever blocking forever: multiprocessing joins its + workers unbounded, and a worker in uninterruptible sleep (wedged usbfs) never + reaps -- which would hold the runner's only job slot indefinitely.""" + t = threading.Thread(target=pool.terminate, daemon=True) + t.start() + t.join(grace) + return not t.is_alive() +``` + +On the `MpTimeoutError` path: write the report first, recording the boards that never +reported so the run stops producing an empty report directory; then `_shutdown_pool`; +then `os._exit(1)` if it did not return. The hard exit is the point — it is the only +way past a kernel-side unkillable child. Use the same helper for the `with Pool(...)` +exit path. + +**1c. Pre-flight rig health check.** `check_rig_health()` runs before the build and +**never aborts**. It probes `/proc` unprivileged (dmesg is restricted on the rig) for a +wedged `usb_hub_wq` worker, and reports a `/proc` too restricted to trust as its own +distinct cause rather than as a diagnosed fault. + +It is deliberately non-fatal: the rig is unattended and every remedy for a real wedge is +manual, so aborting would not fix anything -- it would discard the per-board results the +run can still collect and leave CI red until a human noticed. It emits a GitHub +`::error::` annotation and continues. The automatic containment is 1a and 1b, which bound +a stuck run and explain it without anyone touching the rig. + +**1d. Order the recovery correctly.** In `test/hil/usbtest.py`, attempt +`usb_recover.sh root-cycle` FIRST on a `HUNG` case, and only check for a wedged hub worker +*afterwards*. + +> **Corrected during implementation.** This section originally said to check for a wedged +> worker *before* the cycle and skip it on a hit. That is backwards. Our own stuck +> `testusb` holds the DUT's device lock, so any port event drives a hub worker into +> `usb_lock_device()` on it -- uninterruptible, so it reads `D` in ~100% of samples and the +> confirmation window makes the wrong verdict *more* confident, not less. Cutting VBUS is +> precisely what completes the in-flight URB, returns the ioctl and frees that worker, so +> gating on that signature would suppress the recovery in the exact ordering it exists for. +> A worker still wedged after the cycle is the genuinely unrecoverable case, and that is +> what the code now reports. + +## Verification + +- Unit-test `shutdown_pool` and the `hil_health` detectors against a synthetic `/proc`. + A real wedge cannot be manufactured on demand, so they are tested against fabricated + inputs rather than live hardware. +- Confirm the detectors flag a genuinely wedged rig, and return clean on a healthy one. +- One clean full-fleet `hil_test.py` run to prove `check_rig_health` does not + false-abort. + +## Out of scope + +- **`ra6m5_ek` park and its dfu reset loop.** Dropped by decision. Consequence: the + layer-2 devnum storm remains as standing pressure on controller `03:00.0`. Unplugging + the board or flashing `board_test` by hand resolves it without any code change. +- **An unattended PVE watchdog** that detects the wedge and power-cycles the host. + Declined: more moving parts, and it can cut a running CI job. diff --git a/docs/superpowers/specs/2026-08-15-ci-hs-reset-edges-design.md b/docs/superpowers/specs/2026-08-15-ci-hs-reset-edges-design.md new file mode 100644 index 000000000..e01831d34 --- /dev/null +++ b/docs/superpowers/specs/2026-08-15-ci-hs-reset-edges-design.md @@ -0,0 +1,162 @@ +# Bus-reset edge events + review fix wave — design + +Date: 2026-08-15 +Branch: `fix-ci-hs` (unpushed, 6 commits over master `53fef2833`) + +## Problem + +A max-effort review of the branch produced 15 findings. Four are regressions the branch +itself introduced; the rest are pre-existing or cross-cutting. The load-bearing one: + +`dcd_ci_hs.c` now runs the RM-prescribed reset cleanup at the URI (reset-start) interrupt +but does not tell usbd until the Port Change Detect that ends the reset. For the whole +reset window — a minimum of 3 ms, typically 10–50 ms — usbd still believes the device is +configured while the DCD's queue heads have been zeroed. A class driver writing in that +window (`tud_hid_n_report()`, `tud_cdc_write_flush()`) primes a disabled endpoint over a +zeroed dQH, *after* the cleanup's flush, so the stale prime survives re-enumeration over a +buffer usbd has already released. On a 600 MHz M7 that window is enormous. Master had no +gap: cleanup and event were adjacent statements. + +The stack has no way to express "reset started" — `DCD_EVENT_BUS_RESET` carries the +negotiated speed, which does not exist until the reset ends. That missing vocabulary is +the actual defect; the driver-level workarounds considered (deferring the memclr, guarding +primes with a private flag) only shrink the window. + +## Design + +### 1. Stack: split the bus-reset event into two edges + +`src/device/dcd.h`: + +```c +DCD_EVENT_BUS_RESET_START, // reset signaling detected; bus unusable, speed unknown +DCD_EVENT_BUS_RESET_END, // reset complete; .bus_reset.speed is final +... +#define DCD_EVENT_BUS_RESET DCD_EVENT_BUS_RESET_END // backward compatibility +``` + +No new helper: `dcd_event_bus_reset(rhport, speed, in_isr)` keeps its name and emits +`_END`, so every other port is bit-identical to today; `_START` uses the existing +payload-free `dcd_event_bus_signal()`. The alias keeps unit-test/fuzz references +compiling. + +**Contract (documented in `dcd.h`):** `_START` is optional. A DCD that cannot distinguish +the two edges emits only `_END`, which stays self-sufficient — it performs the full +teardown with or without a preceding `_START`. + +`src/device/usbd.c`: +- `case DCD_EVENT_BUS_RESET_START:` → `usbd_reset(rhport)` only; speed untouched. +- `case DCD_EVENT_BUS_RESET_END:` → unchanged (`usbd_reset()` + latch speed). +- `_usbd_event_str[]` gains both names. +- `TODO:` note that a DCD signalling both edges should not pay for two teardowns — track + a per-rhport "start seen" flag and skip the redundant `usbd_reset()` in `_END`, keeping + the unconditional teardown for the legacy single-event path. + +Cost, accepted deliberately: one extra queued event and one extra `usbd_reset()` per +enumeration on ci_hs only, bounded at one per reset against a default +`CFG_TUD_TASK_QUEUE_SZ` of 16 (queue pressure is the failure PR #3817 fixed, hence the +explicit note). + +### 2. ci_hs: split `bus_reset()` along the register/software line + +- **`bus_reset_begin()` — at URI, inside the reset window (UM10503 25.10.3):** ENDPTCTRL + type-reset loop, `ENDPTNAK`/`ENDPTNAKEN`, `ENDPTSETUPSTAT` and `ENDPTCOMPLETE` + write-back clears, bounded `ENDPTPRIME` drain, `ENDPTFLUSH` all. Emit `_START`. + Registers only — nothing in `_dcd_data` is touched, so no software structure is pulled + out from under a task mid-`dcd_edpt_xfer`. +- **`bus_reset_complete()` — at the PCI ending the reset:** re-flush, `tu_memclr(&_dcd_data)`, + EP0 queue-head re-init, dcache clean. Emit `_END` with the final PSPD speed. + +Two properties fall out: the re-flush kills any prime armed during the window without a +new state flag, and the memclr now happens at the same instant usbd is told, so the +"configured over zeroed queue heads" mismatch is eliminated rather than shrunk. Residual +exposure (a task priming exactly as the ISR memclrs) equals master's. + +The reason-dispatch (`pci_reason`, suspend/URI ordering) is unchanged; only the reset +case's body moves. + +### 3. ci_hs: one bounded-flush helper + +Extract `flush_endpoints(dcd_reg, mask)` — writes `ENDPTFLUSH = mask`, spins bounded by +`CI_HS_BUSY_SPIN` until those bits clear, returns `true` if they cleared — and route all +five flush sites through it (`bus_reset_begin`, `bus_reset_complete`, `dcd_deinit`, +`dcd_edpt_iso_activate`, the setup-time EP0 flush). The unified part is the mechanism +(one bound, one spin idiom, one return convention); callers keep their existing reactions, +all of which currently proceed regardless, and that stays true here — no caller gains new +error handling in this wave. Without this, §2 adds a fifth site to a file that already +carried four hand-rolled variants. + +### 4. Mechanical fixes + +`dcd_ci_hs.c` +- Setup-time EP0 flush waits for completion (via §3's helper) before the SETUP event is + queued, so the flush can no longer still be asserted when the task primes the response — + which also dissolves its interaction with the post-prime verify. This adds a bounded + spin in ISR context; the RM notes a flush waits out any packet already in progress, so + the wait is one packet time (microseconds at HS) and the existing `CI_HS_BUSY_SPIN` + bound caps the pathological case, consistent with the file's other flush sites. +- `dcd_set_address()` writes `DEVICEADDR` only if the status-ZLP prime took. A refused + prime means a newer SETUP superseded the transfer; staging an address whose ACK will + never arrive is wrong. +- Emit `DCD_EVENT_RESUME` only when `!(PORTSC1 & PORTSC1_SUSPEND)` (restores master's + hardware guard, lost in the rework). + +`dcd_lpc_ip3511.c` +- Deliver the setup copy only when known-good: + `if (latch still set) { INTSETSTAT = TU_BIT(0); } else { dcd_event_setup_received(...); }`. +- `TODO:` token on the USB.13 deferral so backlog sweeps surface it. + +`usbd.c` +- The DCD-refusal path in `usbd_edpt_xfer` stops routing through the breakpoint-carrying + assert: a DCD declining a prime is documented and self-healing, not a programming error, + and `TU_BREAKPOINT()` is not gated on `CFG_TUSB_DEBUG` — with a probe attached (always, + on the rig) it halts the target. Log and return false instead. + +BSP +- Delete the seven-line RHPORT block in `lpcxpresso55s28/board.cmake` (byte-identical to + `family.cmake`'s own guards; `board.mk`'s `?=` stays as the idiomatic Make form). +- `lpc11u37.ld`: correct the stale comment (nothing lands in RamUsb2 in either build + system now — the stack owns the whole bank) and keep the ASSERT, re-labelled as + future-proofing. + +## Findings improved for free (documented, no code) + +A reset that starts and never completes — cable pulled mid-reset — now delivers `_START` +and tears usbd down, where before usbd stayed configured on a dead bus. This softens both +the adjudicated UNPLUGGED-removal finding and the deferred aborted-reset item: a stray +later PCI delivering `_END` becomes harmless (usbd already torn down, just latches a +speed) instead of deconfiguring a live device. True detach detection still requires OTGSC +B-session-valid VBUS sensing — board-dependent, still a follow-up. + +## Explicitly deferred + +- Prime verification generalized to all endpoints and all causes (RM 25.10.8.2); the + EP0/SETUP-gated form stays, its flush interaction fixed by §4. +- usbd discards `usbd_control_xfer_cb`/`tud_control_xfer` returns — cross-DCD behavior + change needing its own regression pass, despite `usbd.c` being open here. +- Timed-out flush still proceeds to the memclr (now confined to one helper). +- LPC55S2x USB.3 FORCE_FS workaround; iso-IN 1023 enforcement; 8-byte OUT-spill + enforcement; USB.13 INTONNAK workaround. +- Gating `TU_BREAKPOINT()` on `CFG_TUSB_DEBUG` stack-wide. +- Unguarded `set()` RHPORT knobs in ~14 sibling `board.cmake` files. + +## Verification + +1. `pre-commit run --all-files`; builds for mimxrt1064_evk, lpcxpresso18s37, + lpcxpresso11u37, lpcxpresso55s28, plus Make link checks for the two previously-broken + targets (`host/cdc_msc_hid` on 55s28, `device/cdc_msc_throughput` on 11u37). +2. Cross-DCD build guard: one non-ci_hs, non-ip3511 board (e.g. `stm32f407disco`) to prove + the `DCD_EVENT_BUS_RESET` alias keeps legacy ports compiling untouched. +3. HIL on byte-verified flash (`verifyfile` on every J-Link load — the 1064's silent + flash no-op has struck twice): usbtest 30/30 on mimxrt1064_evk, lpcxpresso55s28, + lpcxpresso11u37; 50× case-9/10 loops on the 1064; 10× case-11/12/24 unlink loops. +4. Reset-path specific: confirm HS enumeration (480) and, with `LOG=2`, that a single + enumeration shows exactly one `_START`/`_END` pair and no spurious RESUME. +5. Suspend/resume exercise on the 1064 (host-side autosuspend on the port) confirming + `SUSPEND`/`RESUME` pairing and no reset misclassification. + +## Success criteria + +All four regressions closed, no new findings in a scoped re-review of the wave diff, every +listed HIL result green on verified flash, and legacy DCDs provably untouched (alias build +check + unchanged `_END` semantics). diff --git a/docs/superpowers/specs/2026-08-16-drop-ep0-prime-verify-design.md b/docs/superpowers/specs/2026-08-16-drop-ep0-prime-verify-design.md new file mode 100644 index 000000000..cc1840972 --- /dev/null +++ b/docs/superpowers/specs/2026-08-16-drop-ep0-prime-verify-design.md @@ -0,0 +1,90 @@ +# Drop the EP0 post-prime verify — design + +Date: 2026-08-16 +Branch: `fix-ci-hs` (unpushed, 19 commits over merge-base `53fef2833`) + +## Context + +The branch grew while chasing a wedge on `mimxrt1064_evk`: the board would stop answering a +host transfer, the URB would never complete, `testusb` would block uninterruptibly and the +whole rig would follow it down. Eight occurrences over four days, across the Linux usbtest +battery's queued control and bulk tests. + +The cause turned out to be silicon: **Errata i.MX RT1064_A / RT1060_A ERR050101**. While an +isochronous IN endpoint is active, an IN token addressed to that same endpoint number on +another device sharing the host silently unprimes one of this device's OUT endpoints — +control, bulk, interrupt or isochronous. NXP states it cannot be detected by software and +raises no interrupt. Moving the usbtest example's iso IN endpoint from 3 to 7 (commit +`42870b15b`) cleared it: 340 consecutive wedge-free runs, where the board previously +re-wedged within hours. + +Before that was known, an earlier theory — a SETUP arriving mid-prime silently cancelling an +EP0 prime — produced a post-prime verification block in `qhd_start_xfer()`. That theory's +supporting capture (EP0's status ZLP armed but unprimed, the device a control transfer ahead +of the host) is explained by ERR050101 just as well, because the errata explicitly covers +*control* OUT endpoints and a control status stage **is** an OUT endpoint. The generalized +version of that verify was already reverted (`565bb0d99`) as both regression-prone and aimed +at a failure the vendor documents as undetectable in software. This spec removes what +remains of it. + +## Change + +Delete the post-prime block in `qhd_start_xfer()` (`src/portable/chipidea/ci_hs/dcd_ci_hs.c`): +the bounded `ENDPTPRIME` drain, the `ENDPTFLUSH`-on-timeout, and the +`ENDPTSTAT | ENDPTCOMPLETE` / `ENDPTSETUPSTAT` verdict. The tail becomes: + +```c + // start transfer + dcd_reg->ENDPTPRIME = TU_BIT(epnum + (dir ? 16 : 0)); + return true; +``` + +This removes two register spins and four volatile reads from every EP0 transfer, and with +them the false-fail path a reviewer flagged: a transfer the interrupt handler has already +completed reads identically to a cancelled prime. + +## Deliberately kept + +- **The pre-prime setup-lockout guard** directly above it — UM10503 25.10.8.1.1 step 4 + verbatim ("Before priming for status/handshake phases ensure that ENDPTSETUPSTAT is '0'"), + and older than the wedge theory. It also keeps `qhd_start_xfer()` returning `bool`, so + `dcd_set_address()`'s gating and the usbd breakpoint removal stay meaningful — no cascade. +- **The setup-time EP0 flush and its completion wait** — the flush is the 25.10.8.1.1 step-3 + remark; the wait exists because an unfinished flush can retire a freshly primed response, + an interaction independent of the verify. +- **The `BUS_RESET_START`/`END` split** and the rest of the review-driven hardening. +- Everything hardware-proven: the rf_tv fix, the lpc11u37 stack move, the lpc55s28 + onboarding, the lpc55 Make OHCI link, and the ERR050101 endpoint move itself. + +The commit message records the corrected attribution of the handoff capture, so the next +reader does not re-derive the superseded theory from the same evidence. + +## Validation + +The "with it" arm is already banked from 2026-08-16: 10x 30/30 batteries plus 15x TEST 27, +15x tests 9/10 and 10x tests 11/12/24, all clean. This is the second half of an A/B. + +1. **Rebase onto current master first** (master has moved: midi2/usbtmc/video), then rebuild — + otherwise the validated tree is not the tree that merges. +2. **Software gates:** `pre-commit run --all-files`; full example builds for + mimxrt1064_evk, lpcxpresso18s37, lpcxpresso11u37, lpcxpresso55s28; the two Make link + canaries (`host/cdc_msc_hid` on lpcxpresso55s28, `device/cdc_msc_throughput` on + lpcxpresso11u37); `ceedling test:all`. +3. **Hardware — mimxrt1064_evk only.** It is the only ci_hs board on the rig; the other two + run ip3511, which this change does not touch. Preconditions: CI idle + (`pgrep -f "hil_test.py [-]-retry"`), board lock held for the whole run. Flash with + `loadfile` (its built-in Program & Verify — JLinkExe V9.66 has no `verifyfile`), then + confirm re-enumeration as `cafe:4010` with serial `BAE96FB95AFA6DBB8F00005002001200`, and + confirm `lsusb -v` still reports the iso IN endpoint as **0x87** so a stale image cannot + masquerade as a pass. +4. **Runs:** 5x the full 30-case battery, then 15x `--tests 9,10,14,21` (queued control, ch9 + subset, both ctrl_out cases) — the control paths the verify actually protected, which a + plain battery samples only once per run. Print a `testusb` D-state scan after every + iteration. + +**Acceptance:** 5/5 batteries at 30/30, 15/15 loops, and no `testusb` D-state outliving its +case runtime. + +**Rollback trigger:** any control-case failure (errno 110 or 71 on cases 9, 10, 14, 21) or a +lingering D-state means the verify was load-bearing after all — restore it and record that +result in the commit message. A negative result is a finding, not a setback. diff --git a/examples/device/99-tinyusb.rules b/examples/device/99-tinyusb-examples.rules index d306bada5..e7a399345 100644 --- a/examples/device/99-tinyusb.rules +++ b/examples/device/99-tinyusb-examples.rules @@ -1,5 +1,6 @@ +# udev rules for running the TinyUSB device examples as a non-root user. # Copy this file to the location of your distribution's udev rules, for example on Ubuntu: -# sudo cp 99-tinyusb.rules /etc/udev/rules.d/ +# sudo cp 99-tinyusb-examples.rules /etc/udev/rules.d/ # Then reload udev configuration by executing: # sudo udevadm control --reload-rules # sudo udevadm trigger diff --git a/examples/device/CMakeLists.txt b/examples/device/CMakeLists.txt index 1432b36bb..02fcc0b87 100644 --- a/examples/device/CMakeLists.txt +++ b/examples/device/CMakeLists.txt @@ -35,6 +35,7 @@ set(EXAMPLE_LIST printer_to_cdc uac2_headset uac2_speaker_fb + usbtest usbtmc video_capture video_capture_2ch diff --git a/examples/device/audio_4_channel_mic/src/usb_descriptors.c b/examples/device/audio_4_channel_mic/src/usb_descriptors.c index 6b9a9bbae..020bc934e 100644 --- a/examples/device/audio_4_channel_mic/src/usb_descriptors.c +++ b/examples/device/audio_4_channel_mic/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] AUDIO | MIDI | HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(AUDIO, 4) | PID_MAP(VENDOR, 5) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4001 //--------------------------------------------------------------------+ // Device Descriptors @@ -99,6 +92,10 @@ enum // Only EP3 is available for ISO #define EPNUM_AUDIO 0x03 +#elif CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101 + // ERR050101 (see CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101): an isochronous IN endpoint needs a number + // no other device on the bus uses + #define EPNUM_AUDIO 0x07 #else #define EPNUM_AUDIO 0x01 #endif diff --git a/examples/device/audio_4_channel_mic_freertos/src/usb_descriptors.c b/examples/device/audio_4_channel_mic_freertos/src/usb_descriptors.c index 216cd062a..0afb3df0a 100644 --- a/examples/device/audio_4_channel_mic_freertos/src/usb_descriptors.c +++ b/examples/device/audio_4_channel_mic_freertos/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] AUDIO | MIDI | HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(AUDIO, 4) | PID_MAP(VENDOR, 5) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4002 //--------------------------------------------------------------------+ // Device Descriptors diff --git a/examples/device/audio_test/src/usb_descriptors.c b/examples/device/audio_test/src/usb_descriptors.c index cea4eb8d1..d16526116 100644 --- a/examples/device/audio_test/src/usb_descriptors.c +++ b/examples/device/audio_test/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] AUDIO | MIDI | HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(AUDIO, 4) | PID_MAP(VENDOR, 5) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4003 //--------------------------------------------------------------------+ // Device Descriptors @@ -99,6 +92,10 @@ enum // Only EP3 is available for ISO #define EPNUM_AUDIO 0x03 +#elif CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101 + // ERR050101 (see CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101): an isochronous IN endpoint needs a number + // no other device on the bus uses + #define EPNUM_AUDIO 0x07 #else #define EPNUM_AUDIO 0x01 #endif diff --git a/examples/device/audio_test_freertos/src/usb_descriptors.c b/examples/device/audio_test_freertos/src/usb_descriptors.c index 37ebf84d3..709425c49 100644 --- a/examples/device/audio_test_freertos/src/usb_descriptors.c +++ b/examples/device/audio_test_freertos/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] AUDIO | MIDI | HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(AUDIO, 4) | PID_MAP(VENDOR, 5) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4004 //--------------------------------------------------------------------+ // Device Descriptors diff --git a/examples/device/audio_test_multi_rate/src/usb_descriptors.c b/examples/device/audio_test_multi_rate/src/usb_descriptors.c index b1f60dd10..008c8cd76 100644 --- a/examples/device/audio_test_multi_rate/src/usb_descriptors.c +++ b/examples/device/audio_test_multi_rate/src/usb_descriptors.c @@ -28,15 +28,8 @@ #include "tusb.h" #include "usb_descriptors.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] AUDIO | MIDI | HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(AUDIO, 4) | PID_MAP(VENDOR, 5) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4005 //--------------------------------------------------------------------+ // Device Descriptors diff --git a/examples/device/cdc_dual_ports/src/usb_descriptors.c b/examples/device/cdc_dual_ports/src/usb_descriptors.c index adfd8cf9d..779221c0c 100644 --- a/examples/device/cdc_dual_ports/src/usb_descriptors.c +++ b/examples/device/cdc_dual_ports/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4006 #define USB_VID 0xCafe #define USB_BCD 0x0200 diff --git a/examples/device/cdc_msc/src/usb_descriptors.c b/examples/device/cdc_msc/src/usb_descriptors.c index 5dc80dee3..140ef2140 100644 --- a/examples/device/cdc_msc/src/usb_descriptors.c +++ b/examples/device/cdc_msc/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4007 #define USB_VID 0xCafe #define USB_BCD 0x0200 diff --git a/examples/device/cdc_msc_freertos/src/usb_descriptors.c b/examples/device/cdc_msc_freertos/src/usb_descriptors.c index f5b015051..8398f0365 100644 --- a/examples/device/cdc_msc_freertos/src/usb_descriptors.c +++ b/examples/device/cdc_msc_freertos/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4008 #define USB_VID 0xCafe #define USB_BCD 0x0200 diff --git a/examples/device/cdc_msc_throughput/src/usb_descriptors.c b/examples/device/cdc_msc_throughput/src/usb_descriptors.c index 3b0ff6e17..dca5a65cf 100644 --- a/examples/device/cdc_msc_throughput/src/usb_descriptors.c +++ b/examples/device/cdc_msc_throughput/src/usb_descriptors.c @@ -26,7 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -#define USB_PID (0x4000 | ((CFG_TUD_CDC) ? (1 << 0) : 0) | ((CFG_TUD_MSC) ? (1 << 1) : 0)) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4009 #define USB_VID 0xCafe #define USB_BCD 0x0200 @@ -64,7 +65,15 @@ enum { }; // Place bulk endpoints on EP>=8 for MAX32690 class parts (bigger FIFO, DPB-capable). -#if CFG_TUD_ENDPOINT_ONE_DIRECTION_ONLY +#if CFG_TUSB_MCU == OPT_MCU_LPC175X_6X || CFG_TUSB_MCU == OPT_MCU_LPC177X_8X || CFG_TUSB_MCU == OPT_MCU_LPC40XX + // LPC 17xx and 40xx endpoint type (bulk/interrupt/iso) are fixed by its number + // 0 control, 1 In, 2 Bulk, 3 Iso, 4 In, 5 Bulk etc ... + #define EPNUM_CDC_NOTIF 0x81 + #define EPNUM_CDC_OUT 0x02 + #define EPNUM_CDC_IN 0x82 + #define EPNUM_MSC_OUT 0x05 + #define EPNUM_MSC_IN 0x85 +#elif CFG_TUD_ENDPOINT_ONE_DIRECTION_ONLY #if TU_CHECK_MCU(OPT_MCU_MAX32650, OPT_MCU_MAX32666, OPT_MCU_MAX32690, OPT_MCU_MAX78002) // Put bulk on EP>=8 so the 2048/4096-byte FIFOs can back double packet buffering #define EPNUM_CDC_NOTIF 0x81 diff --git a/examples/device/cdc_uac2/src/tusb_config.h b/examples/device/cdc_uac2/src/tusb_config.h index 358ff6747..f54a9b606 100644 --- a/examples/device/cdc_uac2/src/tusb_config.h +++ b/examples/device/cdc_uac2/src/tusb_config.h @@ -109,8 +109,10 @@ extern "C" { #define CFG_TUD_AUDIO_FUNC_1_N_FORMATS 2 // Audio format type I specifications -#if defined(__RX__) -#define CFG_TUD_AUDIO_FUNC_1_MAX_SAMPLE_RATE 48000 // 16bit/48kHz is the best quality for Renesas RX +#if defined(__RX__) || (CFG_TUSB_MCU == OPT_MCU_PY32F0) +// RX : 16bit/48kHz is the best quality for Renesas RX +// PY32F0 : 48kHz/16bit keeps ISO packets <= 98 B so both directions fit the fixed EP FIFOs (EP1 512 B, EP2 128 B) +#define CFG_TUD_AUDIO_FUNC_1_MAX_SAMPLE_RATE 48000 #else #define CFG_TUD_AUDIO_FUNC_1_MAX_SAMPLE_RATE 96000 // 24bit/96kHz is the best quality for full-speed, high-speed is needed beyond this #endif @@ -123,7 +125,7 @@ extern "C" { #define CFG_TUD_AUDIO_FUNC_1_FORMAT_1_N_BYTES_PER_SAMPLE_RX 2 #define CFG_TUD_AUDIO_FUNC_1_FORMAT_1_RESOLUTION_RX 16 -#if defined(__RX__) +#if defined(__RX__) || (CFG_TUSB_MCU == OPT_MCU_PY32F0) // 8bit in 8bit slots #define CFG_TUD_AUDIO_FUNC_1_FORMAT_2_N_BYTES_PER_SAMPLE_TX 1 #define CFG_TUD_AUDIO_FUNC_1_FORMAT_2_RESOLUTION_TX 8 diff --git a/examples/device/cdc_uac2/src/usb_descriptors.c b/examples/device/cdc_uac2/src/usb_descriptors.c index fdffc761e..648c56e71 100644 --- a/examples/device/cdc_uac2/src/usb_descriptors.c +++ b/examples/device/cdc_uac2/src/usb_descriptors.c @@ -29,15 +29,8 @@ #include "tusb.h" #include "usb_descriptors.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] AUDIO | MIDI | HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(AUDIO, 4) | PID_MAP(VENDOR, 5) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x400a //--------------------------------------------------------------------+ // Device Descriptors @@ -117,6 +110,16 @@ uint8_t const * tud_descriptor_device_cb(void) #define EPNUM_CDC_IN 0x85 #endif +#elif CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101 + // ERR050101 (see CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101): an isochronous IN endpoint needs a number + // no other device on the bus uses + #define EPNUM_AUDIO_IN 0x07 + #define EPNUM_AUDIO_OUT 0x01 + + #define EPNUM_CDC_NOTIF 0x83 + #define EPNUM_CDC_OUT 0x04 + #define EPNUM_CDC_IN 0x84 + #else #define EPNUM_AUDIO_IN 0x01 #define EPNUM_AUDIO_OUT 0x01 diff --git a/examples/device/dfu/src/usb_descriptors.c b/examples/device/dfu/src/usb_descriptors.c index e4291be2d..5bfb32b2e 100644 --- a/examples/device/dfu/src/usb_descriptors.c +++ b/examples/device/dfu/src/usb_descriptors.c @@ -26,14 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4)) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x400b //--------------------------------------------------------------------+ // Device Descriptors diff --git a/examples/device/dfu_runtime/src/usb_descriptors.c b/examples/device/dfu_runtime/src/usb_descriptors.c index 8fa078da2..273566414 100644 --- a/examples/device/dfu_runtime/src/usb_descriptors.c +++ b/examples/device/dfu_runtime/src/usb_descriptors.c @@ -26,14 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4)) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x400c //--------------------------------------------------------------------+ // Device Descriptors diff --git a/examples/device/dynamic_configuration/src/usb_descriptors.c b/examples/device/dynamic_configuration/src/usb_descriptors.c index c4049414f..838052ea1 100644 --- a/examples/device/dynamic_configuration/src/usb_descriptors.c +++ b/examples/device/dynamic_configuration/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x400d // Configuration mode // 0 : enumerated as CDC/MIDI. Board button is not pressed when enumerating @@ -79,7 +72,7 @@ tusb_desc_device_t const desc_device_1 = .bMaxPacketSize0 = CFG_TUD_ENDPOINT0_SIZE, .idVendor = 0xCafe, - .idProduct = USB_PID + 11, // should be different PID than desc0 + .idProduct = USB_PID + 0x0100, // must differ from desc0's PID and stay outside the 0x40xx per-example space .bcdDevice = 0x0100, .iManufacturer = 0x01, diff --git a/examples/device/hid_boot_interface/src/usb_descriptors.c b/examples/device/hid_boot_interface/src/usb_descriptors.c index b5c31a94a..4d5caa835 100644 --- a/examples/device/hid_boot_interface/src/usb_descriptors.c +++ b/examples/device/hid_boot_interface/src/usb_descriptors.c @@ -27,14 +27,8 @@ #include "tusb.h" #include "usb_descriptors.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4)) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x400e //--------------------------------------------------------------------+ // Device Descriptors diff --git a/examples/device/hid_composite/src/usb_descriptors.c b/examples/device/hid_composite/src/usb_descriptors.c index 46e4b63f9..7f5f74c70 100644 --- a/examples/device/hid_composite/src/usb_descriptors.c +++ b/examples/device/hid_composite/src/usb_descriptors.c @@ -27,15 +27,8 @@ #include "tusb.h" #include "usb_descriptors.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x400f #define USB_VID 0xCafe #define USB_BCD 0x0200 diff --git a/examples/device/hid_composite_freertos/src/usb_descriptors.c b/examples/device/hid_composite_freertos/src/usb_descriptors.c index a745c17b5..a0464afae 100644 --- a/examples/device/hid_composite_freertos/src/usb_descriptors.c +++ b/examples/device/hid_composite_freertos/src/usb_descriptors.c @@ -27,15 +27,8 @@ #include "tusb.h" #include "usb_descriptors.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4011 #define USB_VID 0xCafe #define USB_BCD 0x0200 diff --git a/examples/device/hid_generic_inout/src/usb_descriptors.c b/examples/device/hid_generic_inout/src/usb_descriptors.c index 93e718461..f179b74f7 100644 --- a/examples/device/hid_generic_inout/src/usb_descriptors.c +++ b/examples/device/hid_generic_inout/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4012 //--------------------------------------------------------------------+ // Device Descriptors diff --git a/examples/device/hid_multiple_interface/src/usb_descriptors.c b/examples/device/hid_multiple_interface/src/usb_descriptors.c index cd2d93c44..90aef6dd7 100644 --- a/examples/device/hid_multiple_interface/src/usb_descriptors.c +++ b/examples/device/hid_multiple_interface/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4013 //--------------------------------------------------------------------+ // Device Descriptors diff --git a/examples/device/midi2_device/src/main.c b/examples/device/midi2_device/src/main.c index 2c77652dc..f0162ad04 100644 --- a/examples/device/midi2_device/src/main.c +++ b/examples/device/midi2_device/src/main.c @@ -565,6 +565,19 @@ const char* tud_midi2_fb_name_cb(uint8_t itf, uint8_t fb_idx) { return (fb_idx == 0) ? "Synth Out" : "Keys In"; } +// Sent when the host asks for a Device Identity Notification (Endpoint +// Discovery 'd' filter bit). Same four fields as the MIDI 1.0 Device Inquiry +// reply; 0x7D is the prototyping SysEx ID, placed in the first of the three +// manufacturer bytes. +bool tud_midi2_device_identity_cb(uint8_t itf, tud_midi2_device_identity_t* identity) { + (void)itf; + identity->manufacturer = 0x7D0000; + identity->family = 0x0001; + identity->model = 0x0001; + identity->sw_revision = 0x00010000; + return true; +} + //--------------------------------------------------------------------+ // Initial Setup - Program Change, CC, Per-Note Management //--------------------------------------------------------------------+ diff --git a/examples/device/midi_test/src/usb_descriptors.c b/examples/device/midi_test/src/usb_descriptors.c index 99c798ce1..fc7228c35 100644 --- a/examples/device/midi_test/src/usb_descriptors.c +++ b/examples/device/midi_test/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4014 //--------------------------------------------------------------------+ // Device Descriptors diff --git a/examples/device/midi_test_freertos/src/usb_descriptors.c b/examples/device/midi_test_freertos/src/usb_descriptors.c index 99c798ce1..bfdbc555e 100644 --- a/examples/device/midi_test_freertos/src/usb_descriptors.c +++ b/examples/device/midi_test_freertos/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4015 //--------------------------------------------------------------------+ // Device Descriptors diff --git a/examples/device/msc_dual_lun/src/usb_descriptors.c b/examples/device/msc_dual_lun/src/usb_descriptors.c index b328cf17f..5e036a8c4 100644 --- a/examples/device/msc_dual_lun/src/usb_descriptors.c +++ b/examples/device/msc_dual_lun/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4016 //--------------------------------------------------------------------+ // Device Descriptors diff --git a/examples/device/mtp/src/usb_descriptors.c b/examples/device/mtp/src/usb_descriptors.c index 4c840560e..fefa8a239 100644 --- a/examples/device/mtp/src/usb_descriptors.c +++ b/examples/device/mtp/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] MTP | VENDOR | MIDI | HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) | PID_MAP(MTP, 5)) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4017 #define USB_VID 0xCafe #define USB_BCD 0x0200 diff --git a/examples/device/net_lwip_webserver/skip.txt b/examples/device/net_lwip_webserver/skip.txt index 5e5562087..53836b581 100644 --- a/examples/device/net_lwip_webserver/skip.txt +++ b/examples/device/net_lwip_webserver/skip.txt @@ -1,3 +1,4 @@ +mcu:APM32F0XX mcu:CH32V103 mcu:CH32V20X mcu:LPC11UXX @@ -10,6 +11,7 @@ mcu:SAMD11 mcu:STM32L0 mcu:STM32F0 mcu:KINETIS_KL +mcu:PY32F0 mcu:STM32H7RS mcu:STM32N6 family:broadcom_64bit diff --git a/examples/device/net_lwip_webserver/src/usb_descriptors.c b/examples/device/net_lwip_webserver/src/usb_descriptors.c index 09090bb92..85a6a420c 100644 --- a/examples/device/net_lwip_webserver/src/usb_descriptors.c +++ b/examples/device/net_lwip_webserver/src/usb_descriptors.c @@ -27,16 +27,8 @@ #include "class/net/net_device.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] NET | VENDOR | MIDI | HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID \ - (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) | \ - PID_MAP(ECM_RNDIS, 5) | PID_MAP(NCM, 5)) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4018 // String Descriptor Index enum { diff --git a/examples/device/printer_to_cdc/src/usb_descriptors.c b/examples/device/printer_to_cdc/src/usb_descriptors.c index db7bfe97a..92cd2b6be 100644 --- a/examples/device/printer_to_cdc/src/usb_descriptors.c +++ b/examples/device/printer_to_cdc/src/usb_descriptors.c @@ -29,7 +29,7 @@ #include "usb_descriptors.h" #define USB_VID 0xCafe -#define USB_PID 0x4005 +#define USB_PID 0x4019 #define USB_BCD 0x0200 //--------------------------------------------------------------------+ @@ -67,7 +67,15 @@ uint8_t const *tud_descriptor_device_cb(void) { //--------------------------------------------------------------------+ // Endpoint numbers -#if CFG_TUD_ENDPOINT_ONE_DIRECTION_ONLY +#if CFG_TUSB_MCU == OPT_MCU_LPC175X_6X || CFG_TUSB_MCU == OPT_MCU_LPC177X_8X || CFG_TUSB_MCU == OPT_MCU_LPC40XX + // LPC 17xx and 40xx endpoint type (bulk/interrupt/iso) are fixed by its number + // 0 control, 1 In, 2 Bulk, 3 Iso, 4 In, 5 Bulk etc ... + #define EPNUM_CDC_NOTIF 0x81 + #define EPNUM_CDC_OUT 0x02 + #define EPNUM_CDC_IN 0x82 + #define EPNUM_PRINTER_OUT 0x05 + #define EPNUM_PRINTER_IN 0x85 +#elif CFG_TUD_ENDPOINT_ONE_DIRECTION_ONLY #if TU_CHECK_MCU(OPT_MCU_MAX32650, OPT_MCU_MAX32666, OPT_MCU_MAX32690, OPT_MCU_MAX78002) // Put bulk on EP>=8 so the 2048/4096-byte FIFOs can back double packet buffering #define EPNUM_CDC_NOTIF 0x81 diff --git a/examples/device/uac2_headset/src/usb_descriptors.c b/examples/device/uac2_headset/src/usb_descriptors.c index b554e7195..d8cdd768e 100644 --- a/examples/device/uac2_headset/src/usb_descriptors.c +++ b/examples/device/uac2_headset/src/usb_descriptors.c @@ -28,15 +28,8 @@ #include "tusb.h" #include "usb_descriptors.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] AUDIO | MIDI | HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(AUDIO, 4) | PID_MAP(VENDOR, 5) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x401a //--------------------------------------------------------------------+ // Device Descriptors @@ -105,12 +98,24 @@ uint8_t const * tud_descriptor_device_cb(void) #define EPNUM_AUDIO_OUT 0x0A #define EPNUM_AUDIO_IN 0x0B #define EPNUM_AUDIO_INT 0x01 + #elif TU_CHECK_MCU(OPT_MCU_PY32F0) + // Speaker OUT (196 B) only fits EP1 (512 B FIFO); mic IN (98 B) fits EP2 (128 B) + #define EPNUM_AUDIO_OUT 0x01 + #define EPNUM_AUDIO_IN 0x02 + #define EPNUM_AUDIO_INT 0x03 #else #define EPNUM_AUDIO_IN 0x01 #define EPNUM_AUDIO_OUT 0x02 #define EPNUM_AUDIO_INT 0x03 #endif +#elif CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101 + // ERR050101 (see CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101): an isochronous IN endpoint needs a number + // no other device on the bus uses + #define EPNUM_AUDIO_IN 0x07 + #define EPNUM_AUDIO_OUT 0x01 + #define EPNUM_AUDIO_INT 0x02 + #else #define EPNUM_AUDIO_IN 0x01 #define EPNUM_AUDIO_OUT 0x01 diff --git a/examples/device/uac2_speaker_fb/src/usb_descriptors.c b/examples/device/uac2_speaker_fb/src/usb_descriptors.c index f0c780e38..40a36cbf4 100644 --- a/examples/device/uac2_speaker_fb/src/usb_descriptors.c +++ b/examples/device/uac2_speaker_fb/src/usb_descriptors.c @@ -28,15 +28,8 @@ #include "usb_descriptors.h" #include "common_types.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] AUDIO | MIDI | HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(AUDIO, 4) | PID_MAP(VENDOR, 5) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x401b //--------------------------------------------------------------------+ // Device Descriptors diff --git a/examples/device/usbtest/CMakeLists.txt b/examples/device/usbtest/CMakeLists.txt new file mode 100644 index 000000000..10414ede7 --- /dev/null +++ b/examples/device/usbtest/CMakeLists.txt @@ -0,0 +1,30 @@ +cmake_minimum_required(VERSION 3.20) + +include(${CMAKE_CURRENT_SOURCE_DIR}/../../../hw/bsp/family_support.cmake) + +project(usbtest C CXX ASM) + +# Checks this example is valid for the family and initializes the project +family_initialize_project(${PROJECT_NAME} ${CMAKE_CURRENT_LIST_DIR}) + +# Espressif has its own cmake build system +if(FAMILY STREQUAL "espressif") + return() +endif() + +add_executable(${PROJECT_NAME}) + +# Example source +target_sources(${PROJECT_NAME} PUBLIC + ${CMAKE_CURRENT_SOURCE_DIR}/src/main.c + ${CMAKE_CURRENT_SOURCE_DIR}/src/usb_descriptors.c + ) + +# Example include +target_include_directories(${PROJECT_NAME} PUBLIC + ${CMAKE_CURRENT_SOURCE_DIR}/src + ) + +# Configure compilation flags and libraries for the example without RTOS. +# See the corresponding function in hw/bsp/FAMILY/family.cmake for details. +family_configure_device_example(${PROJECT_NAME} noos) diff --git a/examples/device/usbtest/CMakePresets.json b/examples/device/usbtest/CMakePresets.json new file mode 100644 index 000000000..5cd8971e9 --- /dev/null +++ b/examples/device/usbtest/CMakePresets.json @@ -0,0 +1,6 @@ +{ + "version": 6, + "include": [ + "../../../hw/bsp/BoardPresets.json" + ] +} diff --git a/examples/device/usbtest/Makefile b/examples/device/usbtest/Makefile new file mode 100644 index 000000000..035e90308 --- /dev/null +++ b/examples/device/usbtest/Makefile @@ -0,0 +1,11 @@ +include ../../../hw/bsp/family_support.mk + +INC += \ + src \ + + +# Example source +EXAMPLE_SOURCE += $(wildcard src/*.c) +SRC_C += $(addprefix $(EXAMPLE_PATH)/, $(EXAMPLE_SOURCE)) + +include ../../../hw/bsp/family_rules.mk diff --git a/examples/device/usbtest/README.md b/examples/device/usbtest/README.md new file mode 100644 index 000000000..8fcc4ad36 --- /dev/null +++ b/examples/device/usbtest/README.md @@ -0,0 +1,94 @@ +# usbtest + +Device-side peer of the Linux kernel USB test pair: + +- `usbtest.ko` — host kernel module (`drivers/usb/misc/usbtest.c`) containing ~30 numbered + test cases over bulk/control/interrupt/isochronous transfers. +- `testusb` — userspace dispatcher (`tools/usb/testusb.c`) that tells the module which case + to run via usbfs ioctl. + +This example implements the Gadget-Zero style *source/sink* protocol on a vendor-specific +interface so the whole battery can exercise TinyUSB device controller drivers: + +- bulk IN = infinite source (usbtest pattern 0: all zeros) +- bulk OUT = infinite sink (data discarded) + +## Tiers + +The firmware advertises its capability tier in `bcdDevice` (`0x01TT`); the host script picks +the matching test battery automatically. + +| Tier | Capability | usbtest cases | +|------|------------|---------------| +| 1 | bulk source/sink | 0, 9, 10, 1–8, 11, 12, 24, 13, 29, 17–20, 27, 28 | +| 2 | + vendor control `0x5b`/`0x5c` (ctrl_out) | + 14, 21 | +| 3 | + interrupt source/sink | + 25, 26 | +| 4 | + isochronous source/sink | + 15, 16, 22, 23 | + +This example implements all four tiers using the vendor class with the interrupt +(`CFG_TUD_VENDOR_EP_INT_OUT/IN`) and isochronous (`CFG_TUD_VENDOR_EP_ISO_OUT/IN`) +endpoint pairs and altsetting support (`CFG_TUD_VENDOR_ALT_SETTINGS`): alt 0 +carries no endpoints, alt 1 the full source/sink set, per USB 2.0 5.6.3 (the host +usbtest driver selects alt 1 itself). + +## Test cases + +Directions are from the host's point of view: *write* = host→device (OUT endpoint, device +sinks and discards), *read* = device→host (IN endpoint, device sources zeros and the host +verifies every byte). All checking happens host-side in `usbtest.ko`; a case fails on a data +mismatch, an unexpected short packet/STALL, or a timeout. What each case stresses on the +device/DCD side: + +| # | Name | What it does / what it exercises | +|---|------|----------------------------------| +| 0 | NOP | ioctl round-trip sanity, no USB traffic — proves the interface bound with the right capability profile | +| 9 | ch9 subset | chapter-9 standard control requests (GET_DESCRIPTOR, GET_STATUS, SET/CLEAR_FEATURE, SET_INTERFACE, …) — the EP0 state machine, incl. status stages and ZLPs | +| 10 | queued control | many control URBs in flight at once — EP0 under sustained back-to-back SETUPs | +| 1 / 2 | bulk write / read | plain OUT sink / IN source streams of whole max-size packets — FIFO handling, multi-packet transfers | +| 3 / 4 | bulk write / read vary | same with transfer sizes varying per URB — short packets and packet-boundary edge cases | +| 5–8 | bulk sg write/read (+vary) | scatter-gather queued URBs — continuous packet pressure with no inter-URB gap; classic overflow/babble catcher | +| 11 / 12 | unlink reads / writes | URBs submitted then cancelled mid-flight — the device keeps streaming while the host aborts; DCD abort/cleanup paths | +| 24 | unlink queued writes | unlink from a deep OUT queue — same, under queue pressure | +| 13 | ep halt set/clear | SET_FEATURE(ENDPOINT_HALT), verify the endpoint really STALLs, then CLEAR_FEATURE and verify traffic resumes at DATA0 — stall must abort an armed transfer (and flush any loaded FIFO) | +| 29 | toggle clear | CLEAR_FEATURE(HALT) on a **non-halted** endpoint mid-traffic, purely to reset the data toggle — the DCD must reset DATA0 *without* disarming the queued transfer (historically the most common per-DCD bug in this battery) | +| 17 / 18 | bulk write / read unaligned | bulk streams from oddly-offset host buffers — host DMA-alignment path; the device sees normal traffic | +| 19 / 20 | bulk write / read premapped | bulk streams using host pre-mapped DMA buffers — another host memory path | +| 27 / 28 | bulk write / read perf | sustained maximum-throughput streams, reported in MB/s — real-time FIFO servicing under load | +| 14 | ctrl_out write/read | vendor EP0 request `0x5b` stores wLength bytes, `0x5c` reads them back, sizes varying — multi-packet control-OUT data stages and buffer persistence across requests | +| 21 | ctrl_out unaligned | same from odd host buffer offsets | +| 25 / 26 | int write / read | interrupt OUT sink / IN source at the descriptor's polling interval — interrupt endpoint arming and completion | +| 15 / 16 | iso write / read | isochronous OUT sink / IN source, one packet per (micro)frame with per-packet status — no handshake/retry, DATA0-only; the IN source must re-arm fast enough to make every frame deadline | +| 22 / 23 | iso write / read unaligned | same from odd host buffer offsets | + +Per-case iteration counts and sizes are chosen by `test/hil/usbtest.py` for the negotiated +speed (see its `PARAMS` table); the authoritative case implementations live in the kernel's +`drivers/usb/misc/usbtest.c`. + +## Running + +Use the host script (handles driver binding, per-case parameters, result parsing): + +```bash +python3 test/hil/usbtest.py --serial <board-uid> +``` + +Requirements on the host: `usbtest` kernel module (`CONFIG_USB_TEST`, `modprobe usbtest`), +the `testusb` binary built from kernel `tools/usb/testusb.c`, and sudo (usbfs ioctls + +driver bind/unbind). + +Manual runs are possible but beware `testusb` defaults: always pass explicit `-s`/`-v` +values that are multiples of 512 — the device streams whole max-size packets, so a +non-packet-aligned read length overflows (`-EOVERFLOW`), and never run bare `testusb -a` +(the default parameter set includes cases with invalid parameters and hour-long runtimes +at full speed). + +```bash +# bind: MUST use the 5-field form referencing Gadget Zero (0525:a4a0) so the +# dynamic id inherits its capability profile. A plain "cafe 4010" id leaves +# driver_info NULL, which usbtest_probe() dereferences -> kernel oops. +sudo modprobe usbtest +echo "cafe 4010 0 0525 a4a0" | sudo tee /sys/bus/usb/drivers/usbtest/new_id +# example: bulk write/read +sudo testusb -D /dev/bus/usb/<BBB>/<DDD> -t 1 -c 128 -s 1024 -v 512 +sudo testusb -D /dev/bus/usb/<BBB>/<DDD> -t 2 -c 128 -s 1024 -v 512 +``` diff --git a/examples/device/usbtest/skip.txt b/examples/device/usbtest/skip.txt new file mode 100644 index 000000000..792404fe4 --- /dev/null +++ b/examples/device/usbtest/skip.txt @@ -0,0 +1,13 @@ +mcu:MSP430x5xx +mcu:NUC121 +mcu:SAMD11 +# DCD has no isochronous support (dcd_edpt_iso_alloc refuses), tier-4 cannot enumerate: +mcu:CXD56 +mcu:FT90X +mcu:NUC100 +mcu:NUC120 +mcu:NUC505 +mcu:PIC32MZ +mcu:SAMG +mcu:SAMX7X +mcu:VALENTYUSB_EPTRI diff --git a/examples/device/usbtest/src/CMakeLists.txt b/examples/device/usbtest/src/CMakeLists.txt new file mode 100644 index 000000000..cef2b46ee --- /dev/null +++ b/examples/device/usbtest/src/CMakeLists.txt @@ -0,0 +1,4 @@ +# This file is for ESP-IDF only +idf_component_register(SRCS "main.c" "usb_descriptors.c" + INCLUDE_DIRS "." + REQUIRES boards tinyusb_src) diff --git a/examples/device/usbtest/src/main.c b/examples/device/usbtest/src/main.c new file mode 100644 index 000000000..e57a90161 --- /dev/null +++ b/examples/device/usbtest/src/main.c @@ -0,0 +1,344 @@ +/* + * The MIT License (MIT) + * + * Copyright (c) 2026 Ha Thach (tinyusb.org) + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + */ + +/* Device-side peer of the Linux kernel host test driver drivers/usb/misc/usbtest.c + * (driven from userspace by tools/usb/testusb.c). Implements the Gadget-Zero + * style source/sink protocol on a vendor interface (alt 0 = no endpoints, + * alt 1 = full set, selected by the host usbtest driver): + * - bulk/interrupt/isochronous IN = infinite source (pattern 0: all zeros) + * - bulk/interrupt/isochronous OUT = infinite sink (data discarded) + * - EP0 0x5b/0x5c = control write then read-back (ctrl_out tests) + * See examples/device/usbtest/README.md and test/hil/usbtest.py for usage. + */ + +#include <stdlib.h> +#include <stdio.h> +#include <string.h> + +#include "bsp/board_api.h" +#include "tusb.h" +#include "usb_descriptors.h" + +//--------------------------------------------------------------------+ +// MACRO CONSTANT TYPEDEF PROTYPES +//--------------------------------------------------------------------+ + +/* Blink pattern + * - 250 ms : device not mounted + * - 1000 ms : device mounted + */ +enum { + BLINK_NOT_MOUNTED = 250, + BLINK_MOUNTED = 1000, +}; + +static uint32_t blink_interval_ms = BLINK_NOT_MOUNTED; + +// Source data, all zeros = usbtest pattern 0. Sizes are a multiple of the +// endpoint max packet size: transfers are always whole packets, never an +// unintended short packet or ZLP. +static uint8_t const tx_chunk[CFG_TUD_VENDOR_TX_EPSIZE]; +static uint8_t const int_tx_chunk[USBTEST_INT_EP_MPS]; +#if USBTEST_TIER >= 4 +static uint8_t const iso_tx_chunk[USBTEST_ISO_EP_MPS]; +#endif + +// Interrupt/iso submit one packet per (micro)frame, sized to the NEGOTIATED speed's mps — a +// high-speed build enumerated at full speed must submit the FS length, not the HS-capacity buffer +// size (bulk is exempt: it streams multi-packet transfers). See usb_descriptors.h. +static inline uint16_t usbtest_int_len(void) { + return (tud_speed_get() == TUSB_SPEED_HIGH) ? USBTEST_INT_EP_MPS_HS : USBTEST_INT_EP_MPS_FS; +} +#if USBTEST_TIER >= 4 +static inline uint16_t usbtest_iso_len(void) { + return (tud_speed_get() == TUSB_SPEED_HIGH) ? USBTEST_ISO_EP_MPS_HS : USBTEST_ISO_EP_MPS_FS; +} +#endif + +//------------- prototypes -------------// +void led_blinking_task(void* param); +void usbtest_task(void* param); + +#if CFG_TUSB_OS == OPT_OS_FREERTOS +void freertos_init(void); +#endif + +/*------------- MAIN -------------*/ +int main(void) { + board_init(); + + // If using FreeRTOS: create blinky, tinyusb device, and usbtest source/sink tasks +#if CFG_TUSB_OS == OPT_OS_FREERTOS + freertos_init(); +#else + // init device stack on configured roothub port + tusb_rhport_init_t dev_init = { + .role = TUSB_ROLE_DEVICE, + .speed = TUSB_SPEED_AUTO + }; + tusb_init(BOARD_TUD_RHPORT, &dev_init); + + board_init_after_tusb(); + + while (1) { + tud_task(); // tinyusb device task + usbtest_task(NULL); + led_blinking_task(NULL); + } +#endif +} + +//--------------------------------------------------------------------+ +// Source/sink pumps +//--------------------------------------------------------------------+ + +// Polling keeps all four endpoints armed and self-heals after endpoint halt +// (set/clear feature tests): stall marks the endpoint busy so the calls fail +// quietly until the host clears the halt, then the next tick re-arms. +static void usbtest_pump(void) { + if (tud_vendor_mounted()) { + tud_vendor_read_xfer(); // bulk sink: arm/re-arm, quiet fail if armed or halted + if (tud_vendor_write_available()) { // 0 while bulk IN is busy or halted + tud_vendor_write(tx_chunk, sizeof(tx_chunk)); + } + + tud_vendor_int_read_xfer(); // interrupt sink + if (tud_vendor_int_write_available()) { + tud_vendor_int_write(int_tx_chunk, usbtest_int_len()); + } + +#if USBTEST_TIER >= 4 + tud_vendor_iso_read_xfer(); // isochronous sink + if (tud_vendor_iso_write_available()) { + tud_vendor_iso_write(iso_tx_chunk, usbtest_iso_len()); + } +#endif + } +} + +void usbtest_task(void* param) { + (void) param; + #if CFG_TUSB_OS == OPT_OS_FREERTOS + while (1) { + usbtest_pump(); + vTaskDelay(1); // yield; tx/rx completion callbacks keep the pipes saturated between polls + } + #else + usbtest_pump(); // called from the main loop, one tick per call + #endif +} + +// Invoked when received data from host: discard and immediately re-arm +void tud_vendor_rx_cb(uint8_t idx, const uint8_t* buffer, uint32_t bufsize) { + (void) idx; + (void) buffer; + (void) bufsize; + tud_vendor_read_xfer(); +} + +// Invoked when last bulk tx transfer finished: keep the source saturated +void tud_vendor_tx_cb(uint8_t idx, uint32_t sent_bytes) { + (void) idx; + (void) sent_bytes; + tud_vendor_write(tx_chunk, sizeof(tx_chunk)); +} + +// Interrupt pair: same discard/refill pumps as bulk +void tud_vendor_int_rx_cb(uint8_t idx, const uint8_t* buffer, uint32_t bufsize) { + (void) idx; + (void) buffer; + (void) bufsize; + tud_vendor_int_read_xfer(); +} + +void tud_vendor_int_tx_cb(uint8_t idx, uint32_t sent_bytes) { + (void) idx; + (void) sent_bytes; + tud_vendor_int_write(int_tx_chunk, usbtest_int_len()); +} + +// Isochronous pair: same discard/refill pumps; a completion may be a missed +// frame, re-arm regardless +#if USBTEST_TIER >= 4 +void tud_vendor_iso_rx_cb(uint8_t idx, const uint8_t* buffer, uint32_t bufsize) { + (void) idx; + (void) buffer; + (void) bufsize; + tud_vendor_iso_read_xfer(); +} + +void tud_vendor_iso_tx_cb(uint8_t idx, uint32_t sent_bytes) { + (void) idx; + (void) sent_bytes; + tud_vendor_iso_write(iso_tx_chunk, usbtest_iso_len()); +} +#endif + +//--------------------------------------------------------------------+ +// Vendor control requests (EP0) +//--------------------------------------------------------------------+ + +// Control write/read-back for the ctrl_out tests (14/21), same protocol as +// Gadget Zero: 0x5b stores the host's wLength bytes, 0x5c returns them. +static uint8_t ctrl_buf[1024]; + +// Invoked on vendor control transfers, and by usbd for forwarded standard +// endpoint requests (halt set/clear) whose return value it ignores — return +// false for anything that is not a supported vendor request. +bool tud_vendor_control_xfer_cb(uint8_t rhport, uint8_t stage, tusb_control_request_t const* request) { + if (request->bmRequestType_bit.type != TUSB_REQ_TYPE_VENDOR) { + return false; + } + + switch (request->bRequest) { + case 0x5b: // control WRITE: receive wLength bytes into ctrl_buf + TU_VERIFY(request->bmRequestType_bit.direction == TUSB_DIR_OUT); + TU_VERIFY(request->wValue == 0 && request->wIndex == 0); + TU_VERIFY(request->wLength <= sizeof(ctrl_buf)); + if (stage == CONTROL_STAGE_SETUP) { + return tud_control_xfer(rhport, request, ctrl_buf, request->wLength); + } + return true; // DATA/ACK: payload already landed in ctrl_buf + + case 0x5c: // control READ: send back the previously written bytes + TU_VERIFY(request->bmRequestType_bit.direction == TUSB_DIR_IN); + TU_VERIFY(request->wValue == 0 && request->wIndex == 0); + TU_VERIFY(request->wLength <= sizeof(ctrl_buf)); + if (stage == CONTROL_STAGE_SETUP) { + return tud_control_xfer(rhport, request, ctrl_buf, request->wLength); + } + return true; + + default: + return false; + } +} + +//--------------------------------------------------------------------+ +// Device callbacks +//--------------------------------------------------------------------+ + +// Invoked when device is mounted +void tud_mount_cb(void) { + blink_interval_ms = BLINK_MOUNTED; +} + +// Invoked when device is unmounted +void tud_umount_cb(void) { + blink_interval_ms = BLINK_NOT_MOUNTED; +} + +//--------------------------------------------------------------------+ +// BLINKING TASK +//--------------------------------------------------------------------+ +void led_blinking_task(void* param) { + (void) param; + static uint32_t start_ms = 0; + static bool led_state = false; + + while (1) { + #if CFG_TUSB_OS == OPT_OS_FREERTOS + vTaskDelay(blink_interval_ms / portTICK_PERIOD_MS); + #else + // Blink every interval ms + if (tusb_time_millis_api() - start_ms < blink_interval_ms) { + return; // not enough time + } + #endif + + start_ms += blink_interval_ms; + board_led_write(led_state); + led_state = 1 - led_state; // toggle + } +} + +//--------------------------------------------------------------------+ +// FreeRTOS +//--------------------------------------------------------------------+ +#if CFG_TUSB_OS == OPT_OS_FREERTOS + +#define BLINKY_STACK_SIZE configMINIMAL_STACK_SIZE +#define USBTEST_STACK_SIZE (configMINIMAL_STACK_SIZE*2) + +#ifdef ESP_PLATFORM + #define USBD_STACK_SIZE 4096 + int main(void); + void app_main(void) { + main(); + } +#else + // Increase stack size when debug log is enabled + #define USBD_STACK_SIZE (3*configMINIMAL_STACK_SIZE/2) * (CFG_TUSB_DEBUG ? 2 : 1) +#endif + +// static task allocation +#if configSUPPORT_STATIC_ALLOCATION +StackType_t blinky_stack[BLINKY_STACK_SIZE]; +StaticTask_t blinky_taskdef; + +StackType_t usb_device_stack[USBD_STACK_SIZE]; +StaticTask_t usb_device_taskdef; + +StackType_t usbtest_stack[USBTEST_STACK_SIZE]; +StaticTask_t usbtest_taskdef; +#endif + +// USB Device Driver task: processes all usb events and invokes callbacks +void usb_device_task(void* param) { + (void) param; + + // init device stack on configured roothub port. Must be called after the + // scheduler starts: the USB IRQ handler uses RTOS queue APIs. + tusb_rhport_init_t dev_init = { + .role = TUSB_ROLE_DEVICE, + .speed = TUSB_SPEED_AUTO + }; + tusb_init(BOARD_TUD_RHPORT, &dev_init); + + board_init_after_tusb(); + + // RTOS forever loop + while (1) { + tud_task(); // put thread to waiting state until there is a new event + } +} + +void freertos_init(void) { + #if configSUPPORT_STATIC_ALLOCATION + xTaskCreateStatic(led_blinking_task, "blinky", BLINKY_STACK_SIZE, NULL, 1, blinky_stack, &blinky_taskdef); + xTaskCreateStatic(usb_device_task, "usbd", USBD_STACK_SIZE, NULL, configMAX_PRIORITIES-1, usb_device_stack, &usb_device_taskdef); + xTaskCreateStatic(usbtest_task, "usbtest", USBTEST_STACK_SIZE, NULL, configMAX_PRIORITIES-2, usbtest_stack, &usbtest_taskdef); + #else + xTaskCreate(led_blinking_task, "blinky", BLINKY_STACK_SIZE, NULL, 1, NULL); + xTaskCreate(usb_device_task, "usbd", USBD_STACK_SIZE, NULL, configMAX_PRIORITIES-1, NULL); + xTaskCreate(usbtest_task, "usbtest", USBTEST_STACK_SIZE, NULL, configMAX_PRIORITIES-2, NULL); + #endif + + // only start scheduler for non-espressif mcu (espressif starts it in startup code) + #ifndef ESP_PLATFORM + vTaskStartScheduler(); + #endif +} +#endif diff --git a/examples/device/usbtest/src/tusb_config.h b/examples/device/usbtest/src/tusb_config.h new file mode 100644 index 000000000..dda131ae2 --- /dev/null +++ b/examples/device/usbtest/src/tusb_config.h @@ -0,0 +1,151 @@ +/* + * The MIT License (MIT) + * + * Copyright (c) 2026 Ha Thach (tinyusb.org) + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + */ + +#ifndef TUSB_CONFIG_H_ +#define TUSB_CONFIG_H_ + +#ifdef __cplusplus + extern "C" { +#endif + +//--------------------------------------------------------------------+ +// Board Specific Configuration +//--------------------------------------------------------------------+ + +// RHPort number used for device can be defined by board.mk, default to port 0 +#ifndef BOARD_TUD_RHPORT +#define BOARD_TUD_RHPORT 0 +#endif + +// RHPort max operational speed can defined by board.mk +#ifndef BOARD_TUD_MAX_SPEED +#define BOARD_TUD_MAX_SPEED OPT_MODE_DEFAULT_SPEED +#endif + +//-------------------------------------------------------------------- +// Common Configuration +//-------------------------------------------------------------------- + +// defined by compiler flags for flexibility +#ifndef CFG_TUSB_MCU +#error CFG_TUSB_MCU must be defined +#endif + +#ifndef CFG_TUSB_OS +#define CFG_TUSB_OS OPT_OS_NONE +#endif + +// Espressif IDF requires "freertos/" prefix in include path +#ifdef ESP_PLATFORM +#define CFG_TUSB_OS_INC_PATH freertos/ +#endif + +#ifndef CFG_TUSB_DEBUG +#define CFG_TUSB_DEBUG 0 +#endif + +// Enable Device stack +#define CFG_TUD_ENABLED 1 + +// Default is max speed that hardware controller could support with on-chip PHY +#define CFG_TUD_MAX_SPEED BOARD_TUD_MAX_SPEED + +/* USB DMA on some MCUs can only access a specific SRAM region with restriction on alignment. + * Tinyusb use follows macros to declare transferring memory so that they can be put + * into those specific section. + * e.g + * - CFG_TUSB_MEM SECTION : __attribute__ (( section(".usb_ram") )) + * - CFG_TUSB_MEM_ALIGN : __attribute__ ((aligned(4))) + */ +#ifndef CFG_TUSB_MEM_SECTION +#define CFG_TUSB_MEM_SECTION +#endif + +#ifndef CFG_TUSB_MEM_ALIGN +#define CFG_TUSB_MEM_ALIGN __attribute__ ((aligned(4))) +#endif + +//-------------------------------------------------------------------- +// DEVICE CONFIGURATION +//-------------------------------------------------------------------- + +#ifndef CFG_TUD_ENDPOINT0_SIZE +#define CFG_TUD_ENDPOINT0_SIZE 64 +#endif + +//------------- CLASS -------------// +#define CFG_TUD_CDC 0 +#define CFG_TUD_MSC 0 +#define CFG_TUD_HID 0 +#define CFG_TUD_MIDI 0 +#define CFG_TUD_VENDOR 1 + +// Non-buffered mode: every transfer is submitted with an exact length so the +// host never sees an unexpected short packet or ZLP mid-transfer, which the +// usbtest data-integrity cases treat as failure. +#define CFG_TUD_VENDOR_RX_BUFSIZE 0 +#define CFG_TUD_VENDOR_TX_BUFSIZE 0 + +// App re-arms RX itself: required to recover the sink after halt tests +// (SET_FEATURE/CLEAR_FEATURE endpoint halt) where no completion ever fires. +#define CFG_TUD_VENDOR_RX_MANUAL_XFER 1 + +// Multi-packet IN transfers; must be a multiple of bulk MPS at both speeds (64/512). +// LPC11/13 (ip3511 FS) keep endpoint buffers in a dedicated 2 KB USB RAM: a 2048 B bulk epbuf +// overflows it once the int/iso buffers join, so those parts use 512 (= 8 FS packets). +#if TU_CHECK_MCU(OPT_MCU_LPC11UXX, OPT_MCU_LPC13XX) +#define CFG_TUD_VENDOR_TX_EPSIZE 512 +#else +#define CFG_TUD_VENDOR_TX_EPSIZE 2048 +#endif + +// Interrupt IN/OUT source/sink pair (usbtest cases 25/26). Buffer sizes track the +// per-speed endpoint max packet size (see usb_descriptors.c) so full-speed builds +// don't over-allocate the scarce USB DMA section. +#define CFG_TUD_VENDOR_EP_INT_OUT 1 +#define CFG_TUD_VENDOR_EP_INT_IN 1 +#define CFG_TUD_VENDOR_EP_INT_OUT_BUFSIZE (TUD_OPT_HIGH_SPEED ? 512 : 64) +#define CFG_TUD_VENDOR_EP_INT_IN_BUFSIZE (TUD_OPT_HIGH_SPEED ? 512 : 64) + +// Isochronous IN/OUT source/sink pair (usbtest cases 15/16/22/23), placed in +// altsetting 1: alt 0 has no endpoints so no iso bandwidth is claimed by default +#define CFG_TUD_VENDOR_EP_ISO_OUT 1 +#define CFG_TUD_VENDOR_EP_ISO_IN 1 +#define CFG_TUD_VENDOR_EP_ISO_OUT_BUFSIZE (TUD_OPT_HIGH_SPEED ? 512 : 128) +#define CFG_TUD_VENDOR_EP_ISO_IN_BUFSIZE (TUD_OPT_HIGH_SPEED ? 512 : 128) +#define CFG_TUD_VENDOR_ALT_SETTINGS 1 + +// CH32V20X fsdev port has only 512 B PMA and single-buffered iso can't keep the iso IN endpoint +// fed under load. Double-buffer iso; the descriptor drops iso mps to 32 there so 2x32 = 64 B/ep +// keeps the same PMA budget (see usb_descriptors.h). Other fsdev parts have room and stay single. +#if CFG_TUSB_MCU == OPT_MCU_CH32V20X +#define CFG_TUD_FSDEV_DOUBLE_BUFFERED_ISO_EP 1 +#endif + +#ifdef __cplusplus + } +#endif + +#endif /* TUSB_CONFIG_H_ */ diff --git a/examples/device/usbtest/src/usb_descriptors.c b/examples/device/usbtest/src/usb_descriptors.c new file mode 100644 index 000000000..8453885c4 --- /dev/null +++ b/examples/device/usbtest/src/usb_descriptors.c @@ -0,0 +1,295 @@ +/* + * The MIT License (MIT) + * + * Copyright (c) 2026 Ha Thach (tinyusb.org) + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + */ + +#include "bsp/board_api.h" +#include "tusb.h" +#include "usb_descriptors.h" + +//--------------------------------------------------------------------+ +// Device Descriptors +//--------------------------------------------------------------------+ +static tusb_desc_device_t const desc_device = { + .bLength = sizeof(tusb_desc_device_t), + .bDescriptorType = TUSB_DESC_DEVICE, + .bcdUSB = 0x0200, + .bDeviceClass = 0x00, // per-interface class + .bDeviceSubClass = 0x00, + .bDeviceProtocol = 0x00, + .bMaxPacketSize0 = CFG_TUD_ENDPOINT0_SIZE, + + .idVendor = 0xCafe, + .idProduct = 0x4010, + .bcdDevice = 0x0100 | USBTEST_TIER, + + .iManufacturer = 0x01, + .iProduct = 0x02, + .iSerialNumber = 0x03, + + .bNumConfigurations = 0x01 +}; + +// Invoked when received GET DEVICE DESCRIPTOR +uint8_t const* tud_descriptor_device_cb(void) { + return (uint8_t const*) &desc_device; +} + +//--------------------------------------------------------------------+ +// Configuration Descriptor +//--------------------------------------------------------------------+ + +// Interface must be number 0: testusb -D issues its ioctls against interface 0 +enum { + ITF_NUM_VENDOR = 0, + ITF_NUM_TOTAL +}; + +// Vendor interface, Gadget-Zero style altsettings: alt 0 carries no endpoints (an +// isochronous endpoint must not claim bandwidth in the default altsetting, USB 2.0 +// 5.6.3), alt 1 carries bulk + interrupt (+ isochronous IN/OUT at tier 4). The host +// usbtest driver skips altsettings without pipes and selects alt 1 itself. No TUD_ +// macro covers this layout, hand-rolled. +#if USBTEST_TIER >= 4 + #define USBTEST_EP_COUNT 6 + #define USBTEST_ISO_EPS(_isoout, _isoin, _iso_mps, _iso_interval) \ + ,7, TUSB_DESC_ENDPOINT, _isoout, (uint8_t)(TUSB_XFER_ISOCHRONOUS | (uint8_t)(TUSB_ISO_EP_ATT_ASYNCHRONOUS)), U16_TO_U8S_LE(_iso_mps), _iso_interval,\ + 7, TUSB_DESC_ENDPOINT, _isoin, (uint8_t)(TUSB_XFER_ISOCHRONOUS | (uint8_t)(TUSB_ISO_EP_ATT_ASYNCHRONOUS)), U16_TO_U8S_LE(_iso_mps), _iso_interval +#else + #define USBTEST_EP_COUNT 4 + #define USBTEST_ISO_EPS(_isoout, _isoin, _iso_mps, _iso_interval) +#endif +#define USBTEST_DESC_LEN (9 + 9 + USBTEST_EP_COUNT*7) +#define USBTEST_DESCRIPTOR(_itfnum, _stridx, _epout, _epin, _bulk_mps, _intout, _intin, _int_mps, _int_interval, _isoout, _isoin, _iso_mps, _iso_interval) \ + /* alt 0: zero bandwidth, no endpoints */\ + 9, TUSB_DESC_INTERFACE, _itfnum, 0, 0, TUSB_CLASS_VENDOR_SPECIFIC, 0x00, 0x00, _stridx,\ + /* alt 1: full source/sink set */\ + 9, TUSB_DESC_INTERFACE, _itfnum, 1, USBTEST_EP_COUNT, TUSB_CLASS_VENDOR_SPECIFIC, 0x00, 0x00, _stridx,\ + 7, TUSB_DESC_ENDPOINT, _epout, TUSB_XFER_BULK, U16_TO_U8S_LE(_bulk_mps), 0,\ + 7, TUSB_DESC_ENDPOINT, _epin, TUSB_XFER_BULK, U16_TO_U8S_LE(_bulk_mps), 0,\ + 7, TUSB_DESC_ENDPOINT, _intout, TUSB_XFER_INTERRUPT, U16_TO_U8S_LE(_int_mps), _int_interval,\ + 7, TUSB_DESC_ENDPOINT, _intin, TUSB_XFER_INTERRUPT, U16_TO_U8S_LE(_int_mps), _int_interval\ + USBTEST_ISO_EPS(_isoout, _isoin, _iso_mps, _iso_interval) + +#define CONFIG_TOTAL_LEN (TUD_CONFIG_DESC_LEN + USBTEST_DESC_LEN) + +#if CFG_TUSB_MCU == OPT_MCU_LPC175X_6X || CFG_TUSB_MCU == OPT_MCU_LPC177X_8X || CFG_TUSB_MCU == OPT_MCU_LPC40XX + // LPC 17xx and 40xx endpoint type (bulk/interrupt/iso) are fixed by its number + // 0 control, 1 Interrupt, 2 Bulk, 3 Iso, 4 Interrupt etc ... + #define EPNUM_BULK_OUT 0x02 + #define EPNUM_BULK_IN 0x85 + #define EPNUM_INT_OUT 0x01 + #define EPNUM_INT_IN 0x84 + #define EPNUM_ISO_OUT 0x03 + #define EPNUM_ISO_IN 0x86 + +#elif CFG_TUD_ENDPOINT_ONE_DIRECTION_ONLY + // MCUs that don't support a same endpoint number with different direction IN and OUT + // e.g EP1 OUT & EP1 IN cannot exist together + #if TU_CHECK_MCU(OPT_MCU_MAX32650, OPT_MCU_MAX32666, OPT_MCU_MAX32690, OPT_MCU_MAX78002) + // Put bulk on EP>=8 so the 2048/4096-byte FIFOs can back double packet buffering + #define EPNUM_BULK_OUT 0x08 + #define EPNUM_BULK_IN 0x89 + #define EPNUM_INT_OUT 0x02 + #define EPNUM_INT_IN 0x83 + #define EPNUM_ISO_OUT 0x04 + #define EPNUM_ISO_IN 0x85 + #else + #define EPNUM_BULK_OUT 0x01 + #define EPNUM_BULK_IN 0x82 + #define EPNUM_INT_OUT 0x03 + #define EPNUM_INT_IN 0x84 + #define EPNUM_ISO_OUT 0x05 + #define EPNUM_ISO_IN 0x86 + #endif + +#elif CFG_TUSB_MCU == OPT_MCU_NRF5X + // nRF5x: ISO endpoints are hardware-fixed to EP8 (ISOOUT/ISOIN) + #define EPNUM_BULK_OUT 0x01 + #define EPNUM_BULK_IN 0x81 + #define EPNUM_INT_OUT 0x02 + #define EPNUM_INT_IN 0x82 + #define EPNUM_ISO_OUT 0x08 + #define EPNUM_ISO_IN 0x88 + +#elif CFG_TUSB_MCU == OPT_MCU_MIMXRT1XXX + #define EPNUM_BULK_OUT 0x01 + #define EPNUM_BULK_IN 0x81 + #define EPNUM_INT_OUT 0x02 + #define EPNUM_INT_IN 0x82 + #define EPNUM_ISO_OUT 0x03 + // ERR050101 (see CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101): park the iso IN endpoint clear of the numbers + // other devices use. Override per board when several affected boards share a hub. + #ifndef EPNUM_ISO_IN + #if CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101 + #define EPNUM_ISO_IN 0x87 + #else + #define EPNUM_ISO_IN 0x83 + #endif + #endif + +#else + #define EPNUM_BULK_OUT 0x01 + #define EPNUM_BULK_IN 0x81 + #define EPNUM_INT_OUT 0x02 + #define EPNUM_INT_IN 0x82 + #define EPNUM_ISO_OUT 0x03 + #define EPNUM_ISO_IN 0x83 +#endif + +static uint8_t const desc_fs_configuration[] = { + // Config number, interface count, string index, total length, attribute, power in mA + TUD_CONFIG_DESCRIPTOR(1, ITF_NUM_TOTAL, 0, CONFIG_TOTAL_LEN, 0x00, 100), + + // Interface number, string index, bulk out/in + mps, int out/in + mps + interval, iso out/in + mps + interval + USBTEST_DESCRIPTOR(ITF_NUM_VENDOR, 4, EPNUM_BULK_OUT, 0x80 | EPNUM_BULK_IN, 64, + EPNUM_INT_OUT, 0x80 | EPNUM_INT_IN, USBTEST_INT_EP_MPS_FS, 1, + EPNUM_ISO_OUT, 0x80 | EPNUM_ISO_IN, USBTEST_ISO_EP_MPS_FS, 1) +}; + +#if TUD_OPT_HIGH_SPEED +// Per USB specs: high speed capable device must report device_qualifier and other_speed_configuration + +static uint8_t const desc_hs_configuration[] = { + // Config number, interface count, string index, total length, attribute, power in mA + TUD_CONFIG_DESCRIPTOR(1, ITF_NUM_TOTAL, 0, CONFIG_TOTAL_LEN, 0x00, 100), + + // Interface number, string index, bulk out/in + mps, int out/in + mps + interval, iso out/in + mps + interval (1 ms) + USBTEST_DESCRIPTOR(ITF_NUM_VENDOR, 4, EPNUM_BULK_OUT, 0x80 | EPNUM_BULK_IN, 512, + EPNUM_INT_OUT, 0x80 | EPNUM_INT_IN, USBTEST_INT_EP_MPS_HS, 4, + EPNUM_ISO_OUT, 0x80 | EPNUM_ISO_IN, USBTEST_ISO_EP_MPS_HS, 4) +}; + +// other speed configuration +static uint8_t desc_other_speed_config[CONFIG_TOTAL_LEN]; + +// device qualifier is mostly similar to device descriptor since we don't change configuration based on speed +static tusb_desc_device_qualifier_t const desc_device_qualifier = { + .bLength = sizeof(tusb_desc_device_qualifier_t), + .bDescriptorType = TUSB_DESC_DEVICE_QUALIFIER, + .bcdUSB = 0x0200, + + .bDeviceClass = 0x00, + .bDeviceSubClass = 0x00, + .bDeviceProtocol = 0x00, + + .bMaxPacketSize0 = CFG_TUD_ENDPOINT0_SIZE, + .bNumConfigurations = 0x01, + .bReserved = 0x00 +}; + +// Invoked when received GET DEVICE QUALIFIER DESCRIPTOR request +uint8_t const* tud_descriptor_device_qualifier_cb(void) { + return (uint8_t const*) &desc_device_qualifier; +} + +// Invoked when received GET OTHER SPEED CONFIGURATION DESCRIPTOR request +// Configuration descriptor in the other speed e.g if high speed then this is for full speed and vice versa +uint8_t const* tud_descriptor_other_speed_configuration_cb(uint8_t index) { + (void) index; // for multiple configurations + + // if link speed is high return fullspeed config, and vice versa + // Note: the descriptor type is OTHER_SPEED_CONFIG instead of CONFIG + memcpy(desc_other_speed_config, + (tud_speed_get() == TUSB_SPEED_HIGH) ? desc_fs_configuration : desc_hs_configuration, + CONFIG_TOTAL_LEN); + + desc_other_speed_config[1] = TUSB_DESC_OTHER_SPEED_CONFIG; + + return desc_other_speed_config; +} +#endif // highspeed + +// Invoked when received GET CONFIGURATION DESCRIPTOR +uint8_t const* tud_descriptor_configuration_cb(uint8_t index) { + (void) index; // for multiple configurations + +#if TUD_OPT_HIGH_SPEED + // Although we are highspeed, host may be fullspeed. + return (tud_speed_get() == TUSB_SPEED_HIGH) ? desc_hs_configuration : desc_fs_configuration; +#else + return desc_fs_configuration; +#endif +} + +//--------------------------------------------------------------------+ +// String Descriptors +//--------------------------------------------------------------------+ + +// String Descriptor Index +enum { + STRID_LANGID = 0, + STRID_MANUFACTURER, + STRID_PRODUCT, + STRID_SERIAL, +}; + +// array of pointer to string descriptors +static char const* string_desc_arr[] = { + (const char[]) { 0x09, 0x04 }, // 0: is supported language is English (0x0409) + "TinyUSB", // 1: Manufacturer + "TinyUSB usbtest", // 2: Product + NULL, // 3: Serials will use unique ID if possible + "TinyUSB usbtest source/sink" // 4: Vendor Interface +}; + +static uint16_t _desc_str[32 + 1]; + +// Invoked when received GET STRING DESCRIPTOR request +// Application return pointer to descriptor, whose contents must exist long enough for transfer to complete +uint16_t const* tud_descriptor_string_cb(uint8_t index, uint16_t langid) { + (void) langid; + size_t chr_count; + + switch (index) { + case STRID_LANGID: + memcpy(&_desc_str[1], string_desc_arr[0], 2); + chr_count = 1; + break; + + case STRID_SERIAL: + chr_count = board_usb_get_serial(_desc_str + 1, 32); + break; + + default: + if (!(index < sizeof(string_desc_arr) / sizeof(string_desc_arr[0]))) return NULL; + + const char* str = string_desc_arr[index]; + + // Cap at max char + chr_count = strlen(str); + size_t const max_count = sizeof(_desc_str) / sizeof(_desc_str[0]) - 1; // -1 for string type + if (chr_count > max_count) chr_count = max_count; + + // Convert ASCII string into UTF-16 + for (size_t i = 0; i < chr_count; i++) { + _desc_str[1 + i] = str[i]; + } + break; + } + + // first byte is length (including header), second byte is string type + _desc_str[0] = (uint16_t) ((TUSB_DESC_STRING << 8) | (2 * chr_count + 2)); + + return _desc_str; +} diff --git a/examples/device/usbtest/src/usb_descriptors.h b/examples/device/usbtest/src/usb_descriptors.h new file mode 100644 index 000000000..bcf8b5ec4 --- /dev/null +++ b/examples/device/usbtest/src/usb_descriptors.h @@ -0,0 +1,84 @@ +/* + * The MIT License (MIT) + * + * Copyright (c) 2026 Ha Thach (tinyusb.org) + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + */ + +#ifndef USB_DESCRIPTORS_H_ +#define USB_DESCRIPTORS_H_ + +// Device-capability tier advertised in bcdDevice low byte (0x01TT), read by the +// host script to select which usbtest cases to run: +// 1: bulk source/sink +// 2: + vendor control 0x5b/0x5c (ctrl_out) +// 3: + interrupt source/sink +// 4: + isochronous source/sink +// Default is the full tier 4; a board whose DCD cannot serve a tier lowers it here +// (BOARD_<NAME> is defined by both build systems) and the host battery follows. +#ifndef USBTEST_TIER + #if defined(BOARD_RA2A1_EK) + // RA2A1's RUSB2 instance has no isochronous pipe (other RA parts have pipes 1-2) + #define USBTEST_TIER 3 + #else + #define USBTEST_TIER 4 + #endif +#endif + +// Interrupt/isochronous endpoint max packet sizes, must match the configuration descriptor. +// TUD_OPT_HIGH_SPEED is a compile-time capability flag, NOT the live bus speed, so the full-speed +// config descriptor (and the OTHER_SPEED descriptor served to a HS host) must use full-speed-legal +// sizes regardless of it: interrupt <= 64 B, isochronous <= 1023 B (and both iso EPs must fit the +// 1023 B/frame FS periodic budget). Hence separate _FS / _HS descriptor sizes; the plain macro +// below is the compile-time capability maximum that sizes the source buffers (runtime write +// lengths follow the negotiated speed via tud_speed_get(), see main.c). +// +// The CH32 USB IPs have tiny per-endpoint buffers so tier-4's six endpoints don't fit at the usual +// FS sizes: usbfs gives 64 B/ep (iso must drop to 64), and the CH32V20X fsdev port shares one 512 B +// PMA across every endpoint (needs iso 32 AND a small interrupt mps to fit alongside EP0+bulk+iso). +#if CFG_TUSB_MCU == OPT_MCU_CH32V20X && defined(CFG_TUD_WCH_USBIP_FSDEV) && CFG_TUD_WCH_USBIP_FSDEV + #define USBTEST_INT_EP_MPS_FS 16 + #define USBTEST_ISO_EP_MPS_FS 32 // double-buffered on fsdev: 2x32=64/ep, same 512 B PMA budget +#elif TU_CHECK_MCU(OPT_MCU_CH32V20X, OPT_MCU_CH32V103, OPT_MCU_CH32F20X, OPT_MCU_CH32V307, OPT_MCU_CH583) + // WCH USBFS parts cap every endpoint (except EP3 IN) at 64 B. For the CH32V307 this applies to its + // full-speed (usbfs) port; its high-speed (usbhs) port uses the _HS sizes below via desc_hs. + #define USBTEST_INT_EP_MPS_FS 64 + #define USBTEST_ISO_EP_MPS_FS 64 +#else + #define USBTEST_INT_EP_MPS_FS 64 + #define USBTEST_ISO_EP_MPS_FS 128 +#endif +// RUSB2 (Renesas RA) interrupt pipes 6-9 have a fixed 64-byte single buffer at any speed +// (RA6M5 UM R01UH0891 sec 29.1: "Pipes 6 to 9: Interrupt transfer with 64-byte single buffer"). +#if TU_CHECK_MCU(OPT_MCU_RAXXX) + #define USBTEST_INT_EP_MPS_HS 64 +#else + #define USBTEST_INT_EP_MPS_HS 512 +#endif +#define USBTEST_ISO_EP_MPS_HS 512 + +// Compile-time capability maximum: sizes the source buffers / vendor epbufs for the largest +// packet the build can negotiate. Runtime write lengths follow tud_speed_get() (see main.c) — +// a high-speed build enumerated at full speed submits the _FS lengths. +#define USBTEST_INT_EP_MPS (TUD_OPT_HIGH_SPEED ? USBTEST_INT_EP_MPS_HS : USBTEST_INT_EP_MPS_FS) +#define USBTEST_ISO_EP_MPS (TUD_OPT_HIGH_SPEED ? USBTEST_ISO_EP_MPS_HS : USBTEST_ISO_EP_MPS_FS) + +#endif /* USB_DESCRIPTORS_H_ */ diff --git a/examples/device/usbtmc/src/usb_descriptors.c b/examples/device/usbtmc/src/usb_descriptors.c index ecdcef834..5ba5d9367 100644 --- a/examples/device/usbtmc/src/usb_descriptors.c +++ b/examples/device/usbtmc/src/usb_descriptors.c @@ -28,15 +28,8 @@ #include "class/usbtmc/usbtmc.h" #include "class/usbtmc/usbtmc_device.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x401c #define USB_VID 0xcafe #define USB_BCD 0x0200 diff --git a/examples/device/video_capture/src/usb_descriptors.c b/examples/device/video_capture/src/usb_descriptors.c index b3382c82d..11321a305 100644 --- a/examples/device/video_capture/src/usb_descriptors.c +++ b/examples/device/video_capture/src/usb_descriptors.c @@ -27,15 +27,8 @@ #include "tusb.h" #include "usb_descriptors.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] VIDEO | AUDIO | MIDI | HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(AUDIO, 4) | PID_MAP(VIDEO, 5) | PID_MAP(VENDOR, 6) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x401d #define USB_VID 0xCafe #define USB_BCD 0x0200 @@ -120,6 +113,10 @@ enum { #define EPNUM_VIDEO_IN (CFG_TUD_VIDEO_STREAMING_BULK ? 0x81 : 0x88) #elif TU_CHECK_MCU(OPT_MCU_MAX32650, OPT_MCU_MAX32666, OPT_MCU_MAX32690, OPT_MCU_MAX78002) #define EPNUM_VIDEO_IN 0x81 +#elif CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101 + // ERR050101 (see CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101): an isochronous IN endpoint needs a number + // no other device on the bus uses + #define EPNUM_VIDEO_IN (CFG_TUD_VIDEO_STREAMING_BULK ? 0x81 : 0x87) #else #define EPNUM_VIDEO_IN 0x81 #endif diff --git a/examples/device/video_capture_2ch/src/usb_descriptors.c b/examples/device/video_capture_2ch/src/usb_descriptors.c index 8dc986da6..2630be84b 100644 --- a/examples/device/video_capture_2ch/src/usb_descriptors.c +++ b/examples/device/video_capture_2ch/src/usb_descriptors.c @@ -27,15 +27,8 @@ #include "tusb.h" #include "usb_descriptors.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] VIDEO | AUDIO | MIDI | HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(AUDIO, 4) | PID_MAP(VIDEO, 5) | PID_MAP(VENDOR, 6) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x401e #define USB_VID 0xCafe #define USB_BCD 0x0200 @@ -117,8 +110,15 @@ enum { ITF_NUM_TOTAL }; -#define EPNUM_VIDEO_IN_1 0x81 -#define EPNUM_VIDEO_IN_2 0x82 +#if CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101 && !CFG_TUD_VIDEO_STREAMING_BULK + // ERR050101 (see CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101): an isochronous IN endpoint needs a number + // no other device on the bus uses + #define EPNUM_VIDEO_IN_1 0x86 + #define EPNUM_VIDEO_IN_2 0x87 +#else + #define EPNUM_VIDEO_IN_1 0x81 + #define EPNUM_VIDEO_IN_2 0x82 +#endif #if defined(CFG_EXAMPLE_VIDEO_READONLY) && !defined(CFG_EXAMPLE_VIDEO_DISABLE_MJPEG) #define USE_MJPEG 1 diff --git a/examples/device/webusb_serial/README.md b/examples/device/webusb_serial/README.md index 5ca70f909..15837e59e 100644 --- a/examples/device/webusb_serial/README.md +++ b/examples/device/webusb_serial/README.md @@ -51,4 +51,4 @@ make BOARD=raspberry_pi_pico all After flashing, open the landing page (`https://example.tinyusb.org/webusb-serial/index.html`) in a WebUSB-capable browser such as Chrome, click **Connect**, and select the device — the on-board LED lights solid once connected. Characters typed in the web page are echoed back, and are also mirrored to the CDC serial port (e.g. `/dev/ttyACM0`) and vice versa. -On Linux/macOS you may need to install the udev rules from `examples/device/99-tinyusb.rules` for the browser to access the device. +On Linux/macOS you may need to install the udev rules from `examples/device/99-tinyusb-examples.rules` for the browser to access the device. diff --git a/examples/device/webusb_serial/src/main.c b/examples/device/webusb_serial/src/main.c index 4be5e4db4..e200c334c 100644 --- a/examples/device/webusb_serial/src/main.c +++ b/examples/device/webusb_serial/src/main.c @@ -39,7 +39,7 @@ * is done automatically by firmware. * * - On Linux/macOS, udev permission may need to be updated by - * - copying '/examples/device/99-tinyusb.rules' file to /etc/udev/rules.d/ then + * - copying 'examples/device/99-tinyusb-examples.rules' file to /etc/udev/rules.d/ then * - run 'sudo udevadm control --reload-rules && sudo udevadm trigger' */ diff --git a/examples/device/webusb_serial/src/usb_descriptors.c b/examples/device/webusb_serial/src/usb_descriptors.c index 527837161..5986bffdf 100644 --- a/examples/device/webusb_serial/src/usb_descriptors.c +++ b/examples/device/webusb_serial/src/usb_descriptors.c @@ -27,15 +27,8 @@ #include "tusb.h" #include "usb_descriptors.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] MIDI | HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x401f //--------------------------------------------------------------------+ // Device Descriptors diff --git a/examples/dual/dynamic_switch/src/usb_descriptors.c b/examples/dual/dynamic_switch/src/usb_descriptors.c index ef6d795b7..c6d80e2ab 100644 --- a/examples/dual/dynamic_switch/src/usb_descriptors.c +++ b/examples/dual/dynamic_switch/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] AUDIO | MIDI | HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4)) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4020 #define USB_VID 0xCafe #define USB_BCD 0x0200 diff --git a/examples/dual/host_hid_to_device_cdc/src/usb_descriptors.c b/examples/dual/host_hid_to_device_cdc/src/usb_descriptors.c index 3efa30e20..3dc32e3f4 100644 --- a/examples/dual/host_hid_to_device_cdc/src/usb_descriptors.c +++ b/examples/dual/host_hid_to_device_cdc/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4021 #define USB_VID 0xCafe #define USB_BCD 0x0200 diff --git a/examples/dual/host_info_to_device_cdc/src/usb_descriptors.c b/examples/dual/host_info_to_device_cdc/src/usb_descriptors.c index 3efa30e20..4fa3bcbc7 100644 --- a/examples/dual/host_info_to_device_cdc/src/usb_descriptors.c +++ b/examples/dual/host_info_to_device_cdc/src/usb_descriptors.c @@ -26,15 +26,8 @@ #include "bsp/board_api.h" #include "tusb.h" -/* A combination of interfaces must have a unique product id, since PC will save device driver after the first plug. - * Same VID/PID with different interface e.g MSC (first), then CDC (later) will possibly cause system error on PC. - * - * Auto ProductID layout's Bitmap: - * [MSB] HID | MSC | CDC [LSB] - */ -#define PID_MAP(itf, n) ((CFG_TUD_##itf) ? (1 << (n)) : 0) -#define USB_PID (0x4000 | PID_MAP(CDC, 0) | PID_MAP(MSC, 1) | PID_MAP(HID, 2) | \ - PID_MAP(MIDI, 3) | PID_MAP(VENDOR, 4) ) +// Unique PID per example: guarantees re-enumeration on re-flash and a fresh host driver match. +#define USB_PID 0x4022 #define USB_VID 0xCafe #define USB_BCD 0x0200 diff --git a/examples/host/cdc_msc_hid/only.txt b/examples/host/cdc_msc_hid/only.txt index a2ff93be5..a2f4f273a 100644 --- a/examples/host/cdc_msc_hid/only.txt +++ b/examples/host/cdc_msc_hid/only.txt @@ -3,6 +3,7 @@ family:samd21 family:samd5x_e5x mcu:CH32V20X mcu:KINETIS_KL +mcu:KINETIS_K mcu:LPC175X_6X mcu:LPC177X_8X mcu:LPC18XX diff --git a/examples/host/device_info/only.txt b/examples/host/device_info/only.txt index 4c2cb0f35..7f30218df 100644 --- a/examples/host/device_info/only.txt +++ b/examples/host/device_info/only.txt @@ -4,6 +4,7 @@ family:samd21 family:samd5x_e5x mcu:CH32V20X mcu:KINETIS_KL +mcu:KINETIS_K mcu:LPC175X_6X mcu:LPC177X_8X mcu:LPC18XX diff --git a/hw/bsp/BoardPresets.json b/hw/bsp/BoardPresets.json index 93a8f2c32..a480efc3e 100644 --- a/hw/bsp/BoardPresets.json +++ b/hw/bsp/BoardPresets.json @@ -43,6 +43,10 @@ "inherits": "default" }, { + "name": "apm32f072_dev_board", + "inherits": "default" + }, + { "name": "arduino_nano33_ble", "inherits": "default" }, @@ -511,6 +515,10 @@ "inherits": "default" }, { + "name": "py32f071_dev_board", + "inherits": "default" + }, + { "name": "pybadge", "inherits": "default" }, @@ -1008,6 +1016,11 @@ "configurePreset": "apard32690" }, { + "name": "apm32f072_dev_board", + "description": "Build preset for the apm32f072_dev_board board", + "configurePreset": "apm32f072_dev_board" + }, + { "name": "arduino_nano33_ble", "description": "Build preset for the arduino_nano33_ble board", "configurePreset": "arduino_nano33_ble" @@ -1638,6 +1651,11 @@ "configurePreset": "portenta_c33" }, { + "name": "py32f071_dev_board", + "description": "Build preset for the py32f071_dev_board board", + "configurePreset": "py32f071_dev_board" + }, + { "name": "pybadge", "description": "Build preset for the pybadge board", "configurePreset": "pybadge" @@ -2258,6 +2276,19 @@ ] }, { + "name": "apm32f072_dev_board", + "steps": [ + { + "type": "configure", + "name": "apm32f072_dev_board" + }, + { + "type": "build", + "name": "apm32f072_dev_board" + } + ] + }, + { "name": "arduino_nano33_ble", "steps": [ { @@ -3896,6 +3927,19 @@ ] }, { + "name": "py32f071_dev_board", + "steps": [ + { + "type": "configure", + "name": "py32f071_dev_board" + }, + { + "type": "build", + "name": "py32f071_dev_board" + } + ] + }, + { "name": "pybadge", "steps": [ { diff --git a/hw/bsp/apm32f0xx/FreeRTOSConfig/FreeRTOSConfig.h b/hw/bsp/apm32f0xx/FreeRTOSConfig/FreeRTOSConfig.h new file mode 100644 index 000000000..b8d555ebe --- /dev/null +++ b/hw/bsp/apm32f0xx/FreeRTOSConfig/FreeRTOSConfig.h @@ -0,0 +1,108 @@ +/* + * The MIT License (MIT) + * + * Copyright (c) 2026, Ha Thach (tinyusb.org) + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + * This file is part of the TinyUSB stack. + */ + +#ifndef FREERTOS_CONFIG_H_ +#define FREERTOS_CONFIG_H_ + +#ifndef __IASMARM__ + #include "apm32f0xx.h" +#endif + +#define configENABLE_MPU 0 +#define configENABLE_FPU 0 +#define configENABLE_TRUSTZONE 0 +#define configMINIMAL_SECURE_STACK_SIZE 1024 + +#define configUSE_PREEMPTION 1 +#define configUSE_PORT_OPTIMISED_TASK_SELECTION 0 +#define configCPU_CLOCK_HZ SystemCoreClock +#define configTICK_RATE_HZ 1000 +#define configMAX_PRIORITIES 5 +#define configMINIMAL_STACK_SIZE 128 +#define configTOTAL_HEAP_SIZE ( configSUPPORT_DYNAMIC_ALLOCATION * 4 * 1024 ) +#define configMAX_TASK_NAME_LEN 16 +#define configUSE_16_BIT_TICKS 0 +#define configIDLE_SHOULD_YIELD 1 +#define configUSE_MUTEXES 1 +#define configUSE_RECURSIVE_MUTEXES 1 +#define configUSE_COUNTING_SEMAPHORES 1 +#define configQUEUE_REGISTRY_SIZE 4 +#define configUSE_QUEUE_SETS 0 +#define configUSE_TIME_SLICING 0 +#define configUSE_NEWLIB_REENTRANT 0 +#define configENABLE_BACKWARD_COMPATIBILITY 1 +#define configSTACK_ALLOCATION_FROM_SEPARATE_HEAP 0 + +#define configSUPPORT_STATIC_ALLOCATION 1 +#define configSUPPORT_DYNAMIC_ALLOCATION 0 + +#define configUSE_IDLE_HOOK 0 +#define configUSE_TICK_HOOK 0 +#define configUSE_MALLOC_FAILED_HOOK 0 +#define configCHECK_FOR_STACK_OVERFLOW 2 +#define configCHECK_HANDLER_INSTALLATION 0 + +#define configGENERATE_RUN_TIME_STATS 0 +#define configRECORD_STACK_HIGH_ADDRESS 1 +#define configUSE_TRACE_FACILITY 1 +#define configUSE_STATS_FORMATTING_FUNCTIONS 0 + +#define configUSE_CO_ROUTINES 0 +#define configMAX_CO_ROUTINE_PRIORITIES 2 + +#define configUSE_TIMERS 1 +#define configTIMER_TASK_PRIORITY ( configMAX_PRIORITIES - 2 ) +#define configTIMER_QUEUE_LENGTH 32 +#define configTIMER_TASK_STACK_DEPTH configMINIMAL_STACK_SIZE + +#define INCLUDE_vTaskPrioritySet 0 +#define INCLUDE_uxTaskPriorityGet 0 +#define INCLUDE_vTaskDelete 0 +#define INCLUDE_vTaskSuspend 1 +#define INCLUDE_xResumeFromISR 0 +#define INCLUDE_vTaskDelayUntil 1 +#define INCLUDE_vTaskDelay 1 +#define INCLUDE_xTaskGetSchedulerState 0 +#define INCLUDE_xTaskGetCurrentTaskHandle 1 +#define INCLUDE_uxTaskGetStackHighWaterMark 0 +#define INCLUDE_xTaskGetIdleTaskHandle 0 +#define INCLUDE_xTimerGetTimerDaemonTaskHandle 0 +#define INCLUDE_pcTaskGetTaskName 0 +#define INCLUDE_eTaskGetState 0 +#define INCLUDE_xEventGroupSetBitFromISR 0 +#define INCLUDE_xTimerPendFunctionCall 0 + +#define xPortPendSVHandler PendSV_Handler +#define xPortSysTickHandler SysTick_Handler +#define vPortSVCHandler SVC_Handler + +#define configPRIO_BITS __NVIC_PRIO_BITS +#define configLIBRARY_LOWEST_INTERRUPT_PRIORITY ( ( 1 << configPRIO_BITS ) - 1 ) +#define configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY 2 +#define configKERNEL_INTERRUPT_PRIORITY ( configLIBRARY_LOWEST_INTERRUPT_PRIORITY << ( 8 - configPRIO_BITS ) ) +#define configMAX_SYSCALL_INTERRUPT_PRIORITY ( configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY << ( 8 - configPRIO_BITS ) ) + +#endif diff --git a/hw/bsp/apm32f0xx/boards/apm32f072_dev_board/board.cmake b/hw/bsp/apm32f0xx/boards/apm32f072_dev_board/board.cmake new file mode 100644 index 000000000..d25f675be --- /dev/null +++ b/hw/bsp/apm32f0xx/boards/apm32f072_dev_board/board.cmake @@ -0,0 +1,13 @@ +set(MCU_VARIANT APM32F072xB) +set(MCU_LINKER_NAME APM32F07xxB) + +set(JLINK_DEVICE APM32F072RB) + +function(update_board TARGET) + target_compile_definitions(${TARGET} PUBLIC + ${MCU_VARIANT} + CFG_EXAMPLE_MSC_READONLY + CFG_EXAMPLE_MSC_DUAL_READONLY + CFG_EXAMPLE_VIDEO_READONLY + ) +endfunction() diff --git a/hw/bsp/apm32f0xx/boards/apm32f072_dev_board/board.h b/hw/bsp/apm32f0xx/boards/apm32f072_dev_board/board.h new file mode 100644 index 000000000..2c9e567e5 --- /dev/null +++ b/hw/bsp/apm32f0xx/boards/apm32f072_dev_board/board.h @@ -0,0 +1,49 @@ +/* + * The MIT License (MIT) + * + * Copyright (c) 2024, Ha Thach (tinyusb.org) + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + * This file is part of the TinyUSB stack. + */ + +/* metadata: + name: APM32F072 Dev Board + url: https://www.geehy.com +*/ + +#ifndef BOARD_H_ +#define BOARD_H_ + +#ifdef __cplusplus + extern "C" { +#endif + +// LED +#define LED_PORT GPIOC +#define LED_PIN GPIO_PIN_13 +#define LED_STATE_ON 0 +#define LED_GPIO_CLK_EN() RCM_EnableAHBPeriphClock(RCM_AHB_PERIPH_GPIOC) + +#ifdef __cplusplus + } +#endif + +#endif /* BOARD_H_ */ diff --git a/hw/bsp/apm32f0xx/boards/apm32f072_dev_board/board.mk b/hw/bsp/apm32f0xx/boards/apm32f072_dev_board/board.mk new file mode 100644 index 000000000..f78d2091f --- /dev/null +++ b/hw/bsp/apm32f0xx/boards/apm32f072_dev_board/board.mk @@ -0,0 +1,10 @@ +MCU_VARIANT = APM32F072xB +MCU_LINKER_NAME = APM32F07xxB + +JLINK_DEVICE = APM32F072RB + +CFLAGS += \ + -D${MCU_VARIANT} \ + -DCFG_EXAMPLE_MSC_READONLY \ + -DCFG_EXAMPLE_MSC_DUAL_READONLY \ + -DCFG_EXAMPLE_VIDEO_READONLY diff --git a/hw/bsp/apm32f0xx/family.c b/hw/bsp/apm32f0xx/family.c new file mode 100644 index 000000000..643490797 --- /dev/null +++ b/hw/bsp/apm32f0xx/family.c @@ -0,0 +1,146 @@ +/* + * The MIT License (MIT) + * + * Copyright (c) 2019 Ha Thach (tinyusb.org) + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + * This file is part of the TinyUSB stack. + */ + +/* metadata: + manufacturer: Geehy +*/ + +#include "apm32f0xx.h" +#include "apm32f0xx_rcm.h" +#include "apm32f0xx_gpio.h" +#include "apm32f0xx_misc.h" +#include "bsp/board_api.h" +#include "board.h" + +void USBD_IRQHandler(void); +#if CFG_TUSB_OS == OPT_OS_NONE +void SysTick_Handler(void); +void SVC_Handler(void); +void PendSV_Handler(void); +#endif +void HardFault_Handler(void); +void _init(void); + +//--------------------------------------------------------------------+ +// Forward USB interrupt events to TinyUSB IRQ Handler +//--------------------------------------------------------------------+ +void USBD_IRQHandler(void) { + tud_int_handler(0); +} + +//--------------------------------------------------------------------+ +// Board Init +//--------------------------------------------------------------------+ +void board_init(void) { + // Configure HSE and PLL for a 48 MHz system clock + SystemClockConfig(); + + // Route the 48 MHz PLL clock to USB + RCM_ConfigUSBCLK(RCM_USBCLK_PLLCLK); + + // Enable USB peripheral clock + RCM_EnableAPB1PeriphClock(RCM_APB1_PERIPH_USB); + + // SysTick 1ms tick + SysTick_Config(SystemCoreClock / 1000); + + // LED + LED_GPIO_CLK_EN(); + GPIO_Config_T gpio_config; + GPIO_ConfigStructInit(&gpio_config); + gpio_config.pin = LED_PIN; + gpio_config.mode = GPIO_MODE_OUT; + gpio_config.outtype = GPIO_OUT_TYPE_PP; + gpio_config.speed = GPIO_SPEED_50MHz; + GPIO_Config(LED_PORT, &gpio_config); + + board_led_write(false); +} + +//--------------------------------------------------------------------+ +// Board porting API +//--------------------------------------------------------------------+ +void board_led_write(bool state) { + if (state ^ (!LED_STATE_ON)) { + GPIO_SetBit(LED_PORT, LED_PIN); + } else { + GPIO_ClearBit(LED_PORT, LED_PIN); + } +} + +uint32_t board_button_read(void) { + return 0; +} + +size_t board_get_unique_id(uint8_t id[], size_t max_len) { + (void) max_len; + volatile uint32_t *apm32_uuid = ((volatile uint32_t *) 0x1FFFF7AC); + uint32_t *id32 = (uint32_t *) (uintptr_t) id; + uint8_t const len = 12; + + id32[0] = apm32_uuid[0]; + id32[1] = apm32_uuid[1]; + id32[2] = apm32_uuid[2]; + + return len; +} + +int board_uart_read(uint8_t *buf, int len) { + (void) buf; + (void) len; + return 0; +} + +int board_uart_write(void const *buf, int len) { + (void) buf; + (void) len; + return 0; +} + +#if CFG_TUSB_OS == OPT_OS_NONE +volatile uint32_t system_ticks = 0; + +void SysTick_Handler(void) { + system_ticks++; +} + +uint32_t tusb_time_millis_api(void) { + return system_ticks; +} + +void SVC_Handler(void) { +} + +void PendSV_Handler(void) { +} +#endif + +void HardFault_Handler(void) { + __asm("BKPT #0\n"); +} + +void _init(void) { +} diff --git a/hw/bsp/apm32f0xx/family.cmake b/hw/bsp/apm32f0xx/family.cmake new file mode 100644 index 000000000..199ba7cb5 --- /dev/null +++ b/hw/bsp/apm32f0xx/family.cmake @@ -0,0 +1,83 @@ +include_guard() + +set(APM32_FAMILY apm32f0xx) +set(APM32_SDK ${TOP}/hw/mcu/geehy/APM32F0xx_SDK/Libraries) + +# include board specific +include(${CMAKE_CURRENT_LIST_DIR}/boards/${BOARD}/board.cmake) + +# toolchain set up +set(CMAKE_SYSTEM_CPU cortex-m0plus CACHE INTERNAL "System Processor") +set(CMAKE_TOOLCHAIN_FILE ${TOP}/examples/build_system/cmake/toolchain/arm_${TOOLCHAIN}.cmake) + +set(FAMILY_MCUS APM32F0XX CACHE INTERNAL "") + +#------------------------------------ +# Startup & Linker script +#------------------------------------ +set(STARTUP_FILE_GNU ${APM32_SDK}/Device/Geehy/APM32F0xx/Source/gcc/startup_apm32f072.S) +set(STARTUP_FILE_Clang ${STARTUP_FILE_GNU}) +if (NOT DEFINED LD_FILE_GNU) +set(LD_FILE_GNU ${APM32_SDK}/Device/Geehy/APM32F0xx/Source/gcc/gcc_${MCU_LINKER_NAME}.ld) +endif () +set(LD_FILE_Clang ${LD_FILE_GNU}) + +#------------------------------------ +# BOARD_TARGET +#------------------------------------ +function(family_add_board BOARD_TARGET) + add_library(${BOARD_TARGET} STATIC + ${APM32_SDK}/Device/Geehy/APM32F0xx/Source/system_apm32f0xx.c + ${APM32_SDK}/APM32F0xx_StdPeriphDriver/src/apm32f0xx_gpio.c + ${APM32_SDK}/APM32F0xx_StdPeriphDriver/src/apm32f0xx_misc.c + ${APM32_SDK}/APM32F0xx_StdPeriphDriver/src/apm32f0xx_rcm.c + ) + target_include_directories(${BOARD_TARGET} PUBLIC + ${CMAKE_CURRENT_FUNCTION_LIST_DIR} + ${APM32_SDK}/CMSIS/Include + ${APM32_SDK}/Device/Geehy/APM32F0xx/Include + ${APM32_SDK}/APM32F0xx_StdPeriphDriver/inc + ) + + update_board(${BOARD_TARGET}) +endfunction() + +#------------------------------------ +# Functions +#------------------------------------ +function(family_configure_example TARGET RTOS) + family_configure_common(${TARGET} ${RTOS}) + family_add_tinyusb(${TARGET} OPT_MCU_APM32F0XX) + + target_sources(${TARGET} PUBLIC + ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/family.c + ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/../board.c + ${TOP}/src/portable/st/stm32_fsdev/dcd_stm32_fsdev.c + ${TOP}/src/portable/st/stm32_fsdev/fsdev_common.c + ${STARTUP_FILE_${CMAKE_C_COMPILER_ID}} + ) + target_include_directories(${TARGET} PUBLIC + ${CMAKE_CURRENT_FUNCTION_LIST_DIR} + ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/../../ + ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/boards/${BOARD} + ) + + if (CMAKE_C_COMPILER_ID STREQUAL "GNU") + target_link_options(${TARGET} PUBLIC + "LINKER:--script=${LD_FILE_GNU}" + -nostartfiles + --specs=nosys.specs --specs=nano.specs + ) + elseif (CMAKE_C_COMPILER_ID STREQUAL "Clang") + target_link_options(${TARGET} PUBLIC + "LINKER:--script=${LD_FILE_Clang}" + ) + endif () + set_source_files_properties(${STARTUP_FILE_${CMAKE_C_COMPILER_ID}} PROPERTIES + SKIP_LINTING ON + COMPILE_OPTIONS -w) + + # Flashing + family_add_bin_hex(${TARGET}) + family_flash_jlink(${TARGET}) +endfunction() diff --git a/hw/bsp/apm32f0xx/family.mk b/hw/bsp/apm32f0xx/family.mk new file mode 100644 index 000000000..7e26918c2 --- /dev/null +++ b/hw/bsp/apm32f0xx/family.mk @@ -0,0 +1,38 @@ +APM32_FAMILY = apm32f0xx +APM32_SDK = hw/mcu/geehy/APM32F0xx_SDK/Libraries + +include $(TOP)/$(BOARD_PATH)/board.mk + +CPU_CORE ?= cortex-m0plus + +CFLAGS += \ + -flto + +CFLAGS += \ + -DCFG_TUSB_MCU=OPT_MCU_APM32F0XX + +LDFLAGS += \ + -flto --specs=nosys.specs -nostdlib -nostartfiles + +SRC_C += \ + src/portable/st/stm32_fsdev/dcd_stm32_fsdev.c \ + src/portable/st/stm32_fsdev/fsdev_common.c \ + $(APM32_SDK)/APM32F0xx_StdPeriphDriver/src/apm32f0xx_gpio.c \ + $(APM32_SDK)/APM32F0xx_StdPeriphDriver/src/apm32f0xx_misc.c \ + $(APM32_SDK)/APM32F0xx_StdPeriphDriver/src/apm32f0xx_rcm.c \ + $(APM32_SDK)/Device/Geehy/APM32F0xx/Source/system_apm32f0xx.c + +INC += \ + $(TOP)/$(BOARD_PATH) \ + $(TOP)/$(APM32_SDK)/APM32F0xx_StdPeriphDriver/inc \ + $(TOP)/$(APM32_SDK)/CMSIS/Include \ + $(TOP)/$(APM32_SDK)/Device/Geehy/APM32F0xx/Include + +SRC_S += $(APM32_SDK)/Device/Geehy/APM32F0xx/Source/gcc/startup_apm32f072.S + +LD_FILE ?= $(APM32_SDK)/Device/Geehy/APM32F0xx/Source/gcc/gcc_${MCU_LINKER_NAME}.ld + +# For freeRTOS port source +FREERTOS_PORTABLE_SRC = $(FREERTOS_PORTABLE_PATH)/ARM_CM0 + +flash: flash-jlink diff --git a/hw/bsp/ch32v10x/family.cmake b/hw/bsp/ch32v10x/family.cmake index fb9ccb3a3..287b8c2ff 100644 --- a/hw/bsp/ch32v10x/family.cmake +++ b/hw/bsp/ch32v10x/family.cmake @@ -93,6 +93,6 @@ function(family_configure_example TARGET RTOS) # Flashing family_add_bin_hex(${TARGET}) - family_flash_openocd_wch(${TARGET}) + family_flash_openocd(${TARGET}) #family_flash_uf2(${TARGET} ${UF2_FAMILY_ID}) endfunction() diff --git a/hw/bsp/ch32v10x/family.mk b/hw/bsp/ch32v10x/family.mk index fb699b0bb..443509699 100644 --- a/hw/bsp/ch32v10x/family.mk +++ b/hw/bsp/ch32v10x/family.mk @@ -49,5 +49,5 @@ INC += \ FREERTOS_PORTABLE_SRC = $(FREERTOS_PORTABLE_PATH)/RISC-V -OPENOCD_WCH_OPTION=-f $(TOP)/$(FAMILY_PATH)/wch-riscv.cfg +OPENOCD_OPTION=-f $(TOP)/$(FAMILY_PATH)/wch-riscv.cfg flash: flash-openocd-wch diff --git a/hw/bsp/ch32v20x/family.c b/hw/bsp/ch32v20x/family.c index 76024cfde..8875e4faa 100644 --- a/hw/bsp/ch32v20x/family.c +++ b/hw/bsp/ch32v20x/family.c @@ -27,25 +27,25 @@ manufacturer: WCH * - CFG_TUD_WCH_USBIP_USBFS */ -// Port0: USBD (fsdev) -__attribute__((interrupt)) __attribute__((used)) void USB_LP_CAN1_RX0_IRQHandler(void) { - #if CFG_TUD_WCH_USBIP_FSDEV - tud_int_handler(0); - #endif -} - -__attribute__((interrupt)) __attribute__((used)) void USB_HP_CAN1_TX_IRQHandler(void) { - #if CFG_TUD_WCH_USBIP_FSDEV - tud_int_handler(0); - #endif - -} - -__attribute__((interrupt)) __attribute__((used)) void USBWakeUp_IRQHandler(void) { - #if CFG_TUD_WCH_USBIP_FSDEV - tud_int_handler(0); - #endif -} +// Port0: USBD (fsdev). The USBD raises three IRQ lines (LP/HP/WakeUp) that all funnel into the +// non-reentrant tud_int_handler and can nest (HP preempts LP) with QingKe HWSTK enabled. The +// mainline toolchain's plain __attribute__((interrupt)) emits a software prologue that fights the +// hardware context stack and corrupts the return on nesting. Emit naked handlers that rely on +// HWSTK for context save/restore (equivalent to WCH's "WCH-Interrupt-fast"), which nests safely. +#if CFG_TUD_ENABLED && CFG_TUD_WCH_USBIP_FSDEV + // The `call dcd_int_handler` below lives inside naked asm where LTO cannot see it; without a + // compiler-visible reference, -flto builds (make) internalize/drop the symbol and the link fails. + TU_ATTR_USED static void (*const fsdev_isr_keep)(uint8_t) = dcd_int_handler; + #define FSDEV_NAKED_ISR(name) \ + __attribute__((naked)) __attribute__((used)) void name(void) { \ + __asm volatile("li a0, 0\n\t call dcd_int_handler\n\t mret"); } +#else + #define FSDEV_NAKED_ISR(name) \ + __attribute__((naked)) __attribute__((used)) void name(void) { __asm volatile("mret"); } +#endif +FSDEV_NAKED_ISR(USB_LP_CAN1_RX0_IRQHandler) +FSDEV_NAKED_ISR(USB_HP_CAN1_TX_IRQHandler) +FSDEV_NAKED_ISR(USBWakeUp_IRQHandler) // Port1: USBFS __attribute__((interrupt)) __attribute__((used)) void USBHD_IRQHandler(void) { diff --git a/hw/bsp/ch32v20x/family.cmake b/hw/bsp/ch32v20x/family.cmake index 785f5ee35..a27ff021e 100644 --- a/hw/bsp/ch32v20x/family.cmake +++ b/hw/bsp/ch32v20x/family.cmake @@ -125,7 +125,7 @@ function(family_configure_example TARGET RTOS) # Flashing family_add_bin_hex(${TARGET}) - family_flash_openocd_wch(${TARGET}) + family_flash_openocd(${TARGET}) family_flash_wlink_rs(${TARGET}) #family_flash_uf2(${TARGET} ${UF2_FAMILY_ID}) endfunction() diff --git a/hw/bsp/ch32v20x/family.mk b/hw/bsp/ch32v20x/family.mk index 1d059bcba..1889c4e26 100644 --- a/hw/bsp/ch32v20x/family.mk +++ b/hw/bsp/ch32v20x/family.mk @@ -63,6 +63,6 @@ INC += \ FREERTOS_PORTABLE_SRC = $(FREERTOS_PORTABLE_PATH)/RISC-V -OPENOCD_WCH_OPTION=-f $(TOP)/$(FAMILY_PATH)/wch-riscv.cfg +OPENOCD_OPTION=-f $(TOP)/$(FAMILY_PATH)/wch-riscv.cfg flash: flash-wlink-rs #flash: flash-openocd-wch diff --git a/hw/bsp/ch32v30x/family.c b/hw/bsp/ch32v30x/family.c index aee4e7d4f..02c3c7d44 100644 --- a/hw/bsp/ch32v30x/family.c +++ b/hw/bsp/ch32v30x/family.c @@ -29,6 +29,7 @@ */ #include "stdio.h" +#include <string.h> // https://github.com/openwch/ch32v307/pull/90 // https://github.com/openwch/ch32v20x/pull/12 @@ -166,6 +167,14 @@ uint32_t board_button_read(void) { #endif } +size_t board_get_unique_id(uint8_t id[], size_t max_len) { + volatile uint32_t* ch32_uuid = ((volatile uint32_t*) 0x1FFFF7E8UL); // ESIG unique ID + uint32_t uid[3] = { ch32_uuid[0], ch32_uuid[1], ch32_uuid[2] }; + const size_t len = max_len < sizeof(uid) ? max_len : sizeof(uid); + memcpy(id, uid, len); // byte copy: id[] need not be 4-byte aligned + return len; +} + int board_uart_read(uint8_t* buf, int len) { (void) buf; (void) len; diff --git a/hw/bsp/ch32v30x/family.cmake b/hw/bsp/ch32v30x/family.cmake index b974bd5e7..e33e4b85d 100644 --- a/hw/bsp/ch32v30x/family.cmake +++ b/hw/bsp/ch32v30x/family.cmake @@ -115,6 +115,6 @@ function(family_configure_example TARGET RTOS) # Flashing family_add_bin_hex(${TARGET}) - family_flash_openocd_wch(${TARGET}) + family_flash_openocd(${TARGET}) family_flash_wlink_rs(${TARGET}) endfunction() diff --git a/hw/bsp/ch32v30x/family.mk b/hw/bsp/ch32v30x/family.mk index 5ccdea8ae..59778ec54 100644 --- a/hw/bsp/ch32v30x/family.mk +++ b/hw/bsp/ch32v30x/family.mk @@ -62,5 +62,5 @@ LD_FILE ?= $(FAMILY_PATH)/linker/ch32v30x.ld # For freeRTOS port source FREERTOS_PORTABLE_SRC = $(FREERTOS_PORTABLE_PATH)/RISC-V -OPENOCD_WCH_OPTION=-f $(TOP)/$(FAMILY_PATH)/wch-riscv.cfg +OPENOCD_OPTION=-f $(TOP)/$(FAMILY_PATH)/wch-riscv.cfg flash: flash-openocd-wch diff --git a/hw/bsp/ch583/family.cmake b/hw/bsp/ch583/family.cmake index a379298e5..f4b874e4e 100644 --- a/hw/bsp/ch583/family.cmake +++ b/hw/bsp/ch583/family.cmake @@ -104,5 +104,5 @@ function(family_configure_example TARGET RTOS) # Flashing family_add_bin_hex(${TARGET}) - family_flash_openocd_wch(${TARGET}) + family_flash_openocd(${TARGET}) endfunction() diff --git a/hw/bsp/ch583/family.mk b/hw/bsp/ch583/family.mk index 98d0f9337..3444c4811 100644 --- a/hw/bsp/ch583/family.mk +++ b/hw/bsp/ch583/family.mk @@ -52,7 +52,7 @@ INC += \ LD_FILE ?= $(FAMILY_PATH)/linker/ch582.ld -OPENOCD_WCH_OPTION=-f $(TOP)/$(FAMILY_PATH)/wch-riscv.cfg +OPENOCD_OPTION=-f $(TOP)/$(FAMILY_PATH)/wch-riscv.cfg flash: flash-openocd-wch # For freeRTOS port source diff --git a/hw/bsp/ch583/wch-riscv.cfg b/hw/bsp/ch583/wch-riscv.cfg index 64d595d8e..aa35aa9c5 100644 --- a/hw/bsp/ch583/wch-riscv.cfg +++ b/hw/bsp/ch583/wch-riscv.cfg @@ -9,7 +9,7 @@ sdi newtap $_CHIPNAME cpu -irlen 5 -expected-id 0x00001 set _TARGETNAME $_CHIPNAME.cpu target create $_TARGETNAME.0 wch_riscv -chain-position $_TARGETNAME -$_TARGETNAME.0 configure -work-area-phys 0x20000000 -work-area-size 0x8000 -work-area-backup 1 +$_TARGETNAME.0 configure -work-area-phys 0x20000000 -work-area-size 10000 -work-area-backup 1 set _FLASHNAME $_CHIPNAME.flash flash bank $_FLASHNAME wch_riscv 0x00000000 0 0 0 $_TARGETNAME.0 diff --git a/hw/bsp/family_rules.mk b/hw/bsp/family_rules.mk index ccf49dd0e..011572888 100644 --- a/hw/bsp/family_rules.mk +++ b/hw/bsp/family_rules.mk @@ -130,20 +130,18 @@ flash-pyocd: $(BUILD)/$(PROJECT).hex #pyocd reset -t $(PYOCD_TARGET) # --------------- openocd ----------------- +# OPENOCD can name another build, e.g. one of the vendor forks, though +# https://github.com/hathach/openocd branch tinyusb covers every board here +OPENOCD ?= openocd OPENOCD_OPTION ?= flash-openocd: $(BUILD)/$(PROJECT).elf - openocd $(OPENOCD_OPTION) -c "program $< verify reset exit" + $(OPENOCD) $(OPENOCD_OPTION) -c "program $< verify reset exit" # --------------- openocd-wch ----------------- -# wch-linke is not supported yet in official openOCD yet. We need to either use -# 1. download openocd as part of mounriver studio http://www.mounriver.com/download or -# 2. compiled from https://github.com/hathach/riscv-openocd-wch or -# https://github.com/dragonlock2/miscboards/blob/main/wch/SDK/riscv-openocd.tar.xz -# with ./configure --disable-werror --enable-wlinke --enable-ch347=no -OPENOCD_WCH ?= /home/${USER}/app/riscv-openocd-wch/src/openocd -OPENOCD_WCH_OPTION ?= +# WCH parts need an openocd built with the wlinke adapter. The image is written +# without verify: WCH code flash is not readable back over the debug bus. flash-openocd-wch: $(BUILD)/$(PROJECT).elf - $(OPENOCD_WCH) $(OPENOCD_WCH_OPTION) -c init -c halt -c "flash write_image $<" -c reset -c exit + $(OPENOCD) $(OPENOCD_OPTION) -c init -c halt -c "flash write_image $<" -c reset -c exit # --------------- wlink-rs ----------------- # flash with https://github.com/ch32-rs/wlink diff --git a/hw/bsp/family_support.cmake b/hw/bsp/family_support.cmake index 1f3952205..33ceb49c2 100644 --- a/hw/bsp/family_support.cmake +++ b/hw/bsp/family_support.cmake @@ -688,7 +688,9 @@ function(family_flash_stflash TARGET) endfunction() -# Add flash openocd target +# Add flash openocd target. +# The default 'openocd' should be https://github.com/hathach/openocd (branch tinyusb): which is mainline plus +# every config the rig needs (RP2350, MAX32/MAX78, WCH) and a drop-in superset of the vendor (downstream) forks function(family_flash_openocd TARGET) if (NOT DEFINED OPENOCD) set(OPENOCD openocd) @@ -715,38 +717,20 @@ function(family_flash_openocd TARGET) #set_property(TARGET ${TARGET}-openocd PROPERTY FOLDER ${TARGET}-group) endfunction() - -# Add flash openocd-wch target -# compiled from https://github.com/hathach/riscv-openocd-wch or https://github.com/dragonlock2/miscboards/blob/main/wch/SDK/riscv-openocd.tar.xz -function(family_flash_openocd_wch TARGET) - if (NOT DEFINED OPENOCD) - set(OPENOCD $ENV{HOME}/app/riscv-openocd-wch/src/openocd) +# Add flash openocd adi (Analog Devices) target using the openocd included +# with msdk (MAXIM_PATH), otherwise the default openocd +function(family_flash_openocd_adi TARGET) + # use openocd from msdk if MAXIM_PATH is set, as cmake variable or in the + # environment. Normalize the latter since msdk can be Windows (MinGW) or Linux + if (NOT DEFINED MAXIM_PATH AND DEFINED ENV{MAXIM_PATH}) + file(TO_CMAKE_PATH "$ENV{MAXIM_PATH}" MAXIM_PATH) endif () - family_flash_openocd(${TARGET}) -endfunction() - - -# Add flash openocd adi (Analog Devices) target -# included with msdk or compiled from release branch of https://github.com/analogdevicesinc/openocd -function(family_flash_openocd_adi TARGET) - if (DEFINED MAXIM_PATH) - # use openocd from msdk with MAXIM_PATH cmake variable first if the user specified it - set(OPENOCD ${MAXIM_PATH}/Tools/OpenOCD/openocd) - set(OPENOCD_OPTION2 "-s ${MAXIM_PATH}/Tools/OpenOCD/scripts") - elseif (DEFINED ENV{MAXIM_PATH}) - # use openocd from msdk with MAXIM_PATH environment variable. Normalize - # since msdk can be Windows (MinGW) or Linux - file(TO_CMAKE_PATH "$ENV{MAXIM_PATH}" MAXIM_PATH_NORM) - set(OPENOCD ${MAXIM_PATH_NORM}/Tools/OpenOCD/openocd) - set(OPENOCD_OPTION2 "-s ${MAXIM_PATH_NORM}/Tools/OpenOCD/scripts") - else() - # compiled from source - if (NOT DEFINED OPENOCD_ADI_PATH) - set(OPENOCD_ADI_PATH $ENV{HOME}/app/openocd_adi) + if (MAXIM_PATH) + if (NOT DEFINED OPENOCD) + set(OPENOCD ${MAXIM_PATH}/Tools/OpenOCD/openocd) endif () - set(OPENOCD ${OPENOCD_ADI_PATH}/src/openocd) - set(OPENOCD_OPTION2 "-s ${OPENOCD_ADI_PATH}/tcl") + set(OPENOCD_OPTION2 "-s ${MAXIM_PATH}/Tools/OpenOCD/scripts") endif () family_flash_openocd(${TARGET}) diff --git a/hw/bsp/imxrt/boards/metro_m7_1011/ozone/metro_m7_1011.jdebug b/hw/bsp/imxrt/boards/metro_m7_1011/ozone/metro_m7_1011.jdebug index 90f9b77e5..80489b61c 100644 --- a/hw/bsp/imxrt/boards/metro_m7_1011/ozone/metro_m7_1011.jdebug +++ b/hw/bsp/imxrt/boards/metro_m7_1011/ozone/metro_m7_1011.jdebug @@ -10,6 +10,7 @@ */ void OnProjectLoad (void) { Project.SetTraceSource ("Trace Pins"); + Project.SetTracePortWidth (4); Project.SetTraceTiming (50, 50, 50, 50); Project.SetDevice ("MIMXRT1011xxx4A"); Project.SetHostIF ("USB", ""); @@ -22,7 +23,7 @@ void OnProjectLoad (void) { // timing delay for trace pins in pico seconds, default is 2 nano seconds - File.Open ("../../../../../../examples/cmake-build-metro-m7-1011-sd/device/cdc_msc/cdc_msc.elf"); + File.Open ("../../../../../../examples/cmake-build-metro_m7_1011/device/cdc_msc/cdc_msc.elf"); } /********************************************************************* diff --git a/hw/bsp/imxrt/boards/mimxrt1170_evkb/board.h b/hw/bsp/imxrt/boards/mimxrt1170_evkb/board.h index a6332d896..c041fd47b 100644 --- a/hw/bsp/imxrt/boards/mimxrt1170_evkb/board.h +++ b/hw/bsp/imxrt/boards/mimxrt1170_evkb/board.h @@ -35,6 +35,11 @@ // required since iMXRT MCUX-SDK include this file for board size #define BOARD_FLASH_SIZE (0x1000000U) +// TRACE_ETM: this board wires the 100M PHY reset (ENET_RST_B) to +// GPIO_LPSR_04; the family trace init holds it in reset (RMII lines share +// the trace pads) +#define TRACE_ETM_QUIET_ENET_PHY 1 + // LED: IOMUXC_GPIO_AD_04_GPIO9_IO03 #define LED_PORT BOARD_INITPINS_USER_LED_PERIPHERAL #define LED_PIN BOARD_INITPINS_USER_LED_CHANNEL diff --git a/hw/bsp/imxrt/boards/mimxrt1170_evkb/ozone/mimxrt1176.jdebug b/hw/bsp/imxrt/boards/mimxrt1170_evkb/ozone/mimxrt1176.jdebug new file mode 100644 index 000000000..6931b9cd1 --- /dev/null +++ b/hw/bsp/imxrt/boards/mimxrt1170_evkb/ozone/mimxrt1176.jdebug @@ -0,0 +1,69 @@ +/********************************************************************* +* +* OnProjectLoad +* +* Function description +* Project load routine. Required. +* +* Notes +* MIMXRT1170-EVKB trace requires board rework: the TRACE pads +* (GPIO_DISP_B2_02..06) are factory-wired to the ENET_1G PHY - populate +* 0-ohm R1881-R1886 to route them to the Cortex Debug+ETM connector J58 +* (EVKB Hardware User Guide 3.2). +* Firmware must be built with TRACE_ETM=1 (trace pad mux + CSTRACE clock +* + JTAG_nTRST/DMIC_DATA1 pad fix in board_init). +* +* Validated: port width 1, 0 ns sample timing, 50 MHz CSTRACE root +* (25 MHz TRACE_CLK pin, set by trace_etm_init - the rework path corrupts +* at the stock 132 MHz root) while the CM7 runs 996 MHz. A startup-burst +* trace overflow is expected and benign; width 4 fails at any timing. +* +********************************************************************** +*/ +void OnProjectLoad (void) { + // declares the CSSYS TPIU/funnel (system APB-AP, not in the CM7 ROM table) + Project.SetJLinkScript ("./mimxrt1176_trace.JLinkScript"); + Project.SetTraceSource ("Trace Pins"); + Project.SetTracePortWidth (1); + Project.SetTraceTiming (0, 0, 0, 0); + Project.SetSWO (0); + Edit.SysVar (VAR_TRACE_CORE_CLOCK, 996000000); + Project.AddSvdFile ("$(InstallDir)/Config/CPU/Cortex-M7F.svd"); + + Project.SetDevice ("MIMXRT1176xxxA_M7"); + Project.SetHostIF ("USB", ""); + Project.SetTargetIF ("SWD"); + Project.SetTIFSpeed ("4 MHz"); + + File.Open ("../../../../../../examples/cmake-build-mimxrt1170_evkb/device/cdc_msc/cdc_msc.elf"); +} + +/********************************************************************* +* +* AfterTargetReset +* +* Function description +* JTAG_nTRST shares its pad with DMIC_DATA1, which drives it low by +* default and kills ETM trace - mux the pad to GPIO before the app runs +* (EVKB Hardware User Guide 3.2). No SP/PC init here: the app boots from +* FlexSPI NOR, so the ROM bootloader must perform it (SEGGER wiki +* "i.MXRT1176" / NXP community solution for ERR050708 boards). +* +********************************************************************** +*/ +void AfterTargetReset (void) { + Target.WriteU32 (0x40C08028, 0xA); // IOMUXC GPIO_LPSR_10 -> GPIO12_IO10 +} + +/********************************************************************* +* +* AfterTargetDownload +* +* Function description +* Intentionally empty: SP/PC init from the vector table would bypass the +* ROM bootloader's FlexSPI setup (see AfterTargetReset note). +* +********************************************************************** +*/ +void AfterTargetDownload (void) { +} diff --git a/hw/bsp/imxrt/boards/mimxrt1170_evkb/ozone/mimxrt1176_trace.JLinkScript b/hw/bsp/imxrt/boards/mimxrt1170_evkb/ozone/mimxrt1176_trace.JLinkScript new file mode 100644 index 000000000..ef39235d8 --- /dev/null +++ b/hw/bsp/imxrt/boards/mimxrt1170_evkb/ozone/mimxrt1176_trace.JLinkScript @@ -0,0 +1,11 @@ +/* RT1176 pin trace: the CSSYS TPIU and ATB funnel live on the system APB-AP + * (AP2) and are not discoverable from the CM7 ROM table - declare them, or + * J-Link aborts with "Required trace components for pin trace not found!" + * (addresses per i.MX RT1170 RM memory map: CSSYS TPIU E004_6000, CSSYS ATB + * Funnel E004_5000; same values as SEGGER's RT1176 trace example script). + */ +int ConfigTargetSettings(void) { + JLINK_ExecCommand("CORESIGHT_SetTPIUBaseAddr = 0xE0046000 ForceUnlock = 1 APIndex = 2"); + JLINK_ExecCommand("CORESIGHT_SetCSTFBaseAddr = 0xE0045000 ForceUnlock = 1 APIndex = 2"); + return 0; +} diff --git a/hw/bsp/imxrt/family.c b/hw/bsp/imxrt/family.c index 9f3297e9a..4bd7993f0 100644 --- a/hw/bsp/imxrt/family.c +++ b/hw/bsp/imxrt/family.c @@ -106,6 +106,74 @@ static void init_usb_phy(uint8_t usb_id) { usb_phy->TX = phytx; } +#ifdef TRACE_ETM +static void trace_etm_init(void) { +#if defined(CPU_MIMXRT1011DAE5A) + // Metro M7 rev A "ETM Trace" rework: 4-bit TRACE + TRACE_CLK on the added + // 2x10 header; the RT1011 has a single mux option per trace signal + IOMUXC_SetPinMux(IOMUXC_GPIO_AD_00_ARM_TRACE0, 0U); + IOMUXC_SetPinMux(IOMUXC_GPIO_13_ARM_TRACE1, 0U); + IOMUXC_SetPinMux(IOMUXC_GPIO_12_ARM_TRACE2, 0U); + IOMUXC_SetPinMux(IOMUXC_GPIO_11_ARM_TRACE3, 0U); + IOMUXC_SetPinMux(IOMUXC_GPIO_AD_02_ARM_TRACE_CLK, 0U); + // fast slew, max speed, high drive + IOMUXC_SetPinConfig(IOMUXC_GPIO_AD_00_ARM_TRACE0, 0x00F1U); + IOMUXC_SetPinConfig(IOMUXC_GPIO_13_ARM_TRACE1, 0x00F1U); + IOMUXC_SetPinConfig(IOMUXC_GPIO_12_ARM_TRACE2, 0x00F1U); + IOMUXC_SetPinConfig(IOMUXC_GPIO_11_ARM_TRACE3, 0x00F1U); + IOMUXC_SetPinConfig(IOMUXC_GPIO_AD_02_ARM_TRACE_CLK, 0x00F1U); + + // TRACE_CLK_ROOT already runs 132 MHz (PLL2/4) from BOARD_BootClockRUN, + // which leaves it gated - just ungate + CLOCK_EnableClock(kCLOCK_Trace); +#elif defined(CPU_MIMXRT1176DVMAA_cm7) + // JTAG_nTRST pad is shared with DMIC_DATA1 which drives it low by default and + // breaks ETM trace - switch the pad to GPIO (MIMXRT1170-EVKB HUG 3.2) + IOMUXC_SetPinMux(IOMUXC_GPIO_LPSR_10_GPIO12_IO10, 0U); + +#ifdef TRACE_ETM_QUIET_ENET_PHY + // Hold the 100M Ethernet PHY (RTL8201) in reset: its RMII lines are + // hardwired to the trace pads and drive against the stream at speed. The + // reset net is a BOARD property (mimxrt1170_evkb: ENET_RST_B = + // GPIO_LPSR_04), hence the board.h gate. + IOMUXC_SetPinMux(IOMUXC_GPIO_LPSR_04_GPIO12_IO04, 0U); + GPIO12->GDIR |= (1U << 4); + GPIO12->DR &= ~(1U << 4); +#endif + + // TRACE0-3 + TRACE_CLK on GPIO_DISP_B2_02..06, fast slew + high drive + IOMUXC_SetPinMux(IOMUXC_GPIO_DISP_B2_02_ARM_TRACE00, 0U); + IOMUXC_SetPinMux(IOMUXC_GPIO_DISP_B2_03_ARM_TRACE01, 0U); + IOMUXC_SetPinMux(IOMUXC_GPIO_DISP_B2_04_ARM_TRACE02, 0U); + IOMUXC_SetPinMux(IOMUXC_GPIO_DISP_B2_05_ARM_TRACE03, 0U); + IOMUXC_SetPinMux(IOMUXC_GPIO_DISP_B2_06_ARM_TRACE_CLK, 0U); + IOMUXC_SetPinConfig(IOMUXC_GPIO_DISP_B2_02_ARM_TRACE00, IOMUXC_SW_PAD_CTL_PAD_DSE_MASK); + IOMUXC_SetPinConfig(IOMUXC_GPIO_DISP_B2_03_ARM_TRACE01, IOMUXC_SW_PAD_CTL_PAD_DSE_MASK); + IOMUXC_SetPinConfig(IOMUXC_GPIO_DISP_B2_04_ARM_TRACE02, IOMUXC_SW_PAD_CTL_PAD_DSE_MASK); + IOMUXC_SetPinConfig(IOMUXC_GPIO_DISP_B2_05_ARM_TRACE03, IOMUXC_SW_PAD_CTL_PAD_DSE_MASK); + IOMUXC_SetPinConfig(IOMUXC_GPIO_DISP_B2_06_ARM_TRACE_CLK, IOMUXC_SW_PAD_CTL_PAD_DSE_MASK); + + // 100 MHz CSTRACE root (OscRc400M/4) -> 50 MHz TRACE_CLK pin (= root/2). + // With the PHY held in reset the CLK line is clean here; 133 MHz root + // (66 MHz pin) is marginal on this board, stock 132 MHz corrupts. + CLOCK_SetRootClockMux(kCLOCK_Root_Cstrace, kCLOCK_CSTRACE_ClockRoot_MuxOscRc400M); + CLOCK_SetRootClockDiv(kCLOCK_Root_Cstrace, 4); + CLOCK_EnableClock(kCLOCK_Cstrace); + + // Enable the CM7 slave port on the platform trace funnel (E004_3000): the + // debugger only programs the CSSYS funnel/TPIU it was told about, and this + // in-between funnel resets with all ports disabled, silently eating the ETM + // stream. Core-side CoreSight accesses honor the lock, hence the LAR unlock. + *(volatile uint32_t *) 0xE0043FB0 = 0xC5ACCE55; + *(volatile uint32_t *) 0xE0043000 |= 1U; +#else + #error "TRACE_ETM: no trace pin setup for this MCU variant" +#endif +} +#else + #define trace_etm_init() +#endif + void board_init(void) { // make sure the dcache is on. #if defined(__DCACHE_PRESENT) && __DCACHE_PRESENT @@ -119,10 +187,7 @@ void board_init(void) { SystemCoreClockUpdate(); BOARD_ConfigMPU(); // defined in board.h - -#ifdef TRACE_ETM - //CLOCK_EnableClock(kCLOCK_Trace); -#endif + trace_etm_init(); #if CFG_TUSB_OS == OPT_OS_NONE // 1ms tick timer diff --git a/hw/bsp/kinetis_k/family.c b/hw/bsp/kinetis_k/family.c index a5af83931..2df3313b8 100644 --- a/hw/bsp/kinetis_k/family.c +++ b/hw/bsp/kinetis_k/family.c @@ -35,10 +35,25 @@ #include "fsl_clock.h" #include "fsl_uart.h" #include "fsl_sysmpu.h" +#include "common/tusb_fifo.h" #include "board/clock_config.h" #include "board/pin_mux.h" +#ifdef UART_DEV +// RX ring buffer filled by the RDRF interrupt so board_uart_read() is non-blocking +// and does not drop bytes to UART overrun (see stm32 family for reference). +static uint8_t uart_rx_ff_buf[32]; +static tu_fifo_t uart_rx_ff; + +void UART0_RX_TX_IRQHandler(void) { + if (UART_DEV->S1 & UART_S1_RDRF_MASK) { + uint8_t byte = UART_DEV->D; // reading S1 then D clears RDRF (and any overrun) + tu_fifo_write(&uart_rx_ff, &byte); + } +} +#endif + //--------------------------------------------------------------------+ // Forward USB interrupt events to TinyUSB IRQ Handler //--------------------------------------------------------------------+ @@ -89,6 +104,9 @@ void board_init(void) { .enableRx = true }; UART_Init(UART_DEV, &uart_config, UART_CLOCK); + tu_fifo_config(&uart_rx_ff, uart_rx_ff_buf, sizeof(uart_rx_ff_buf), false); + UART_DEV->C2 |= UART_C2_RIE_MASK; // enable RX data register full interrupt + NVIC_EnableIRQ(UART0_RX_TX_IRQn); #endif // USB @@ -112,14 +130,11 @@ uint32_t board_button_read(void) { } int board_uart_read(uint8_t *buf, int len) { - (void) buf; - (void) len; #ifdef UART_DEV - // Read blocking will block until there is data -// UART_ReadBlocking(UART_DEV, buf, len); -// return len; - return 0; + return (int) tu_fifo_read_n(&uart_rx_ff, buf, (uint16_t) len); #else + (void) buf; + (void) len; return 0; #endif } @@ -144,6 +159,19 @@ int board_uart_write(void const *buf, int len) { #endif } +size_t board_get_unique_id(uint8_t id[], size_t max_len) { + (void) max_len; + // Kinetis 128-bit Unique Identification Register (SIM->UIDH/UIDMH/UIDML/UIDL) + uint32_t* id32 = (uint32_t*) (uintptr_t) id; + + id32[0] = SIM->UIDH; + id32[1] = SIM->UIDMH; + id32[2] = SIM->UIDML; + id32[3] = SIM->UIDL; + + return 16; +} + #if CFG_TUSB_OS == OPT_OS_NONE volatile uint32_t system_ticks = 0; diff --git a/hw/bsp/kinetis_k/family.cmake b/hw/bsp/kinetis_k/family.cmake index e1b5c221e..e408c0f4a 100644 --- a/hw/bsp/kinetis_k/family.cmake +++ b/hw/bsp/kinetis_k/family.cmake @@ -63,7 +63,7 @@ function(family_configure_example TARGET RTOS) ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/family.c ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/../board.c ${TOP}/src/portable/chipidea/ci_fs/dcd_ci_fs.c - ${TOP}/src/portable/nxp/khci/hcd_khci.c + ${TOP}/src/portable/chipidea/ci_fs/hcd_ci_fs.c ${STARTUP_FILE_${CMAKE_C_COMPILER_ID}} ) target_include_directories(${TARGET} PUBLIC diff --git a/hw/bsp/kinetis_k/family.mk b/hw/bsp/kinetis_k/family.mk index b1e1fb3aa..5d0e4a702 100644 --- a/hw/bsp/kinetis_k/family.mk +++ b/hw/bsp/kinetis_k/family.mk @@ -17,8 +17,8 @@ LDFLAGS += \ --specs=nosys.specs --specs=nano.specs \ SRC_C += \ - src/portable/nxp/khci/dcd_khci.c \ - src/portable/nxp/khci/hcd_khci.c \ + src/portable/chipidea/ci_fs/dcd_ci_fs.c \ + src/portable/chipidea/ci_fs/hcd_ci_fs.c \ $(MCU_DIR)/system_${MCU_VARIANT}.c \ $(MCU_DIR)/drivers/fsl_clock.c \ $(SDK_DIR)/drivers/gpio/fsl_gpio.c \ diff --git a/hw/bsp/kinetis_k32l/family.cmake b/hw/bsp/kinetis_k32l/family.cmake index 020695589..950682363 100644 --- a/hw/bsp/kinetis_k32l/family.cmake +++ b/hw/bsp/kinetis_k32l/family.cmake @@ -64,8 +64,8 @@ function(family_configure_example TARGET RTOS) target_sources(${TARGET} PUBLIC ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/family.c ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/../board.c - ${TOP}/src/portable/nxp/khci/dcd_khci.c - ${TOP}/src/portable/nxp/khci/hcd_khci.c + ${TOP}/src/portable/chipidea/ci_fs/dcd_ci_fs.c + ${TOP}/src/portable/chipidea/ci_fs/hcd_ci_fs.c ${STARTUP_FILE_${CMAKE_C_COMPILER_ID}} ) target_include_directories(${TARGET} PUBLIC diff --git a/hw/bsp/kinetis_k32l/family.mk b/hw/bsp/kinetis_k32l/family.mk index a99fb5dbe..357128aa5 100644 --- a/hw/bsp/kinetis_k32l/family.mk +++ b/hw/bsp/kinetis_k32l/family.mk @@ -13,8 +13,8 @@ LDFLAGS += \ -specs=nosys.specs -specs=nano.specs SRC_C += \ - src/portable/nxp/khci/dcd_khci.c \ - src/portable/nxp/khci/hcd_khci.c \ + src/portable/chipidea/ci_fs/dcd_ci_fs.c \ + src/portable/chipidea/ci_fs/hcd_ci_fs.c \ $(MCUX_DEVICES)/K32L/$(MCU_VARIANT)/system_$(MCU_VARIANT).c \ $(MCUX_DEVICES)/K32L/$(MCU_VARIANT)/drivers/fsl_clock.c \ $(MCUX_CORE)/drivers/gpio/fsl_gpio.c \ diff --git a/hw/bsp/kinetis_kl/family.cmake b/hw/bsp/kinetis_kl/family.cmake index 230a3057d..b74f4f8b9 100644 --- a/hw/bsp/kinetis_kl/family.cmake +++ b/hw/bsp/kinetis_kl/family.cmake @@ -62,7 +62,7 @@ function(family_configure_example TARGET RTOS) ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/family.c ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/../board.c ${TOP}/src/portable/chipidea/ci_fs/dcd_ci_fs.c - ${TOP}/src/portable/nxp/khci/hcd_khci.c + ${TOP}/src/portable/chipidea/ci_fs/hcd_ci_fs.c ${STARTUP_FILE_${CMAKE_C_COMPILER_ID}} ) target_include_directories(${TARGET} PUBLIC diff --git a/hw/bsp/kinetis_kl/family.mk b/hw/bsp/kinetis_kl/family.mk index 201ab99dc..9c780a868 100644 --- a/hw/bsp/kinetis_kl/family.mk +++ b/hw/bsp/kinetis_kl/family.mk @@ -17,8 +17,8 @@ LDFLAGS += \ -specs=nosys.specs -specs=nano.specs \ SRC_C += \ - src/portable/nxp/khci/dcd_khci.c \ - src/portable/nxp/khci/hcd_khci.c \ + src/portable/chipidea/ci_fs/dcd_ci_fs.c \ + src/portable/chipidea/ci_fs/hcd_ci_fs.c \ $(MCU_DIR)/system_$(MCU).c \ $(MCU_DIR)/drivers/fsl_clock.c \ $(SDK_DIR)/drivers/gpio/fsl_gpio.c \ diff --git a/hw/bsp/lpc11/boards/lpcxpresso11u37/board.mk b/hw/bsp/lpc11/boards/lpcxpresso11u37/board.mk index fdc17374b..718c46bbf 100644 --- a/hw/bsp/lpc11/boards/lpcxpresso11u37/board.mk +++ b/hw/bsp/lpc11/boards/lpcxpresso11u37/board.mk @@ -4,8 +4,7 @@ MCU_DRV = 11xx CFLAGS += \ -DCORE_M0 \ -DCFG_EXAMPLE_MSC_READONLY \ - -DCFG_EXAMPLE_VIDEO_READONLY \ - -DCFG_TUSB_MEM_SECTION='__attribute__((section(".data.$$RAM2")))' + -DCFG_EXAMPLE_VIDEO_READONLY # mcu driver cause following warnings CFLAGS += \ diff --git a/hw/bsp/lpc11/boards/lpcxpresso11u37/lpc11u37.ld b/hw/bsp/lpc11/boards/lpcxpresso11u37/lpc11u37.ld index 8e0a4e4c6..b7237a3ec 100644 --- a/hw/bsp/lpc11/boards/lpcxpresso11u37/lpc11u37.ld +++ b/hw/bsp/lpc11/boards/lpcxpresso11u37/lpc11u37.ld @@ -172,6 +172,22 @@ SECTIONS . = ALIGN(4) ; _end_noinit = .; } > RamLoc8 + /* Main (MSP/ISR) stack lives at the top of the USB SRAM bank: the 8K main bank is packed so + tight that only ~280 B remained above .bss, and ISR frames overflowed into the topmost task + stack (cdc_msc_freertos hard fault). Nothing else is placed in this bank in either build + system, so the stack owns all 2 KB; the ASSERT is future-proofing in case USB buffers are + ever mapped here again. + + This bank is clocked by SYSAHBCLKCTRL[27] (USBRAM enable), and the stack is used from the + first instruction of the reset handler - long before any TinyUSB or BSP code could turn a + clock on. It works because the boot ROM hands over with that bit already set. Anything that + gates the USB RAM clock to save power will hard fault at reset, not at USB init. */ + __user_stack_top = ORIGIN(RamUsb2) + LENGTH(RamUsb2); + /* Stated as an addition, not a subtraction: ld arithmetic is unsigned, so an overflowing + bank would underflow the difference into a huge positive value and pass silently. */ + ASSERT(ADDR(.noinit_RAM2) + SIZEOF(.noinit_RAM2) + 0x200 <= __user_stack_top, + "main stack headroom in RamUsb2 below 512 bytes") + PROVIDE(_pvHeapStart = DEFINED(__user_heap_base) ? __user_heap_base : .); PROVIDE(_vStackTop = DEFINED(__user_stack_top) ? __user_stack_top : __top_RamLoc8 - 0); diff --git a/hw/bsp/lpc15/family.c b/hw/bsp/lpc15/family.c index bbfee1b51..5178ad68b 100644 --- a/hw/bsp/lpc15/family.c +++ b/hw/bsp/lpc15/family.c @@ -122,6 +122,19 @@ uint32_t board_button_read(void) return Chip_GPIO_GetPinState(LPC_GPIO, BUTTON_PORT, BUTTON_PIN) ? 0 : 1; } +size_t board_get_unique_id(uint8_t id[], size_t max_len) +{ + // IAP ReadUID (cmd 58) returns status + 4 words = full 128-bit UID + unsigned int command[5] = { IAP_READ_UID_CMD, 0, 0, 0, 0 }; + unsigned int result[5]; + iap_entry(command, result); + TU_ASSERT(result[0] == IAP_CMD_SUCCESS, 0); + + size_t const len = tu_min32(max_len, 16); + memcpy(id, &result[1], len); + return len; +} + int board_uart_read(uint8_t* buf, int len) { (void) buf; (void) len; diff --git a/hw/bsp/lpc18/boards/lpcxpresso18s37/board.h b/hw/bsp/lpc18/boards/lpcxpresso18s37/board.h index 2cf4dbdf8..1be07c49e 100644 --- a/hw/bsp/lpc18/boards/lpcxpresso18s37/board.h +++ b/hw/bsp/lpc18/boards/lpcxpresso18s37/board.h @@ -76,6 +76,12 @@ static inline void board_lpc18_pinmux(void) Chip_SCU_SetPinMuxing(pinmuxing, sizeof(pinmuxing) / sizeof(PINMUX_GRP_T)); } + +// TRACE_ETM builds: no trace header is wired out on the LPCXpresso18S37 - +// provide the no-op the family init expects (see mcb1800/ea4357 for a +// board that routes the trace pins) +static inline void board_trace_pinmux(void) {} + #ifdef __cplusplus } #endif diff --git a/hw/bsp/lpc18/boards/mcb1800/board.h b/hw/bsp/lpc18/boards/mcb1800/board.h index dba7a62a3..ec7f1fa95 100644 --- a/hw/bsp/lpc18/boards/mcb1800/board.h +++ b/hw/bsp/lpc18/boards/mcb1800/board.h @@ -48,15 +48,6 @@ static inline void board_lpc18_pinmux(void) { const PINMUX_GRP_T pinmuxing[] = { - // ETM Trace - #ifdef TRACE_ETM - { 0xF, 4, SCU_MODE_FUNC2 | SCU_MODE_HIGHSPEEDSLEW_EN }, - { 0xF, 5, SCU_MODE_FUNC3 | SCU_MODE_HIGHSPEEDSLEW_EN }, - { 0xF, 6, SCU_MODE_FUNC3 | SCU_MODE_HIGHSPEEDSLEW_EN }, - { 0xF, 7, SCU_MODE_FUNC3 | SCU_MODE_HIGHSPEEDSLEW_EN }, - { 0xF, 8, SCU_MODE_FUNC3 | SCU_MODE_HIGHSPEEDSLEW_EN }, - #endif - // LEDs { 0xD, 10, (SCU_MODE_INBUFF_EN | SCU_MODE_INACT | SCU_MODE_FUNC4) }, { 0xD, 11, (SCU_MODE_INBUFF_EN | SCU_MODE_INACT | SCU_MODE_FUNC4 | SCU_MODE_PULLDOWN) }, @@ -96,6 +87,24 @@ static inline void board_lpc18_pinmux(void) { } } +#ifdef TRACE_ETM +// Must run AFTER Chip_SetupCoreClock: muxing the trace pins earlier starts +// TRACECLK at the boot clock and the mid-init frequency switch desyncs the +// trace decoder ("Unknown trace data packet"). +static inline void board_trace_pinmux(void) { + // SCU_MODE_INACT: disable the pull-up on the 60 MHz trace lines - leaving it + // enabled degrades the edges enough for intermittent decode corruption. + const PINMUX_GRP_T trace_pinmux[] = { + { 0xF, 4, SCU_MODE_INACT | SCU_MODE_FUNC2 | SCU_MODE_HIGHSPEEDSLEW_EN }, + { 0xF, 5, SCU_MODE_INACT | SCU_MODE_FUNC3 | SCU_MODE_HIGHSPEEDSLEW_EN }, + { 0xF, 6, SCU_MODE_INACT | SCU_MODE_FUNC3 | SCU_MODE_HIGHSPEEDSLEW_EN }, + { 0xF, 7, SCU_MODE_INACT | SCU_MODE_FUNC3 | SCU_MODE_HIGHSPEEDSLEW_EN }, + { 0xF, 8, SCU_MODE_INACT | SCU_MODE_FUNC3 | SCU_MODE_HIGHSPEEDSLEW_EN }, + }; + Chip_SCU_SetPinMuxing(trace_pinmux, sizeof(trace_pinmux) / sizeof(PINMUX_GRP_T)); +} +#endif + #ifdef __cplusplus } #endif diff --git a/hw/bsp/lpc18/boards/mcb1800/ozone/lpc1857.jdebug b/hw/bsp/lpc18/boards/mcb1800/ozone/lpc1857.jdebug index f94960f09..a6dcef0eb 100644 --- a/hw/bsp/lpc18/boards/mcb1800/ozone/lpc1857.jdebug +++ b/hw/bsp/lpc18/boards/mcb1800/ozone/lpc1857.jdebug @@ -10,7 +10,7 @@ */ void OnProjectLoad (void) { Project.AddSvdFile ("Cortex-M3.svd"); - Project.AddSvdFile ("../../../../../../../cmsis-svd/data/NXP/LPC18xx.svd"); + //Project.AddSvdFile ("../../../../../../../cmsis-svd/data/NXP/LPC18xx.svd"); Project.SetDevice ("LPC1857"); Project.SetHostIF ("USB", ""); @@ -20,8 +20,8 @@ void OnProjectLoad (void) { Project.SetTraceSource ("Trace Pins"); Project.SetTracePortWidth (4); - //File.Open ("../../../../../../examples/cmake-build-mcb1800/device/cdc_msc/cdc_msc.elf"); - File.Open ("../../../../../../examples/cmake-build-mcb1800/host/cdc_msc_hid/cdc_msc_hid.elf"); + File.Open ("../../../../../../examples/cmake-build-mcb1800/device/cdc_msc/cdc_msc.elf"); + //File.Open ("../../../../../../examples/cmake-build-mcb1800/host/cdc_msc_hid/cdc_msc_hid.elf"); } /********************************************************************* * diff --git a/hw/bsp/lpc18/family.c b/hw/bsp/lpc18/family.c index 8a612d9d8..58c2193fc 100644 --- a/hw/bsp/lpc18/family.c +++ b/hw/bsp/lpc18/family.c @@ -69,6 +69,7 @@ void SystemInit(void) { #ifdef TRACE_ETM // Trace clock is limited to 60MHz, limit CPU clock to 120MHz Chip_SetupCoreClock(CLKIN_CRYSTAL, 120000000UL, true); + board_trace_pinmux(); // after clock setup so TRACECLK starts at its final frequency #else // CPU clock max to 180 Mhz Chip_SetupCoreClock(CLKIN_CRYSTAL, MAX_CLOCK_FREQ, true); diff --git a/hw/bsp/lpc40/boards/ea4088_quickstart/ozone/ea4088_quickstart.jdebug b/hw/bsp/lpc40/boards/ea4088_quickstart/ozone/ea4088_quickstart.jdebug index 6aaf1076d..20de1e915 100644 --- a/hw/bsp/lpc40/boards/ea4088_quickstart/ozone/ea4088_quickstart.jdebug +++ b/hw/bsp/lpc40/boards/ea4088_quickstart/ozone/ea4088_quickstart.jdebug @@ -21,7 +21,7 @@ void OnProjectLoad (void) { Project.SetTracePortWidth (4); // User settings - File.Open ("../../../../../../examples/device/cdc_msc/cmake-build-ea4088-quickstart/cdc_msc.elf"); + File.Open ("../../../../../../examples/cmake-build-ea4088_quickstart/device/cdc_msc/cdc_msc.elf"); } /********************************************************************* diff --git a/hw/bsp/lpc40/family.c b/hw/bsp/lpc40/family.c index d8a63576b..750bd2660 100644 --- a/hw/bsp/lpc40/family.c +++ b/hw/bsp/lpc40/family.c @@ -135,6 +135,19 @@ uint32_t board_button_read(void) { return BUTTON_ACTIV_STATE == Chip_GPIO_GetPinState(LPC_GPIO, BUTTON_PORT, BUTTON_PIN); } +size_t board_get_unique_id(uint8_t id[], size_t max_len) { + // IAP ReadUID (cmd 58) returns status + 4 words = full 128-bit UID + // (lpcopen's Chip_IAP_ReadUID() only returns the first word) + unsigned int command[5] = { IAP_READ_UID_CMD, 0, 0, 0, 0 }; + unsigned int result[5]; + iap_entry(command, result); + TU_ASSERT(result[0] == IAP_CMD_SUCCESS, 0); + + size_t const len = tu_min32(max_len, 16); + memcpy(id, &result[1], len); + return len; +} + int board_uart_read(uint8_t *buf, int len) { //return UART_ReceiveByte(BOARD_UART_PORT); (void) buf; diff --git a/hw/bsp/lpc43/boards/ea4357/board.h b/hw/bsp/lpc43/boards/ea4357/board.h index fca617361..0152825f5 100644 --- a/hw/bsp/lpc43/boards/ea4357/board.h +++ b/hw/bsp/lpc43/boards/ea4357/board.h @@ -83,6 +83,22 @@ static const PINMUX_GRP_T pinmuxing[] = { // { 0, 3, SCU_MODE_INACT | SCU_MODE_INBUFF_EN | SCU_MODE_ZIF_DIS | SCU_MODE_HIGHSPEEDSLEW_EN | SCU_MODE_FUNC0 }, //}; +#ifdef TRACE_ETM +// Must run AFTER Chip_SetupCoreClock: muxing the trace pins earlier starts +// TRACECLK at the boot clock and the mid-init frequency switch desyncs the +// trace decoder. SCU_MODE_INACT keeps pull-ups off the 60 MHz lines. +static inline void board_trace_pinmux(void) { + const PINMUX_GRP_T trace_pinmux[] = { + { 0xF, 4, SCU_MODE_INACT | SCU_MODE_FUNC2 | SCU_MODE_HIGHSPEEDSLEW_EN }, + { 0xF, 5, SCU_MODE_INACT | SCU_MODE_FUNC3 | SCU_MODE_HIGHSPEEDSLEW_EN }, + { 0xF, 6, SCU_MODE_INACT | SCU_MODE_FUNC3 | SCU_MODE_HIGHSPEEDSLEW_EN }, + { 0xF, 7, SCU_MODE_INACT | SCU_MODE_FUNC3 | SCU_MODE_HIGHSPEEDSLEW_EN }, + { 0xF, 8, SCU_MODE_INACT | SCU_MODE_FUNC3 | SCU_MODE_HIGHSPEEDSLEW_EN }, + }; + Chip_SCU_SetPinMuxing(trace_pinmux, sizeof(trace_pinmux) / sizeof(PINMUX_GRP_T)); +} +#endif + #ifdef __cplusplus } #endif diff --git a/hw/bsp/lpc43/boards/lpcxpresso43s67/board.h b/hw/bsp/lpc43/boards/lpcxpresso43s67/board.h index 4427905e8..6a317b5dc 100644 --- a/hw/bsp/lpc43/boards/lpcxpresso43s67/board.h +++ b/hw/bsp/lpc43/boards/lpcxpresso43s67/board.h @@ -71,6 +71,12 @@ static const PINMUX_GRP_T pinmuxing[] = { {0x2, 5, SCU_MODE_INBUFF_EN | SCU_MODE_PULLUP | SCU_MODE_FUNC4 }, }; + +// TRACE_ETM builds: no trace header is wired out on the LPCXpresso43S67 - +// provide the no-op the family init expects (see mcb1800/ea4357 for a +// board that routes the trace pins) +static inline void board_trace_pinmux(void) {} + #ifdef __cplusplus } #endif diff --git a/hw/bsp/lpc43/family.c b/hw/bsp/lpc43/family.c index 5aff49704..7f0722a33 100644 --- a/hw/bsp/lpc43/family.c +++ b/hw/bsp/lpc43/family.c @@ -59,6 +59,30 @@ void SystemInit(void); // Invoked by startup code void SystemInit(void) { +#if defined(__ICCARM__) && !defined(DONT_RESET_ON_RESTART) + __disable_irq(); +#endif + + if (Chip_CREG_OnChipFlashIsPresent()) { + // The boot ROM configures flash for its 96 MHz clock, and debugger core + // resets can preserve it. Use safe timing before switching the M4 to 204 MHz. + Chip_CREG_SetFLASHAccess(FLASHTIM_SAFE_SETTING); + __DSB(); + __ISB(); + } + +#if defined(__ICCARM__) && !defined(DONT_RESET_ON_RESTART) + // A debugger restart resets the M4 core, but can leave LPC43 peripherals and + // pending interrupts active. Match the GCC startup sequence, which the IAR + // startup lacks, before the C runtime can reuse peripheral DMA memory. + LPC_RGU->RESET_CTRL[0] = 0x10DF1000u; + LPC_RGU->RESET_CTRL[1] = 0x01DFF7FFu; + for (uint32_t i = 0; i < 8; i++) { + NVIC->ICPR[i] = UINT32_MAX; + } + __enable_irq(); +#endif + #ifdef __USE_LPCOPEN unsigned int *pSCB_VTOR = (unsigned int *) 0xE000ED08; @@ -89,7 +113,13 @@ void SystemInit(void) // Chip_SCU_ClockPinMuxSet(pinclockmuxing[i].pinnum, pinclockmuxing[i].modefunc); // } +#ifdef TRACE_ETM + // Trace clock is limited to 60MHz, limit CPU clock to 120MHz + Chip_SetupCoreClock(CLKIN_CRYSTAL, 120000000UL, true); + board_trace_pinmux(); // after clock setup so TRACECLK starts at its final frequency +#else Chip_SetupXtalClocking(); +#endif } void board_init(void) diff --git a/hw/bsp/lpc55/boards/lpcxpresso55s28/board.cmake b/hw/bsp/lpc55/boards/lpcxpresso55s28/board.cmake index b3d6ec722..d7992eec6 100644 --- a/hw/bsp/lpc55/boards/lpcxpresso55s28/board.cmake +++ b/hw/bsp/lpc55/boards/lpcxpresso55s28/board.cmake @@ -8,10 +8,6 @@ set(JLINK_OPTION "-USB 000727031389") set(PYOCD_TARGET LPC55S28) set(NXPLINK_DEVICE LPC55S28:LPCXpresso55S28) -# device fullspeed, host highspeed -set(RHPORT_DEVICE 0) -set(RHPORT_HOST 1) - function(update_board TARGET) target_compile_definitions(${TARGET} PUBLIC CPU_LPC55S28JBD100 diff --git a/hw/bsp/lpc55/boards/lpcxpresso55s28/board.mk b/hw/bsp/lpc55/boards/lpcxpresso55s28/board.mk index db2e11fd7..aecb5a100 100644 --- a/hw/bsp/lpc55/boards/lpcxpresso55s28/board.mk +++ b/hw/bsp/lpc55/boards/lpcxpresso55s28/board.mk @@ -2,9 +2,9 @@ MCU_VARIANT = LPC55S28 MCU_CORE = LPC55S28 MCU_DRIVER_VARIANT = LPC55S69 -# device fullspeed, host highspeed -RHPORT_DEVICE ?= 0 -RHPORT_HOST ?= 1 +# device highspeed, host fullspeed +RHPORT_DEVICE ?= 1 +RHPORT_HOST ?= 0 CFLAGS += -DCPU_LPC55S28JBD100 diff --git a/hw/bsp/lpc55/family.c b/hw/bsp/lpc55/family.c index e021caf35..11bf86827 100644 --- a/hw/bsp/lpc55/family.c +++ b/hw/bsp/lpc55/family.c @@ -170,6 +170,14 @@ uint32_t board_button_read(void) { return BUTTON_STATE_ACTIVE == GPIO_PinRead(GPIO, BUTTON_PORT, BUTTON_PIN); } +size_t board_get_unique_id(uint8_t id[], size_t max_len) { + // 128-bit UUID in the flash PFR region at 0x0009FC70 (UM11126 rev 2.1 section 48.8) + const uint8_t* uuid = (const uint8_t*) 0x0009FC70; + size_t const len = tu_min32(max_len, 16); + memcpy(id, uuid, len); + return len; +} + int board_uart_read(uint8_t* buf, int len) { (void) buf; (void) len; diff --git a/hw/bsp/lpc55/family.mk b/hw/bsp/lpc55/family.mk index a9b6f6af1..a640cc793 100644 --- a/hw/bsp/lpc55/family.mk +++ b/hw/bsp/lpc55/family.mk @@ -36,6 +36,8 @@ ifeq ($(RHPORT_HOST), 1) SRC_C += $(TOP)/src/portable/nxp/lpc_ip3516/hcd_lpc_ip3516.c else CFLAGS += -DBOARD_TUH_MAX_SPEED=OPT_MODE_FULL_SPEED + # host on port 0 uses the OHCI controller (mirrors family.cmake) + SRC_C += $(TOP)/src/portable/ohci/ohci.c endif # mcu driver cause following warnings diff --git a/hw/bsp/mcx/family.cmake b/hw/bsp/mcx/family.cmake index 89e2aadf2..b2b4fd45b 100644 --- a/hw/bsp/mcx/family.cmake +++ b/hw/bsp/mcx/family.cmake @@ -94,10 +94,19 @@ function(family_configure_example TARGET RTOS) family_add_tinyusb(${TARGET} OPT_MCU_MCXA15) endif() + # PORT is set per board (board.cmake), so pick the driver at configure time. Spelled out + # rather than $<IF:${PORT},...> so the port path stays greppable: test/hil/helper/hil_select.py + # maps a portable-driver change to the families whose build file names that directory. + if (PORT) + set(PORT_SRC ${TOP}/src/portable/chipidea/ci_hs/dcd_ci_hs.c) + else () + set(PORT_SRC ${TOP}/src/portable/chipidea/ci_fs/dcd_ci_fs.c) + endif () + target_sources(${TARGET} PUBLIC ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/family.c ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/../board.c - ${TOP}/src/portable/chipidea/$<IF:${PORT},ci_hs/dcd_ci_hs.c,ci_fs/dcd_ci_fs.c> + ${PORT_SRC} ${STARTUP_FILE_${CMAKE_C_COMPILER_ID}} ) target_include_directories(${TARGET} PUBLIC diff --git a/hw/bsp/nrf/boards/nrf52840dk/ozone/nrf52840.jdebug b/hw/bsp/nrf/boards/nrf52840dk/ozone/nrf52840.jdebug index fa7ab9e23..40f28baa9 100644 --- a/hw/bsp/nrf/boards/nrf52840dk/ozone/nrf52840.jdebug +++ b/hw/bsp/nrf/boards/nrf52840dk/ozone/nrf52840.jdebug @@ -21,7 +21,7 @@ void OnProjectLoad (void) { Project.SetTracePortWidth (4); // User settings - File.Open ("../../../../../../examples/device/cdc_msc/cmake-build-pca10056/cdc_msc.elf"); + File.Open ("../../../../../../examples/cmake-build-nrf52840dk/device/cdc_msc/cdc_msc.elf"); } /********************************************************************* diff --git a/hw/bsp/nrf/boards/nrf5340dk/ozone/nrf5340.jdebug b/hw/bsp/nrf/boards/nrf5340dk/ozone/nrf5340.jdebug index 4ad0376a4..34b1841b0 100644 --- a/hw/bsp/nrf/boards/nrf5340dk/ozone/nrf5340.jdebug +++ b/hw/bsp/nrf/boards/nrf5340dk/ozone/nrf5340.jdebug @@ -29,9 +29,13 @@ void OnProjectLoad (void) { Project.SetTraceSource ("Trace Pins"); Project.SetTracePortWidth (4); + // +3 ns sample point: the DK's analog-switch/SWO stubs make TD=0 marginal + // (Ozone sends TraceSampleAdjust TD=0 when no timing is set); solid across + // the +2..+4 ns band with SB27/SB28 cut, SB57 (SWO) left intact + Project.SetTraceTiming (3000, 3000, 3000, 3000); // User settings - File.Open ("../../../../../../examples/device/cdc_msc/cmake-build-pca10095/cdc_msc.elf"); + File.Open ("../../../../../../examples/cmake-build-nrf5340dk/device/cdc_msc/cdc_msc.elf"); } /********************************************************************* diff --git a/hw/bsp/nrf/family.c b/hw/bsp/nrf/family.c index 31a6bac9e..3cef4dac3 100644 --- a/hw/bsp/nrf/family.c +++ b/hw/bsp/nrf/family.c @@ -165,6 +165,12 @@ static nrfx_gpiote_t _gpiote = NRFX_GPIOTE_INSTANCE(0); // //--------------------------------------------------------------------+ void board_init(void) { +#if defined(TRACE_ETM) && defined(NRF5340_XXAA) + // SystemInit (ENABLE_TRACE) sets the TAD trace port to 64 MHz, which is + // marginal through the DK's switch stubs - 16 MHz streams reliably (matches the validated nrf52840dk) and is + // ample bandwidth for the 64 MHz core + NRF_TAD_S->TRACEPORTSPEED = TAD_TRACEPORTSPEED_TRACEPORTSPEED_16MHz; +#endif #if !defined(NRF54H20_XXAA) && !defined(NRF54LM20A_ENGA_XXAA) // stop LF clock just in case we jump from application without reset NRF_CLOCK->TASKS_LFCLKSTOP = 1UL; diff --git a/hw/bsp/py32f0/FreeRTOSConfig/FreeRTOSConfig.h b/hw/bsp/py32f0/FreeRTOSConfig/FreeRTOSConfig.h new file mode 100644 index 000000000..6b3ae0cd5 --- /dev/null +++ b/hw/bsp/py32f0/FreeRTOSConfig/FreeRTOSConfig.h @@ -0,0 +1,108 @@ +/* + * The MIT License (MIT) + * + * Copyright (c) 2026, Ha Thach (tinyusb.org) + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + * This file is part of the TinyUSB stack. + */ + +#ifndef FREERTOS_CONFIG_H_ +#define FREERTOS_CONFIG_H_ + +#ifndef __IASMARM__ + #include "py32f0xx.h" +#endif + +#define configENABLE_MPU 0 +#define configENABLE_FPU 0 +#define configENABLE_TRUSTZONE 0 +#define configMINIMAL_SECURE_STACK_SIZE 1024 + +#define configUSE_PREEMPTION 1 +#define configUSE_PORT_OPTIMISED_TASK_SELECTION 0 +#define configCPU_CLOCK_HZ SystemCoreClock +#define configTICK_RATE_HZ 1000 +#define configMAX_PRIORITIES 5 +#define configMINIMAL_STACK_SIZE 128 +#define configTOTAL_HEAP_SIZE ( configSUPPORT_DYNAMIC_ALLOCATION * 4 * 1024 ) +#define configMAX_TASK_NAME_LEN 16 +#define configUSE_16_BIT_TICKS 0 +#define configIDLE_SHOULD_YIELD 1 +#define configUSE_MUTEXES 1 +#define configUSE_RECURSIVE_MUTEXES 1 +#define configUSE_COUNTING_SEMAPHORES 1 +#define configQUEUE_REGISTRY_SIZE 4 +#define configUSE_QUEUE_SETS 0 +#define configUSE_TIME_SLICING 0 +#define configUSE_NEWLIB_REENTRANT 0 +#define configENABLE_BACKWARD_COMPATIBILITY 1 +#define configSTACK_ALLOCATION_FROM_SEPARATE_HEAP 0 + +#define configSUPPORT_STATIC_ALLOCATION 1 +#define configSUPPORT_DYNAMIC_ALLOCATION 0 + +#define configUSE_IDLE_HOOK 0 +#define configUSE_TICK_HOOK 0 +#define configUSE_MALLOC_FAILED_HOOK 0 +#define configCHECK_FOR_STACK_OVERFLOW 2 +#define configCHECK_HANDLER_INSTALLATION 0 + +#define configGENERATE_RUN_TIME_STATS 0 +#define configRECORD_STACK_HIGH_ADDRESS 1 +#define configUSE_TRACE_FACILITY 1 +#define configUSE_STATS_FORMATTING_FUNCTIONS 0 + +#define configUSE_CO_ROUTINES 0 +#define configMAX_CO_ROUTINE_PRIORITIES 2 + +#define configUSE_TIMERS 1 +#define configTIMER_TASK_PRIORITY ( configMAX_PRIORITIES - 2 ) +#define configTIMER_QUEUE_LENGTH 32 +#define configTIMER_TASK_STACK_DEPTH configMINIMAL_STACK_SIZE + +#define INCLUDE_vTaskPrioritySet 0 +#define INCLUDE_uxTaskPriorityGet 0 +#define INCLUDE_vTaskDelete 0 +#define INCLUDE_vTaskSuspend 1 +#define INCLUDE_xResumeFromISR 0 +#define INCLUDE_vTaskDelayUntil 1 +#define INCLUDE_vTaskDelay 1 +#define INCLUDE_xTaskGetSchedulerState 0 +#define INCLUDE_xTaskGetCurrentTaskHandle 1 +#define INCLUDE_uxTaskGetStackHighWaterMark 0 +#define INCLUDE_xTaskGetIdleTaskHandle 0 +#define INCLUDE_xTimerGetTimerDaemonTaskHandle 0 +#define INCLUDE_pcTaskGetTaskName 0 +#define INCLUDE_eTaskGetState 0 +#define INCLUDE_xEventGroupSetBitFromISR 0 +#define INCLUDE_xTimerPendFunctionCall 0 + +#define xPortPendSVHandler PendSV_Handler +#define xPortSysTickHandler SysTick_Handler +#define vPortSVCHandler SVC_Handler + +#define configPRIO_BITS __NVIC_PRIO_BITS +#define configLIBRARY_LOWEST_INTERRUPT_PRIORITY ( ( 1 << configPRIO_BITS ) - 1 ) +#define configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY 2 +#define configKERNEL_INTERRUPT_PRIORITY ( configLIBRARY_LOWEST_INTERRUPT_PRIORITY << ( 8 - configPRIO_BITS ) ) +#define configMAX_SYSCALL_INTERRUPT_PRIORITY ( configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY << ( 8 - configPRIO_BITS ) ) + +#endif diff --git a/hw/bsp/py32f0/boards/py32f071_dev_board/board.cmake b/hw/bsp/py32f0/boards/py32f071_dev_board/board.cmake new file mode 100644 index 000000000..e759f4af1 --- /dev/null +++ b/hw/bsp/py32f0/boards/py32f071_dev_board/board.cmake @@ -0,0 +1,12 @@ +set(PYOCD_TARGET py32f071xb) +set(PY32_SERIES PY32F071) +set(LD_FILE_GNU ${CMAKE_CURRENT_LIST_DIR}/py32f071xb.ld) + +function(update_board TARGET) + target_compile_definitions(${TARGET} PUBLIC + PY32F071xB + CFG_EXAMPLE_MSC_READONLY + CFG_EXAMPLE_MSC_DUAL_READONLY + CFG_EXAMPLE_VIDEO_READONLY + ) +endfunction() diff --git a/hw/bsp/py32f0/boards/py32f071_dev_board/board.h b/hw/bsp/py32f0/boards/py32f071_dev_board/board.h new file mode 100644 index 000000000..ddd67bb6a --- /dev/null +++ b/hw/bsp/py32f0/boards/py32f071_dev_board/board.h @@ -0,0 +1,75 @@ +/* + * The MIT License (MIT) + * + * Copyright (c) 2026, Ha Thach (tinyusb.org) + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + * This file is part of the TinyUSB stack. + */ + +#ifndef BOARD_H_ +#define BOARD_H_ + +#ifdef __cplusplus + extern "C" { +#endif + +#define LED_PORT GPIOB +#define LED_PIN GPIO_PIN_2 +#define LED_STATE_ON 0 + +#define BUTTON_PORT GPIOB +#define BUTTON_PIN GPIO_PIN_0 +#define BUTTON_STATE_ACTIVE 0 + +static inline void board_py32f0_clock_init(void) { + RCC_OscInitTypeDef RCC_OscInitStruct = {0}; + RCC_ClkInitTypeDef RCC_ClkInitStruct = {0}; + + RCC_OscInitStruct.OscillatorType = RCC_OSCILLATORTYPE_HSE | RCC_OSCILLATORTYPE_HSI | + RCC_OSCILLATORTYPE_LSI | RCC_OSCILLATORTYPE_LSE; + RCC_OscInitStruct.HSIState = RCC_HSI_ON; + RCC_OscInitStruct.HSIDiv = RCC_HSI_DIV1; + RCC_OscInitStruct.HSICalibrationValue = RCC_HSICALIBRATION_16MHz; + RCC_OscInitStruct.HSEState = RCC_HSE_ON; + RCC_OscInitStruct.HSEFreq = RCC_HSE_16_32MHz; + RCC_OscInitStruct.LSIState = RCC_LSI_OFF; + RCC_OscInitStruct.LSEState = RCC_LSE_OFF; + RCC_OscInitStruct.PLL.PLLState = RCC_PLL_ON; + RCC_OscInitStruct.PLL.PLLSource = RCC_PLLSOURCE_HSE; + RCC_OscInitStruct.PLL.PLLMUL = RCC_PLL_MUL2; + if (HAL_RCC_OscConfig(&RCC_OscInitStruct) != HAL_OK) { + while (1) {} + } + + RCC_ClkInitStruct.ClockType = RCC_CLOCKTYPE_HCLK | RCC_CLOCKTYPE_SYSCLK | RCC_CLOCKTYPE_PCLK1; + RCC_ClkInitStruct.SYSCLKSource = RCC_SYSCLKSOURCE_PLLCLK; + RCC_ClkInitStruct.AHBCLKDivider = RCC_SYSCLK_DIV1; + RCC_ClkInitStruct.APB1CLKDivider = RCC_HCLK_DIV1; + if (HAL_RCC_ClockConfig(&RCC_ClkInitStruct, FLASH_LATENCY_1) != HAL_OK) { + while (1) {} + } +} + +#ifdef __cplusplus + } +#endif + +#endif diff --git a/hw/bsp/py32f0/boards/py32f071_dev_board/board.mk b/hw/bsp/py32f0/boards/py32f071_dev_board/board.mk new file mode 100644 index 000000000..e7fbc338d --- /dev/null +++ b/hw/bsp/py32f0/boards/py32f071_dev_board/board.mk @@ -0,0 +1,12 @@ +PY32_SERIES = PY32F071 + +CFLAGS += \ + -DPY32F071xB \ + -DCFG_EXAMPLE_MSC_READONLY \ + -DCFG_EXAMPLE_MSC_DUAL_READONLY \ + -DCFG_EXAMPLE_VIDEO_READONLY + +LD_FILE = $(BOARD_PATH)/py32f071xb.ld +PYOCD_TARGET = py32f071xb + +flash: flash-pyocd diff --git a/hw/bsp/py32f0/boards/py32f071_dev_board/py32f071_hal_conf.h b/hw/bsp/py32f0/boards/py32f071_dev_board/py32f071_hal_conf.h new file mode 100644 index 000000000..e513bf8cb --- /dev/null +++ b/hw/bsp/py32f0/boards/py32f071_dev_board/py32f071_hal_conf.h @@ -0,0 +1,105 @@ +/* + * The MIT License (MIT) + * + * Copyright (c) 2026, Ha Thach (tinyusb.org) + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + * This file is part of the TinyUSB stack. + */ + +#ifndef PY32F071_HAL_CONF_H_ +#define PY32F071_HAL_CONF_H_ + +#ifdef __cplusplus + extern "C" { +#endif + +#define HAL_MODULE_ENABLED +#define HAL_RCC_MODULE_ENABLED +#define HAL_FLASH_MODULE_ENABLED +#define HAL_GPIO_MODULE_ENABLED +#define HAL_PWR_MODULE_ENABLED +#define HAL_CORTEX_MODULE_ENABLED + +#if !defined(HSI_VALUE) + #define HSI_VALUE ((uint32_t) 8000000) +#endif + +#if !defined(HSE_VALUE) + #define HSE_VALUE ((uint32_t) 24000000) +#endif + +#if !defined(HSE_STARTUP_TIMEOUT) + #define HSE_STARTUP_TIMEOUT ((uint32_t) 200) +#endif + +#if !defined(LSI_VALUE) + #define LSI_VALUE ((uint32_t) 32768) +#endif + +#if !defined(LSE_VALUE) + #define LSE_VALUE ((uint32_t) 32768) +#endif + +#if !defined(LSE_STARTUP_TIMEOUT) + #define LSE_STARTUP_TIMEOUT ((uint32_t) 5000) +#endif + +#define VDD_VALUE ((uint32_t) 3300) +#define TICK_INT_PRIORITY ((uint32_t) 3) +#define USE_RTOS 0 +#define PREFETCH_ENABLE 0 + +#ifdef HAL_MODULE_ENABLED + #include "py32f0xx_hal.h" +#endif + +#ifdef HAL_RCC_MODULE_ENABLED + #include "py32f071_hal_rcc.h" +#endif + +#ifdef HAL_FLASH_MODULE_ENABLED + #include "py32f071_hal_flash.h" +#endif + +#ifdef HAL_GPIO_MODULE_ENABLED + #include "py32f071_hal_gpio.h" +#endif + +#ifdef HAL_PWR_MODULE_ENABLED + #include "py32f071_hal_pwr.h" +#endif + +#ifdef HAL_CORTEX_MODULE_ENABLED + #include "py32f071_hal_cortex.h" +#endif + +#ifdef USE_FULL_ASSERT +void assert_failed(uint8_t *file, uint32_t line); + #define assert_param(expr) ((expr) ? (void) 0U : assert_failed((uint8_t *) __FILE__, __LINE__)) +#else + #define assert_param(expr) ((void) 0U) +#endif + +#ifdef __cplusplus + } +#endif + +#endif diff --git a/hw/bsp/py32f0/boards/py32f071_dev_board/py32f071xb.ld b/hw/bsp/py32f0/boards/py32f071_dev_board/py32f071xb.ld new file mode 100644 index 000000000..bb6b88d2c --- /dev/null +++ b/hw/bsp/py32f0/boards/py32f071_dev_board/py32f071xb.ld @@ -0,0 +1,122 @@ +ENTRY(Reset_Handler) + +_estack = ORIGIN(RAM) + LENGTH(RAM); +_Min_Heap_Size = 0x200; +_Min_Stack_Size = 0x400; + +MEMORY +{ + RAM (xrw) : ORIGIN = 0x20000000, LENGTH = 16K + FLASH (rx) : ORIGIN = 0x08000000, LENGTH = 128K +} + +SECTIONS +{ + .isr_vector : + { + . = ALIGN(4); + KEEP(*(.isr_vector)) + . = ALIGN(4); + } >FLASH + + .text : + { + . = ALIGN(4); + *(.text) + *(.text*) + *(.glue_7) + *(.glue_7t) + *(.eh_frame) + KEEP(*(.init)) + KEEP(*(.fini)) + . = ALIGN(4); + _etext = .; + } >FLASH + + .rodata : + { + . = ALIGN(4); + *(.rodata) + *(.rodata*) + . = ALIGN(4); + } >FLASH + + .ARM.extab : + { + *(.ARM.extab* .gnu.linkonce.armextab.*) + } >FLASH + + .ARM : + { + __exidx_start = .; + *(.ARM.exidx*) + __exidx_end = .; + } >FLASH + + .preinit_array : + { + PROVIDE_HIDDEN(__preinit_array_start = .); + KEEP(*(.preinit_array*)) + PROVIDE_HIDDEN(__preinit_array_end = .); + } >FLASH + + .init_array : + { + PROVIDE_HIDDEN(__init_array_start = .); + KEEP(*(SORT(.init_array.*))) + KEEP(*(.init_array*)) + PROVIDE_HIDDEN(__init_array_end = .); + } >FLASH + + .fini_array : + { + PROVIDE_HIDDEN(__fini_array_start = .); + KEEP(*(SORT(.fini_array.*))) + KEEP(*(.fini_array*)) + PROVIDE_HIDDEN(__fini_array_end = .); + } >FLASH + + _sidata = LOADADDR(.data); + + .data : + { + . = ALIGN(4); + _sdata = .; + *(.data) + *(.data*) + . = ALIGN(4); + _edata = .; + } >RAM AT> FLASH + + . = ALIGN(4); + .bss : + { + _sbss = .; + __bss_start__ = _sbss; + *(.bss) + *(.bss*) + *(COMMON) + . = ALIGN(4); + _ebss = .; + __bss_end__ = _ebss; + } >RAM + + ._user_heap_stack : + { + . = ALIGN(8); + PROVIDE(end = .); + PROVIDE(_end = .); + . = . + _Min_Heap_Size; + . = . + _Min_Stack_Size; + . = ALIGN(8); + } >RAM + + /DISCARD/ : + { + libc.a(*) + libm.a(*) + libgcc.a(*) + } + + .ARM.attributes 0 : { *(.ARM.attributes) } +} diff --git a/hw/bsp/py32f0/family.c b/hw/bsp/py32f0/family.c new file mode 100644 index 000000000..1428b1fa9 --- /dev/null +++ b/hw/bsp/py32f0/family.c @@ -0,0 +1,133 @@ +/* + * The MIT License (MIT) + * + * Copyright (c) 2026, Ha Thach (tinyusb.org) + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + * This file is part of the TinyUSB stack. + */ + +/* metadata: + manufacturer: Puya +*/ + +#include "py32f0xx_hal.h" +#include "bsp/board_api.h" +#include "board.h" + +void USB_IRQHandler(void) { + tud_int_handler(0); +} + +void USBD_IRQHandler(void) { + tud_int_handler(0); +} + +void HAL_MspInit(void) { + __HAL_RCC_SYSCFG_CLK_ENABLE(); + __HAL_RCC_PWR_CLK_ENABLE(); +} + +void board_init(void) { + HAL_Init(); + board_py32f0_clock_init(); + + __HAL_RCC_SYSCFG_CLK_ENABLE(); + __HAL_RCC_PWR_CLK_ENABLE(); + __HAL_RCC_GPIOA_CLK_ENABLE(); + __HAL_RCC_GPIOB_CLK_ENABLE(); + +#if CFG_TUSB_OS == OPT_OS_NONE + SysTick_Config(SystemCoreClock / 1000); +#elif CFG_TUSB_OS == OPT_OS_FREERTOS + SysTick->CTRL &= ~1U; + NVIC_SetPriority(USB_IRQn, configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY); +#endif + + GPIO_InitTypeDef GPIO_InitStruct; + + GPIO_InitStruct.Pin = LED_PIN; + GPIO_InitStruct.Mode = GPIO_MODE_OUTPUT_PP; + GPIO_InitStruct.Pull = GPIO_PULLUP; + GPIO_InitStruct.Speed = GPIO_SPEED_FREQ_HIGH; + HAL_GPIO_Init(LED_PORT, &GPIO_InitStruct); + board_led_write(false); + + GPIO_InitStruct.Pin = BUTTON_PIN; + GPIO_InitStruct.Mode = GPIO_MODE_INPUT; + GPIO_InitStruct.Pull = GPIO_PULLUP; + GPIO_InitStruct.Speed = GPIO_SPEED_FREQ_HIGH; + HAL_GPIO_Init(BUTTON_PORT, &GPIO_InitStruct); + + __HAL_RCC_USB_CLK_ENABLE(); +} + +void board_led_write(bool state) { + GPIO_PinState pin_state = (GPIO_PinState) (state ? LED_STATE_ON : (1 - LED_STATE_ON)); + HAL_GPIO_WritePin(LED_PORT, LED_PIN, pin_state); +} + +uint32_t board_button_read(void) { + return BUTTON_STATE_ACTIVE == HAL_GPIO_ReadPin(BUTTON_PORT, BUTTON_PIN); +} + +size_t board_get_unique_id(uint8_t id[], size_t max_len) { + (void) max_len; + volatile uint32_t * py32_uuid = (volatile uint32_t *) UID_BASE; + uint32_t* id32 = (uint32_t*) (uintptr_t) id; + uint8_t const len = 12; + + id32[0] = py32_uuid[0]; + id32[1] = py32_uuid[1]; + id32[2] = py32_uuid[2]; + + return len; +} + +int board_uart_read(uint8_t *buf, int len) { + (void) buf; (void) len; + return 0; +} + +int board_uart_write(void const *buf, int len) { + (void) buf; (void) len; + return -1; +} + +#if CFG_TUSB_OS == OPT_OS_NONE +volatile uint32_t system_ticks = 0; + +void SysTick_Handler(void) { + HAL_IncTick(); + system_ticks++; +} + +uint32_t tusb_time_millis_api(void) { + return system_ticks; +} +#endif + +void HardFault_Handler(void) { + __asm("BKPT #0\n"); +} + +void _init(void); +void _init(void) { +} diff --git a/hw/bsp/py32f0/family.cmake b/hw/bsp/py32f0/family.cmake new file mode 100644 index 000000000..df6dec42b --- /dev/null +++ b/hw/bsp/py32f0/family.cmake @@ -0,0 +1,89 @@ +include_guard() + +include(${CMAKE_CURRENT_LIST_DIR}/boards/${BOARD}/board.cmake) + +if (NOT DEFINED MCU_VARIANT) + set(MCU_VARIANT PY32F071) +endif () +string(TOLOWER ${MCU_VARIANT} PY32_SERIES_LOWER) +set(PY32_SDK_NAME ${MCU_VARIANT}_Firmware) +set(PY32_TEMPLATE ${MCU_VARIANT}xx_Templates) +set(PY32_STARTUP startup_${PY32_SERIES_LOWER}xx.s) +set(PY32_SYSTEM_SOURCE system_${PY32_SERIES_LOWER}.c) +set(PY32_HAL_PREFIX ${PY32_SERIES_LOWER}) + +set(PY32_SDK ${TOP}/hw/mcu/puya/${PY32_SDK_NAME}) +set(PY32_HAL ${PY32_SDK}/Drivers/${MCU_VARIANT}_HAL_Driver) +set(PY32_CMSIS ${PY32_SDK}/Drivers/CMSIS/Device/${MCU_VARIANT}) + +set(CMAKE_SYSTEM_CPU cortex-m0plus CACHE INTERNAL "System Processor") +set(CMAKE_TOOLCHAIN_FILE ${TOP}/examples/build_system/cmake/toolchain/arm_${TOOLCHAIN}.cmake) + +set(FAMILY_MCUS PY32F0 CACHE INTERNAL "") + +set(STARTUP_FILE_GNU ${PY32_SDK}/Templates/${PY32_TEMPLATE}/EIDE/${PY32_STARTUP}) +set(STARTUP_FILE_Clang ${STARTUP_FILE_GNU}) +set(LD_FILE_Clang ${LD_FILE_GNU}) + +function(family_add_board BOARD_TARGET) + add_library(${BOARD_TARGET} STATIC + ${PY32_SDK}/Templates/${PY32_TEMPLATE}/Src/${PY32_SYSTEM_SOURCE} + ${PY32_HAL}/Src/${PY32_HAL_PREFIX}_hal.c + ${PY32_HAL}/Src/${PY32_HAL_PREFIX}_hal_cortex.c + ${PY32_HAL}/Src/${PY32_HAL_PREFIX}_hal_flash.c + ${PY32_HAL}/Src/${PY32_HAL_PREFIX}_hal_gpio.c + ${PY32_HAL}/Src/${PY32_HAL_PREFIX}_hal_pwr.c + ${PY32_HAL}/Src/${PY32_HAL_PREFIX}_hal_rcc.c + ${PY32_HAL}/Src/${PY32_HAL_PREFIX}_hal_rcc_ex.c + ) + target_include_directories(${BOARD_TARGET} PUBLIC + ${CMAKE_CURRENT_FUNCTION_LIST_DIR} + ${PY32_SDK}/Drivers/CMSIS/Include + ${PY32_CMSIS}/Include + ${PY32_HAL}/Inc + ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/boards/${BOARD} + ) + target_compile_definitions(${BOARD_TARGET} PRIVATE + USE_HAL_DRIVER + ) + update_board(${BOARD_TARGET}) +endfunction() + +function(family_configure_example TARGET RTOS) + family_configure_common(${TARGET} ${RTOS}) + family_add_tinyusb(${TARGET} OPT_MCU_PY32F0) + + target_sources(${TARGET} PUBLIC + ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/family.c + ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/../board.c + ${TOP}/src/portable/mentor/musb/dcd_musb.c + ${STARTUP_FILE_${CMAKE_C_COMPILER_ID}} + ) + target_include_directories(${TARGET} PUBLIC + ${CMAKE_CURRENT_FUNCTION_LIST_DIR} + ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/../../ + ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/boards/${BOARD} + ) + + if (CMAKE_C_COMPILER_ID STREQUAL "GNU") + target_link_options(${TARGET} PUBLIC + "LINKER:--script=${LD_FILE_GNU}" + -nostartfiles + --specs=nosys.specs --specs=nano.specs + ) + elseif (CMAKE_C_COMPILER_ID STREQUAL "Clang") + message(FATAL_ERROR "Clang is not supported") + elseif (CMAKE_C_COMPILER_ID STREQUAL "IAR") + message(FATAL_ERROR "IAR is not supported") + endif () + + if (CMAKE_C_COMPILER_ID STREQUAL "GNU" OR CMAKE_C_COMPILER_ID STREQUAL "Clang") + set_source_files_properties(${CMAKE_CURRENT_FUNCTION_LIST_DIR}/family.c PROPERTIES COMPILE_FLAGS "-Wno-missing-prototypes") + endif () + set_source_files_properties(${STARTUP_FILE_${CMAKE_C_COMPILER_ID}} PROPERTIES + SKIP_LINTING ON + COMPILE_OPTIONS -w) + + family_add_bin_hex(${TARGET}) + family_flash_pyocd(${TARGET}) +endfunction() diff --git a/hw/bsp/py32f0/family.mk b/hw/bsp/py32f0/family.mk new file mode 100644 index 000000000..b01a9eefa --- /dev/null +++ b/hw/bsp/py32f0/family.mk @@ -0,0 +1,58 @@ +UF2_FAMILY_ID = 0x0 + +include $(TOP)/$(BOARD_PATH)/board.mk + +MCU_VARIANT ?= PY32F071 +PY32_SERIES_LOWER = $(subst PY32F,py32f,$(MCU_VARIANT)) +PY32_SDK_NAME = $(MCU_VARIANT)_Firmware +PY32_TEMPLATE = $(MCU_VARIANT)xx_Templates +PY32_STARTUP = startup_$(PY32_SERIES_LOWER)xx.s +PY32_SYSTEM_SOURCE = system_$(PY32_SERIES_LOWER).c +PY32_HAL_PREFIX = $(PY32_SERIES_LOWER) + +SDK_DIR = hw/mcu/puya/$(PY32_SDK_NAME) +HAL_DIR = $(SDK_DIR)/Drivers/$(MCU_VARIANT)_HAL_Driver +CMSIS_DIR = $(SDK_DIR)/Drivers/CMSIS +CMSIS_DEVICE_DIR = $(CMSIS_DIR)/Device/$(MCU_VARIANT) + +DEPS_SUBMODULES += $(SDK_DIR) + +CFLAGS += \ + -DCFG_TUSB_MCU=OPT_MCU_PY32F0 \ + -DUSE_HAL_DRIVER + +# Puya HAL leaves some CMSIS-compatible callback parameters intentionally unused. +CFLAGS += -Wno-error=unused-parameter + +MCU_DIR = $(SDK_DIR) + +SRC_C += \ + src/portable/mentor/musb/dcd_musb.c \ + hw/bsp/py32f0/family.c \ + $(SDK_DIR)/Templates/$(PY32_TEMPLATE)/Src/$(PY32_SYSTEM_SOURCE) \ + $(HAL_DIR)/Src/$(PY32_HAL_PREFIX)_hal.c \ + $(HAL_DIR)/Src/$(PY32_HAL_PREFIX)_hal_cortex.c \ + $(HAL_DIR)/Src/$(PY32_HAL_PREFIX)_hal_flash.c \ + $(HAL_DIR)/Src/$(PY32_HAL_PREFIX)_hal_gpio.c \ + $(HAL_DIR)/Src/$(PY32_HAL_PREFIX)_hal_pwr.c \ + $(HAL_DIR)/Src/$(PY32_HAL_PREFIX)_hal_rcc.c \ + $(HAL_DIR)/Src/$(PY32_HAL_PREFIX)_hal_rcc_ex.c + +SRC_S += $(SDK_DIR)/Templates/$(PY32_TEMPLATE)/EIDE/$(PY32_STARTUP) + +INC += \ + $(TOP)/hw/bsp/py32f0 \ + $(TOP)/$(CMSIS_DIR)/Include \ + $(TOP)/$(CMSIS_DEVICE_DIR)/Include \ + $(TOP)/$(HAL_DIR)/Inc \ + $(TOP)/hw/bsp/py32f0/boards/$(BOARD) + +SKIP_NANOLIB = 1 + +LDFLAGS += \ + -nostdlib -nostartfiles \ + --specs=nosys.specs --specs=nano.specs \ + -Wl,--gc-sections \ + -Wl,--print-memory-usage + +CPU_CORE ?= cortex-m0plus diff --git a/hw/bsp/ra/boards/ra6m5_ek/ozone/ra6m5.jdebug b/hw/bsp/ra/boards/ra6m5_ek/ozone/ra6m5.jdebug index ca18fed7c..466658f39 100644 --- a/hw/bsp/ra/boards/ra6m5_ek/ozone/ra6m5.jdebug +++ b/hw/bsp/ra/boards/ra6m5_ek/ozone/ra6m5.jdebug @@ -15,7 +15,7 @@ void OnProjectLoad (void) { Project.SetDevice ("R7FA6M5BH"); Project.SetHostIF ("USB", ""); Project.SetTargetIF ("SWD"); - Project.SetTIFSpeed ("50 MHz"); + Project.SetTIFSpeed ("4 MHz"); // 50 MHz SWD gave intermittent "Failed to initialize DAP" Project.SetTraceSource ("Trace Pins"); Project.SetTracePortWidth (4); diff --git a/hw/bsp/ra/boards/ra8m1_ek/ozone/ra8m1.jdebug b/hw/bsp/ra/boards/ra8m1_ek/ozone/ra8m1.jdebug index 242a15db9..02d568240 100644 --- a/hw/bsp/ra/boards/ra8m1_ek/ozone/ra8m1.jdebug +++ b/hw/bsp/ra/boards/ra8m1_ek/ozone/ra8m1.jdebug @@ -10,10 +10,11 @@ */ void OnProjectLoad (void) { Project.SetTraceSource ("Trace Pins"); + Project.SetTracePortWidth (4); Project.SetDevice ("R7FA8M1AH"); Project.SetHostIF ("USB", ""); Project.SetTargetIF ("SWD"); - Project.SetTIFSpeed ("50 MHz"); + Project.SetTIFSpeed ("4 MHz"); // 50 MHz SWD gave intermittent "Failed to initialize DAP" Project.AddSvdFile ("$(InstallDir)/Config/CPU/Cortex-M85F.svd"); Project.AddSvdFile ("../../../../../../../cmsis-svd-data/data/Renesas/R7FA6M5BH.svd"); @@ -32,7 +33,7 @@ void OnProjectLoad (void) { void BeforeTargetConnect (void) { // Trace pin init is done by J-Link script file as J-Link script files are IDE independent //Project.SetJLinkScript("../../../debug.jlinkscript"); - Project.SetJLinkScript ("$(ProjectDir)/Renesas_RA8_TracePins.pex"); + Project.SetJLinkScript ("./ra8m1_trace.JLinkScript"); } /********************************************************************* @@ -40,12 +41,14 @@ void BeforeTargetConnect (void) { * AfterTargetConnect * * Function description -* Event handler routine. Optional. +* Cache the boot-ROM range for the trace decoder on --attach sessions +* too (the download hook that normally does this is skipped on attach). * ********************************************************************** */ -//void AfterTargetConnect (void) { -//} +void AfterTargetConnect (void) { + Exec.Command("ReadIntoTraceCache 0x0 0x10000"); +} /********************************************************************* * @@ -83,8 +86,12 @@ void BeforeTargetConnect (void) { */ void AfterTargetDownload (void) { _SetupTarget(); + // RA8 executes chip-ROM code at runtime (seen at ~0x3B20); without this the + // trace decoder dies there ("not covered by trace cache" -> unknown packet) + Exec.Command("ReadIntoTraceCache 0x0 0x10000"); } + /********************************************************************* * * BeforeTargetDisconnect diff --git a/hw/bsp/ra/boards/ra8m1_ek/ozone/ra8m1_trace.JLinkScript b/hw/bsp/ra/boards/ra8m1_ek/ozone/ra8m1_trace.JLinkScript new file mode 100644 index 000000000..3869c1eb7 --- /dev/null +++ b/hw/bsp/ra/boards/ra8m1_ek/ozone/ra8m1_trace.JLinkScript @@ -0,0 +1,20 @@ +/* RA8M1 pin trace: CSTF funnel and TMC are off the core ROM table (AP1) - + * declare them or trace init cannot route the M85 ETM stream (addresses per + * SEGGER's RA8 trace example script). Pins + TRCKCR belong to firmware + * (trace_etm_init): the SEGGER pex also muxed them at trace start, and its + * clock choice fought the firmware's mid-run = decoder desync. + */ +int ConfigTargetSettings(void) { + JLINK_ExecCommand("CORESIGHT_SetCSTFBaseAddr = 0x80013000 ForceUnlock = 1 APIndex = 1"); + JLINK_ExecCommand("CORESIGHT_SetTMCBaseAddr = 0x80014000 ForceUnlock = 1 APIndex = 1"); + return 0; +} + +/* Replace J-Link's built-in RA8 trace start, which enables the trace clock + * from reset - bsp_clock_init's MOCO -> 480 MHz PLL switch would then step + * TRCLK mid-stream and desync the decoder. The firmware's trace_etm_init + * enables TRCKCR at the final clock instead. + */ +int OnTraceStart(void) { + return 0; +} diff --git a/hw/bsp/ra/boards/ra8m1_ek/ra_gen/bsp_clock_cfg.h b/hw/bsp/ra/boards/ra8m1_ek/ra_gen/bsp_clock_cfg.h index f2f1ae0c9..25638f02b 100644 --- a/hw/bsp/ra/boards/ra8m1_ek/ra_gen/bsp_clock_cfg.h +++ b/hw/bsp/ra/boards/ra8m1_ek/ra_gen/bsp_clock_cfg.h @@ -51,6 +51,9 @@ #define BSP_CFG_CANFDCLK_DIV (BSP_CLOCKS_CANFD_CLOCK_DIV_8) /* CANFDCLK Div /8 */ #define BSP_CFG_I3CCLK_DIV (BSP_CLOCKS_I3C_CLOCK_DIV_3) /* I3CCLK Div /3 */ #define BSP_CFG_UCK_DIV (BSP_CLOCKS_USB_CLOCK_DIV_5) /* UCK Div /5 */ -#define BSP_CFG_U60CK_DIV (BSP_CLOCKS_USB_CLOCK_DIV_8) /* U60CK Div /8 */ +/* U60CK Div /8: PLL1P 480 MHz -> 60 MHz. Hand-fixed: Smart Configurator emitted the USB_ macro + * namespace (BSP_CLOCKS_USB_CLOCK_DIV_8 = 7, rejected by USB60CKDIVCR -> link clock ran at + * 480 MHz); configuration.xml already says u60ck.div.8, so keep the USB60_ macro if regenerating. */ +#define BSP_CFG_U60CK_DIV (BSP_CLOCKS_USB60_CLOCK_DIV_8) #define BSP_CFG_OCTA_DIV (BSP_CLOCKS_OCTA_CLOCK_DIV_4) /* OCTASPICLK Div /4 */ #endif /* BSP_CLOCK_CFG_H_ */ diff --git a/hw/bsp/ra/family.c b/hw/bsp/ra/family.c index c8a4d33d9..307644972 100644 --- a/hw/bsp/ra/family.c +++ b/hw/bsp/ra/family.c @@ -99,13 +99,32 @@ void board_init(void) { R_IOPORT_Open(&IOPORT_CFG_CTRL, &IOPORT_CFG_NAME); #ifdef TRACE_ETM - // TRCKCR is protected by PRCR bit0 register - R_SYSTEM->PRCR = (uint16_t) (BSP_PRV_PRCR_KEY | 0x01); + // TRCKCR is only writable while a debugger is connected (RA HUM) - a + // standalone boot must skip trace init or the write wedges the chip into + // an un-attachable crash loop (recover: power-cycle + immediate erase) + if (DCB->DHCSR & DCB_DHCSR_C_DEBUGEN_Msk) { + // TRCKCR is protected by PRCR bit0 register + R_SYSTEM->PRCR = (uint16_t) (BSP_PRV_PRCR_KEY | 0x01); - // Enable trace clock (max 100Mhz). Since PLL/CPU is 200Mhz, clock div = 2 - R_SYSTEM->TRCKCR = R_SYSTEM_TRCKCR_TRCKEN_Msk | 0x01; + // TCLK pin = TRCLK/2; set the divider with TRCKEN=0 first (HUM procedure). + // Values are the empirical per-board ceilings. +#if defined(BSP_MCU_GROUP_RA8M1) + // 480 MHz CPU: /4 -> 120 MHz TRCLK, 60 MHz pin - chip max, clean on + // EK-RA8M1 with the committed empty-OnTraceStart JLinkScript (which + // defers the trace clock to firmware; without it the FSP MOCO->PLL + // switch steps the clock mid-stream and any divider fails) + R_SYSTEM->TRCKCR = 0x02; + R_SYSTEM->TRCKCR = R_SYSTEM_TRCKCR_TRCKEN_Msk | 0x02; +#else + // RA6M5 200 MHz CPU: /4 -> 50 MHz TRCLK, 25 MHz pin. /2 (50 MHz pin) is + // silent on the EK-RA6M5 in every combination - board path ceiling, + // reconfirmed with J9 closed and the OnTraceStart override + R_SYSTEM->TRCKCR = 0x02; + R_SYSTEM->TRCKCR = R_SYSTEM_TRCKCR_TRCKEN_Msk | 0x02; +#endif - R_SYSTEM->PRCR = (uint16_t) BSP_PRV_PRCR_KEY; + R_SYSTEM->PRCR = (uint16_t) BSP_PRV_PRCR_KEY; + } #endif #if CFG_TUSB_OS == OPT_OS_FREERTOS diff --git a/hw/bsp/rp2040/boards/raspberry_pi_pico2/board.cmake b/hw/bsp/rp2040/boards/raspberry_pi_pico2/board.cmake index 0a7dd4d23..08384b0cd 100644 --- a/hw/bsp/rp2040/boards/raspberry_pi_pico2/board.cmake +++ b/hw/bsp/rp2040/boards/raspberry_pi_pico2/board.cmake @@ -1,3 +1,17 @@ set(PICO_PLATFORM rp2350-arm-s) set(PICO_BOARD pico2) #set(OPENOCD_SERIAL E6614103E77C5A24) + +if (TRACE_ETM STREQUAL "1") + # TRACECLK is clk_sys/2 and must stay constant once trace is armed (a step + # desyncs the decoder), so the trace clock is pinned from crt0 onwards. + # 48 MHz (24 MHz TRACECLK) holds full-width trace on a typical fly-wire + # seating; a fresh, tight seating supports up to 72-80 MHz (re-qualify per + # the etm-trace skill), and >80 MHz needs a V3 probe + real trace board. + add_compile_definitions( + SYS_CLK_KHZ=48000 + PLL_SYS_VCO_FREQ_HZ=1440000000 + PLL_SYS_POSTDIV1=6 + PLL_SYS_POSTDIV2=5 + ) +endif () diff --git a/hw/bsp/rp2040/boards/raspberry_pi_pico2/ozone/rp2350.jdebug b/hw/bsp/rp2040/boards/raspberry_pi_pico2/ozone/rp2350.jdebug new file mode 100644 index 000000000..ff48eb673 --- /dev/null +++ b/hw/bsp/rp2040/boards/raspberry_pi_pico2/ozone/rp2350.jdebug @@ -0,0 +1,70 @@ +/********************************************************************* +* +* OnProjectLoad +* +* Function description +* Project load routine. Required. +* +* Notes +* Pico 2 has no trace connector - fly-wire GPIO1-5 to the MIPI20: +* TRACECLK=GPIO1->12, D0=GPIO2->14, D1=GPIO3->16, D2=GPIO4->18, +* D3=GPIO5->20 (SEGGER validates this board the same way). Firmware must +* be built with TRACE_ETM=1: it pins clk_sys to 48 MHz (board.cmake) so +* the 4-bit port never saturates and the clock never steps mid-stream, +* and keeps the us-timer free of TIMER DBGPAUSE (family.c). The whole +* chip-side trace path (ETM/funnel/TPIU/pin mux) is armed by J-Link's +* built-in RP2350 script at every resume - do NOT set a custom +* JLinkScript here: it would replace that script and J-Link then fails +* with "Required trace components for pin trace not found". +* GPIO1 is the default UART0 RX: console TX still works, RX is lost. +* +********************************************************************** +*/ +void OnProjectLoad (void) { + Project.SetTraceSource ("Trace Pins"); + Project.SetTracePortWidth (4); + Project.SetSWO (0); + Edit.SysVar (VAR_TRACE_CORE_CLOCK, 48000000); + Project.AddSvdFile ("$(InstallDir)/Config/CPU/Cortex-M33F.svd"); + + Project.SetDevice ("RP2350_M33_0"); + Project.SetHostIF ("USB", ""); + Project.SetTargetIF ("SWD"); + Project.SetTIFSpeed ("25 MHz"); + + File.Open ("../../../../../../examples/cmake-build-raspberry_pi_pico2/device/cdc_msc/cdc_msc.elf"); +} + +/********************************************************************* +* +* AfterTargetReset +* +* Function description +* Event handler routine. +* - Sets the PC register to program reset value. +* - Sets the SP register to program reset value on Cortex-M. +* +********************************************************************** +*/ +void AfterTargetReset (void) { + // intentionally empty: the RP2350 bootrom must run to validate the + // IMAGE_DEF and hand over to the app - setting SP/PC from the vector + // table bypasses it and the pico-sdk runtime never comes up +} + +/********************************************************************* +* +* AfterTargetDownload +* +* Function description +* Event handler routine. +* - Sets the PC register to program reset value. +* - Sets the SP register to program reset value on Cortex-M. +* +********************************************************************** +*/ +void AfterTargetDownload (void) { + // intentionally empty: the RP2350 bootrom must run to validate the + // IMAGE_DEF and hand over to the app - setting SP/PC from the vector + // table bypasses it and the pico-sdk runtime never comes up +} diff --git a/hw/bsp/rp2040/family.c b/hw/bsp/rp2040/family.c index 55feec159..e12f51b14 100644 --- a/hw/bsp/rp2040/family.c +++ b/hw/bsp/rp2040/family.c @@ -153,7 +153,7 @@ static stdio_driver_t stdio_rtt = { .in_chars = stdio_rtt_read }; -void stdio_rtt_init(void) { +static void stdio_rtt_init(void) { stdio_set_driver_enabled(&stdio_rtt, true); } #endif @@ -161,11 +161,28 @@ void stdio_rtt_init(void) { //--------------------------------------------------------------------+ // //--------------------------------------------------------------------+ +#if defined(TRACE_ETM) && defined(PICO_RP2350) && PICO_RP2350 == 1 +// J-Link's built-in RP2350 device script re-arms the whole chip-side trace +// path (ETM/funnel/TPIU/pins) via OnTraceStart at every resume, so firmware +// must NOT touch it - it only keeps the us-timer running while cores sit +// debug-halted (default TIMER DBGPAUSE freezes it, and sleep_ms() then spins +// forever after any debugger session). +static void trace_etm_init(void) { + *(volatile uint32_t*) 0x400B002Cu = 0; // TIMER0 DBGPAUSE + *(volatile uint32_t*) 0x400B802Cu = 0; // TIMER1 DBGPAUSE +} +#else + #define trace_etm_init() +#endif + void board_init(void) { #if (CFG_TUH_ENABLED && CFG_TUH_RPI_PIO_USB) || (CFG_TUD_ENABLED && CFG_TUD_RPI_PIO_USB) // Set the system clock to a multiple of 12mhz for bit-banging USB with pico-usb #if defined(PICO_RP2350) && PICO_RP2350 == 1 + #ifdef TRACE_ETM + #error "TRACE_ETM pins clk_sys to 48 MHz (board.cmake) - too slow for PIO-USB, and a runtime clock switch desyncs the trace stream" + #endif set_sys_clock_khz(156000, true); // rp2350 default is 150Mhz #else set_sys_clock_khz(120000, true); // rp2040 default is 125Mhz @@ -199,10 +216,18 @@ void board_init(void) #endif #ifdef UART_DEV - bi_decl(bi_2pins_with_func(UART_TX_PIN, UART_RX_PIN, GPIO_FUNC_UART)); uart_inst = uart_get_instance(UART_DEV); +#if defined(TRACE_ETM) && defined(PICO_RP2350) && PICO_RP2350 == 1 + // GPIO1 (default UART RX) is TRACECLK: TX-only console, and never touch + // GPIO1 - even a brief re-mux gaps the trace clock and desyncs the probe + bi_decl(bi_1pin_with_name(UART_TX_PIN, "UART TX")); + stdio_uart_init_full(uart_inst, CFG_BOARD_UART_BAUDRATE, UART_TX_PIN, -1); +#else + bi_decl(bi_2pins_with_func(UART_TX_PIN, UART_RX_PIN, GPIO_FUNC_UART)); stdio_uart_init_full(uart_inst, CFG_BOARD_UART_BAUDRATE, UART_TX_PIN, UART_RX_PIN); #endif +#endif + trace_etm_init(); #if defined(LOGGER_RTT) stdio_rtt_init(); diff --git a/hw/bsp/rp2040/family.cmake b/hw/bsp/rp2040/family.cmake index aab9a4fae..43b1dc234 100644 --- a/hw/bsp/rp2040/family.cmake +++ b/hw/bsp/rp2040/family.cmake @@ -28,6 +28,8 @@ elseif (PICO_PLATFORM STREQUAL "rp2350-arm-s" OR PICO_PLATFORM STREQUAL "rp2350" set(OPENOCD_TARGET rp2350) elseif (PICO_PLATFORM STREQUAL "rp2350-riscv") set(JLINK_DEVICE rp2350_riscv_0) + # rp2350-riscv.cfg needs the raspberrypi/openocd fork: mainline's riscv + # target does not take the -dap/-ap-num the config uses set(OPENOCD_TARGET rp2350-riscv) endif() diff --git a/hw/bsp/samd5x_e5x/boards/same54_xplained/ozone/same54.jdebug b/hw/bsp/samd5x_e5x/boards/same54_xplained/ozone/same54.jdebug new file mode 100644 index 000000000..a0f8d5c3d --- /dev/null +++ b/hw/bsp/samd5x_e5x/boards/same54_xplained/ozone/same54.jdebug @@ -0,0 +1,89 @@ +/********************************************************************* +* +* OnProjectLoad +* +* Function description +* Project load routine. Required. +* +* Notes +* SAM E54 Xplained Pro carries a populated 20-pin Cortex Debug+ETM +* connector (Table 5-11): TRACECLK=PC27, D0=PC28, D1=PC26, D2=PC25, +* D3=PC24 - no rework needed. Firmware must be built with TRACE_ETM=1 +* (mux function H on those pins in board_init); TPIU/ETM are +* ROM-table-discoverable so no J-Link script is required. +* Trace clock is CPU/2 = 60 MHz at the stock 120 MHz core. +* +********************************************************************** +*/ +void OnProjectLoad (void) { + Project.SetTraceSource ("Trace Pins"); + Project.SetTracePortWidth (4); + Project.SetSWO (0); + Edit.SysVar (VAR_TRACE_CORE_CLOCK, 120000000); + Project.AddSvdFile ("$(InstallDir)/Config/CPU/Cortex-M4F.svd"); + + Project.SetDevice ("ATSAME54P20"); + Project.SetHostIF ("USB", ""); + Project.SetTargetIF ("SWD"); + Project.SetTIFSpeed ("4 MHz"); + + File.Open ("../../../../../../examples/cmake-build-same54_xplained/device/cdc_msc/cdc_msc.elf"); +} + +/********************************************************************* +* +* AfterTargetReset +* +* Function description +* Event handler routine. +* - Sets the PC register to program reset value. +* - Sets the SP register to program reset value on Cortex-M. +* +********************************************************************** +*/ +void AfterTargetReset (void) { + unsigned int SP; + unsigned int PC; + unsigned int VectorTableAddr; + + VectorTableAddr = Elf.GetBaseAddr(); + + if (VectorTableAddr == 0xFFFFFFFF) { + Util.Log("Project file error: failed to get program base"); + } else { + SP = Target.ReadU32(VectorTableAddr); + Target.SetReg("SP", SP); + + PC = Target.ReadU32(VectorTableAddr + 4); + Target.SetReg("PC", PC); + } +} + +/********************************************************************* +* +* AfterTargetDownload +* +* Function description +* Event handler routine. +* - Sets the PC register to program reset value. +* - Sets the SP register to program reset value on Cortex-M. +* +********************************************************************** +*/ +void AfterTargetDownload (void) { + unsigned int SP; + unsigned int PC; + unsigned int VectorTableAddr; + + VectorTableAddr = Elf.GetBaseAddr(); + + if (VectorTableAddr == 0xFFFFFFFF) { + Util.Log("Project file error: failed to get program base"); + } else { + SP = Target.ReadU32(VectorTableAddr); + Target.SetReg("SP", SP); + + PC = Target.ReadU32(VectorTableAddr + 4); + Target.SetReg("PC", PC); + } +} diff --git a/hw/bsp/samd5x_e5x/family.c b/hw/bsp/samd5x_e5x/family.c index 71ef2d6ce..a2c3b70de 100644 --- a/hw/bsp/samd5x_e5x/family.c +++ b/hw/bsp/samd5x_e5x/family.c @@ -87,6 +87,31 @@ void USB_3_Handler(void) { USB_Any_Handler(); } static void max3421_init(void); #endif +#if defined(TRACE_ETM) +// same54_xplained routes 4-bit trace to its 20-pin Cortex Debug+ETM header: +// TRACECLK=PC27, D0=PC28, D1=PC26, D2=PC25, D3=PC24 - all peripheral +// function H (CM4 trace). TPIU/ETM are ROM-table-discoverable; the debugger +// arms them, firmware only muxes the pins. +static void trace_etm_init(void) { + // the CM4 trace unit runs from its own GCLK channel (47): feed it GCLK0 + // (CPU clock) - without this the pins mux fine but the port stays silent + GCLK->PCHCTRL[47].reg = GCLK_PCHCTRL_GEN_GCLK0 | GCLK_PCHCTRL_CHEN; + while (!(GCLK->PCHCTRL[47].reg & GCLK_PCHCTRL_CHEN)) {} + + const uint8_t pin[] = {24, 25, 26, 27, 28}; + for (unsigned i = 0; i < 5; i++) { + PORT->Group[2].PINCFG[pin[i]].reg = PORT_PINCFG_PMUXEN | PORT_PINCFG_DRVSTR; + if (pin[i] & 1) { + PORT->Group[2].PMUX[pin[i] >> 1].bit.PMUXO = 7; // function H + } else { + PORT->Group[2].PMUX[pin[i] >> 1].bit.PMUXE = 7; + } + } +} +#else + #define trace_etm_init() +#endif + void board_init(void) { // Clock init ( follow hpl_init.c ) hri_nvmctrl_set_CTRLA_RWS_bf(NVMCTRL, 0); @@ -104,6 +129,8 @@ void board_init(void) { // Init 1ms tick timer (samd SystemCoreClock may not correct) SystemCoreClock = CONF_CPU_FREQUENCY; + trace_etm_init(); + #if CFG_TUSB_OS == OPT_OS_NONE SysTick_Config(CONF_CPU_FREQUENCY / 1000); #elif CFG_TUSB_OS == OPT_OS_FREERTOS diff --git a/hw/bsp/same7x/boards/same70_xplained/board.h b/hw/bsp/same7x/boards/same70_xplained/board.h index 85e23deb8..86edf606f 100644 --- a/hw/bsp/same7x/boards/same70_xplained/board.h +++ b/hw/bsp/same7x/boards/same70_xplained/board.h @@ -52,6 +52,10 @@ extern "C" { #define UART_PORT_CLOCK ID_USART1 #define BOARD_USART USART1 +// TRACE_ETM: this board wires the KSZ8081 PHY reset to PC10; the family +// trace init holds it in reset (RMII rx lines share the trace pads) +#define TRACE_ETM_QUIET_ENET_PHY 1 + static inline void board_vbus_set(uint8_t rhport, bool state) { (void) rhport; (void) state; diff --git a/hw/bsp/same7x/boards/same70_xplained/ozone/same70.jdebug b/hw/bsp/same7x/boards/same70_xplained/ozone/same70.jdebug new file mode 100644 index 000000000..f024f0ceb --- /dev/null +++ b/hw/bsp/same7x/boards/same70_xplained/ozone/same70.jdebug @@ -0,0 +1,115 @@ +/********************************************************************* +* +* OnProjectLoad +* +* Function description +* Project load routine. Required. +* +* Notes +* SAM E70 Xplained: solder a 20-pin 50-mil header on the J403 ETM footprint +* (bottom side). TRACECLK=PD8 (peripheral D), TRACED0-3=PD4-7 (peripheral C), +* shared with the Ethernet PHY - no Ethernet while tracing. TRACE_ETM=1 +* builds mux the pins and clock the TPIU from PCK3=MCK; TPIU/ETM are +* ROM-table-discoverable so no J-Link script is required. +* Trace clock pin is PCK3/2 = 75 MHz at the stock 300 MHz core (MCK 150). +* +********************************************************************** +*/ +void OnProjectLoad (void) { + Project.SetTraceSource ("Trace Pins"); + Project.SetTracePortWidth (4); + Project.SetSWO (0); + Edit.SysVar (VAR_TRACE_CORE_CLOCK, 300000000); + Project.AddSvdFile ("$(InstallDir)/Config/CPU/Cortex-M7F.svd"); + + Project.SetDevice ("ATSAME70Q21B"); + Project.SetHostIF ("USB", ""); + Project.SetTargetIF ("SWD"); + Project.SetTIFSpeed ("4 MHz"); + + File.Open ("../../../../../../examples/cmake-build-same70_xplained/device/cdc_msc/cdc_msc.elf"); +} + +/********************************************************************* +* +* AfterTargetReset +* +* Function description +* Event handler routine. +* - Sets the PC register to program reset value. +* - Sets the SP register to program reset value on Cortex-M. +* +********************************************************************** +*/ +void AfterTargetReset (void) { + unsigned int SP; + unsigned int PC; + unsigned int VectorTableAddr; + + VectorTableAddr = Elf.GetBaseAddr(); + + if (VectorTableAddr == 0xFFFFFFFF) { + Util.Log("Project file error: failed to get program base"); + } else { + SP = Target.ReadU32(VectorTableAddr); + Target.SetReg("SP", SP); + + PC = Target.ReadU32(VectorTableAddr + 4); + Target.SetReg("PC", PC); + } + + // TPIU trace clock = PCK3; it must run BEFORE Ozone arms the trace + // components (TPIU programming while PCK3 is stopped is lost and the port + // emits unformatted garbage). A target reset wipes the PMC, so this runs + // in the post-reset/post-download hooks, not AfterTargetConnect. + Target.WriteU32 (0x400E064C, 0x00000014); // PMC_PCK3: CSS=MCK, PRESS=/2 + Target.WriteU32 (0x400E0600, 0x00000800); // PMC_SCER: PCK3 on + // wait for PCKRDY3 (bounded) before Ozone arms the trace components + int i; + i = 0; + while (((Target.ReadU32 (0x400E0668) & 0x00000800) == 0) && (i < 100)) { + i = i + 1; + } +} + +/********************************************************************* +* +* AfterTargetDownload +* +* Function description +* Event handler routine. +* - Sets the PC register to program reset value. +* - Sets the SP register to program reset value on Cortex-M. +* +********************************************************************** +*/ +void AfterTargetDownload (void) { + unsigned int SP; + unsigned int PC; + unsigned int VectorTableAddr; + + VectorTableAddr = Elf.GetBaseAddr(); + + if (VectorTableAddr == 0xFFFFFFFF) { + Util.Log("Project file error: failed to get program base"); + } else { + SP = Target.ReadU32(VectorTableAddr); + Target.SetReg("SP", SP); + + PC = Target.ReadU32(VectorTableAddr + 4); + Target.SetReg("PC", PC); + } + + // TPIU trace clock = PCK3; it must run BEFORE Ozone arms the trace + // components (TPIU programming while PCK3 is stopped is lost and the port + // emits unformatted garbage). A target reset wipes the PMC, so this runs + // in the post-reset/post-download hooks, not AfterTargetConnect. + Target.WriteU32 (0x400E064C, 0x00000014); // PMC_PCK3: CSS=MCK, PRESS=/2 + Target.WriteU32 (0x400E0600, 0x00000800); // PMC_SCER: PCK3 on + // wait for PCKRDY3 (bounded) before Ozone arms the trace components + int i; + i = 0; + while (((Target.ReadU32 (0x400E0668) & 0x00000800) == 0) && (i < 100)) { + i = i + 1; + } +} diff --git a/hw/bsp/same7x/family.c b/hw/bsp/same7x/family.c index d02e6c5f1..8ec6a708b 100644 --- a/hw/bsp/same7x/family.c +++ b/hw/bsp/same7x/family.c @@ -62,6 +62,38 @@ void board_init(void) { /* Disable Watchdog */ hri_wdt_set_MR_WDDIS_bit(WDT); +#if defined(TRACE_ETM) + // same70_xplained J403 (Cortex Debug+ETM footprint, bottom side) carries + // 4-bit trace: TRACECLK=PD8 (peripheral D), TRACED0-3=PD4-7 (peripheral C). +#ifdef TRACE_ETM_QUIET_ENET_PHY + // The trace pins double as the Ethernet PHY's RMII receive lines + // (PD4=CRS_DV, PD5/6=RXD0/1, PD7=RXER - PHY OUTPUTS): hold the KSZ8081 in + // reset or it drives against the trace stream. The reset net is a BOARD + // property (same70_xplained: PHY_RESET=PC10), hence the board.h gate. + _pmc_enable_periph_clock(ID_PIOC); + gpio_set_pin_level(GPIO(GPIO_PORTC, 10), false); + gpio_set_pin_direction(GPIO(GPIO_PORTC, 10), GPIO_DIRECTION_OUT); + gpio_set_pin_function(GPIO(GPIO_PORTC, 10), GPIO_PIN_FUNCTION_OFF); +#endif + + // The TPIU is clocked from PCK3 (datasheet 16.7.4) - run it from MCK. + // skip if the debugger already started PCK3 (reprogramming glitches the + // clock mid-stream and desyncs the decoder) + uint32_t const pck3 = PMC_PCK_CSS_MCK | PMC_PCK_PRES(1); // MCK/2 = 75 MHz -> 37.5 MHz pin + if (PMC->PMC_PCK[3] != pck3 || !(PMC->PMC_SR & PMC_SR_PCKRDY3)) { + PMC->PMC_PCK[3] = pck3; + PMC->PMC_SCER = PMC_SCER_PCK3; + while (!(PMC->PMC_SR & PMC_SR_PCKRDY3)) {} + } + _pmc_enable_periph_clock(ID_PIOD); + uint32_t const clk_pin = PIO_PD8D_TPIU_TRACECLK; + uint32_t const dat_pin = PIO_PD4C_TPIU_TRACED0 | PIO_PD5C_TPIU_TRACED1 | + PIO_PD6C_TPIU_TRACED2 | PIO_PD7C_TPIU_TRACED3; + PIOD->PIO_ABCDSR[0] = (PIOD->PIO_ABCDSR[0] | clk_pin) & ~dat_pin; // D=11, C=01 + PIOD->PIO_ABCDSR[1] |= clk_pin | dat_pin; + PIOD->PIO_PDR = clk_pin | dat_pin; // hand the pins to the peripheral +#endif + #ifdef LED_PIN _pmc_enable_periph_clock(LED_PORT_CLOCK); gpio_set_pin_level(LED_PIN, LED_STATE_OFF); diff --git a/hw/bsp/stm32h5/boards/stm32h563nucleo/board.h b/hw/bsp/stm32h5/boards/stm32h563nucleo/board.h index 18c13017a..959dc4828 100644 --- a/hw/bsp/stm32h5/boards/stm32h563nucleo/board.h +++ b/hw/bsp/stm32h5/boards/stm32h563nucleo/board.h @@ -88,7 +88,12 @@ static inline void SystemClock_Config(void) { RCC_OscInitStruct.PLL.PLLState = RCC_PLL_ON; RCC_OscInitStruct.PLL.PLLSource = RCC_PLL1_SOURCE_HSE; RCC_OscInitStruct.PLL.PLLM = 4; + #ifdef TRACE_ETM + RCC_OscInitStruct.PLL.PLLN = 100; // 100 MHz core: the Nucleo trace path (CN5 via solder bridges) + // corrupts the trace stream at higher TRACECLK (= SYSCLK/2) + #else RCC_OscInitStruct.PLL.PLLN = 250; + #endif RCC_OscInitStruct.PLL.PLLP = 2; RCC_OscInitStruct.PLL.PLLQ = 2; RCC_OscInitStruct.PLL.PLLR = 2; diff --git a/hw/bsp/stm32h5/boards/stm32h563nucleo/ozone/stm32h563.jdebug b/hw/bsp/stm32h5/boards/stm32h563nucleo/ozone/stm32h563.jdebug new file mode 100644 index 000000000..04f4e4582 --- /dev/null +++ b/hw/bsp/stm32h5/boards/stm32h563nucleo/ozone/stm32h563.jdebug @@ -0,0 +1,113 @@ +/********************************************************************* +* +* OnProjectLoad +* +* Function description +* Project load routine. Required. +* +* Notes +* Nucleo-H563ZI trace: PE2-PE6 reach the MIPI-20 connector (CN5) only +* with SB8, SB9, SB64, SB68, SB70, SB71, SB78 removed (MB1404 UM). +* Firmware must be built with TRACE_ETM=1 (trace pin + DBGMCU init). +* +* The trace path through the solder bridges is signal-marginal: reliable +* only at 100 MHz core (TRACE_ETM builds select this automatically in +* board.h), port width 1 and +5 ns sample timing (validated empirically; +* width 4 or 250 MHz core corrupts the stream within ~100 ms). +* +********************************************************************** +*/ +void OnProjectLoad (void) { + Project.SetTraceSource ("Trace Pins"); + Project.SetTracePortWidth (1); + Project.SetTraceTiming (5000, 5000, 5000, 5000); + Project.SetSWO (0); + Edit.SysVar (VAR_TRACE_CORE_CLOCK, 100000000); + Project.AddSvdFile ("$(InstallDir)/Config/CPU/Cortex-M33F.svd"); + + Project.SetDevice ("STM32H563ZI"); + Project.SetHostIF ("USB", ""); + Project.SetTargetIF ("SWD"); + Project.SetTIFSpeed ("4 MHz"); + + File.Open ("../../../../../../examples/cmake-build-stm32h563nucleo/device/cdc_msc/cdc_msc.elf"); +} + +/********************************************************************* +* +* AfterTargetConnect +* +* Function description +* Enable the trace clock domain (DBGMCU_CR: TRACE_IOEN | TRACE_CLKEN | +* TRACE_MODE=4-bit) BEFORE Ozone touches the trace CoreSight components. +* On STM32H5 an access to unclocked trace components hangs the debug AP +* ("Failed to read target status") until the board is power-cycled. +* +********************************************************************** +*/ +void AfterTargetConnect (void) { + unsigned int cr; + cr = Target.ReadU32(0x44024004); // DBGMCU_CR + // clock the domain (CLKEN|MODE=4-bit) but keep the pins OFF (clear IOEN): + // the pins must only go live via firmware trace_etm_init() AFTER the system + // clock switch, or the mid-trace frequency change desyncs the decoder. + Target.WriteU32(0x44024004, (cr & 0xFFFFFFEF) | 0xE0); +} + +/********************************************************************* +* +* AfterTargetReset +* +* Function description +* Event handler routine. +* - Sets the PC register to program reset value. +* - Sets the SP register to program reset value on Cortex-M. +* +********************************************************************** +*/ +void AfterTargetReset (void) { + unsigned int SP; + unsigned int PC; + unsigned int VectorTableAddr; + + VectorTableAddr = Elf.GetBaseAddr(); + + if (VectorTableAddr == 0xFFFFFFFF) { + Util.Log("Project file error: failed to get program base"); + } else { + SP = Target.ReadU32(VectorTableAddr); + Target.SetReg("SP", SP); + + PC = Target.ReadU32(VectorTableAddr + 4); + Target.SetReg("PC", PC); + } +} + +/********************************************************************* +* +* AfterTargetDownload +* +* Function description +* Event handler routine. +* - Sets the PC register to program reset value. +* - Sets the SP register to program reset value on Cortex-M. +* +********************************************************************** +*/ +void AfterTargetDownload (void) { + unsigned int SP; + unsigned int PC; + unsigned int VectorTableAddr; + + VectorTableAddr = Elf.GetBaseAddr(); + + if (VectorTableAddr == 0xFFFFFFFF) { + Util.Log("Project file error: failed to get program base"); + } else { + SP = Target.ReadU32(VectorTableAddr); + Target.SetReg("SP", SP); + + PC = Target.ReadU32(VectorTableAddr + 4); + Target.SetReg("PC", PC); + } +} diff --git a/hw/bsp/stm32h5/family.c b/hw/bsp/stm32h5/family.c index 867171734..36a95ac8c 100644 --- a/hw/bsp/stm32h5/family.c +++ b/hw/bsp/stm32h5/family.c @@ -99,6 +99,24 @@ static UART_HandleTypeDef UartHandle = { }; #endif +#ifdef TRACE_ETM +static void trace_etm_init(void) { + // H5 trace pins are PE2 to PE6 (Nucleo-144: requires trace solder-bridge config, see board docs) + GPIO_InitTypeDef gpio_init; + gpio_init.Pin = GPIO_PIN_2 | GPIO_PIN_3 | GPIO_PIN_4 | GPIO_PIN_5 | GPIO_PIN_6; + gpio_init.Mode = GPIO_MODE_AF_PP; + gpio_init.Pull = GPIO_PULLUP; + gpio_init.Speed = GPIO_SPEED_FREQ_VERY_HIGH; + gpio_init.Alternate = GPIO_AF0_TRACE; + HAL_GPIO_Init(GPIOE, &gpio_init); + + // Enable trace port + clock, synchronous 4-bit mode + DBGMCU->CR |= DBGMCU_CR_TRACE_IOEN | DBGMCU_CR_TRACE_CLKEN | DBGMCU_CR_TRACE_MODE; +} +#else +#define trace_etm_init() +#endif + void board_init(void) { // Cache UID before ICACHE is enabled (STM32H5 errata: reading UID_BASE with ICACHE causes hard fault) volatile uint32_t* stm32_uuid = (volatile uint32_t*) UID_BASE; @@ -127,6 +145,8 @@ void board_init(void) { __HAL_RCC_GPIOI_CLK_ENABLE(); #endif + trace_etm_init(); + #if CFG_TUSB_OS == OPT_OS_NONE // 1ms tick timer SysTick_Config(SystemCoreClock / 1000); diff --git a/hw/bsp/stm32h5/linker/stm32h533xx_flash.icf b/hw/bsp/stm32h5/linker/stm32h533xx_flash.icf new file mode 100644 index 000000000..dc97788ae --- /dev/null +++ b/hw/bsp/stm32h5/linker/stm32h533xx_flash.icf @@ -0,0 +1,32 @@ +/*###ICF### Section handled by ICF editor, don't touch! ****/ +/*-Editor annotation file-*/ +/* IcfEditorFile="$TOOLKIT_DIR$\config\ide\IcfEditor\cortex_v1_0.xml" */ +/*-Specials-*/ +define symbol __ICFEDIT_intvec_start__ = 0x08000000; +/*-Memory Regions-*/ +define symbol __ICFEDIT_region_ROM_start__ = 0x08000000; +define symbol __ICFEDIT_region_ROM_end__ = 0x0807FFFF; +define symbol __ICFEDIT_region_RAM_start__ = 0x20000000; +define symbol __ICFEDIT_region_RAM_end__ = 0x20043FFF; + +/*-Sizes-*/ +define symbol __ICFEDIT_size_cstack__ = 0x1000; +define symbol __ICFEDIT_size_heap__ = 0x200; +/**** End of ICF editor section. ###ICF###*/ + + +define memory mem with size = 4G; +define region ROM_region = mem:[from __ICFEDIT_region_ROM_start__ to __ICFEDIT_region_ROM_end__]; +define region RAM_region = mem:[from __ICFEDIT_region_RAM_start__ to __ICFEDIT_region_RAM_end__]; + +define block CSTACK with alignment = 8, size = __ICFEDIT_size_cstack__ { }; +define block HEAP with alignment = 8, size = __ICFEDIT_size_heap__ { }; + +initialize by copy { readwrite }; +do not initialize { section .noinit }; + +place at address mem:__ICFEDIT_intvec_start__ { readonly section .intvec }; + +place in ROM_region { readonly }; +place in RAM_region { readwrite, + block CSTACK, block HEAP }; diff --git a/hw/bsp/stm32h7/boards/stm32h743eval/ozone/stm32h743.jdebug b/hw/bsp/stm32h7/boards/stm32h743eval/ozone/stm32h743.jdebug index 32a8155c1..f9780147d 100644 --- a/hw/bsp/stm32h7/boards/stm32h743eval/ozone/stm32h743.jdebug +++ b/hw/bsp/stm32h7/boards/stm32h743eval/ozone/stm32h743.jdebug @@ -10,6 +10,7 @@ */ void OnProjectLoad (void) { Project.SetTraceSource ("Trace Pins"); + Project.SetTracePortWidth (4); Project.SetTraceTiming (100, 100, 100, 100); Project.SetSWO (0); Edit.SysVar (VAR_TRACE_CORE_CLOCK, 200000000); @@ -22,7 +23,7 @@ void OnProjectLoad (void) { Project.SetTargetIF ("SWD"); Project.SetTIFSpeed ("50 MHz"); - File.Open ("../../../../../../examples/device/cdc_msc/cmake-build-stm32h743eval/cdc_msc.elf"); + File.Open ("../../../../../../examples/cmake-build-stm32h743eval/device/cdc_msc/cdc_msc.elf"); // File.Open ("../../../../../../examples/cmake-build-stm32h743eval_host1/host/cdc_msc_hid/cdc_msc_hid.elf"); } diff --git a/hw/bsp/stm32h7rs/boards/stm32h7s3nucleo/board.h b/hw/bsp/stm32h7rs/boards/stm32h7s3nucleo/board.h index 996eb1515..098fc0bed 100644 --- a/hw/bsp/stm32h7rs/boards/stm32h7s3nucleo/board.h +++ b/hw/bsp/stm32h7rs/boards/stm32h7s3nucleo/board.h @@ -123,7 +123,13 @@ static inline void SystemClock_Config(void) RCC_OscInitStruct.PLL1.PLLState = RCC_PLL_ON; RCC_OscInitStruct.PLL1.PLLSource = RCC_PLLSOURCE_HSE; RCC_OscInitStruct.PLL1.PLLM = 12; +#ifdef TRACE_ETM + // 300 MHz core -> 50 MHz trace clock: at 600 MHz the stream survives idle + // but dies (unknown trace packet) during IRQ-heavy bursts, e.g. USB traffic + RCC_OscInitStruct.PLL1.PLLN = 150; +#else RCC_OscInitStruct.PLL1.PLLN = 300; +#endif RCC_OscInitStruct.PLL1.PLLP = 1; RCC_OscInitStruct.PLL1.PLLQ = 2; RCC_OscInitStruct.PLL1.PLLR = 2; diff --git a/hw/bsp/stm32h7rs/boards/stm32h7s3nucleo/ozone/stm32h7s3.jdebug b/hw/bsp/stm32h7rs/boards/stm32h7s3nucleo/ozone/stm32h7s3.jdebug new file mode 100644 index 000000000..f6658d2d7 --- /dev/null +++ b/hw/bsp/stm32h7rs/boards/stm32h7s3nucleo/ozone/stm32h7s3.jdebug @@ -0,0 +1,94 @@ +/********************************************************************* +* +* OnProjectLoad +* +* Function description +* Project load routine. Required. +* +* Notes +* Nucleo-H7S3L8 wires 4-bit trace to the CN1 MIPI20 natively (no rework): +* TRACE_CLK/D0 = PE2/PE3, TRACE_D1/D2/D3 = PG14/PD2/PC12 (MB1737 Table 7). +* Firmware must be built with TRACE_ETM=1 (trace pin mux + DBGMCU +* DBGCKEN/TRACECLKEN in board_init). +* +* TRACE_ETM builds run a 300 MHz core (board.h) -> 50 MHz trace clock +* (cpu/3/2). Width 2 on the stub-free D0/D1 lines: SB11/SB12 (default ON) +* stub D2/D3 onto Zio CN8 and the 4-bit stream dies under IRQ-heavy USB +* traffic (idle is clean) - remove SB11/SB12 to try width 4 / 600 MHz. +* +********************************************************************** +*/ +void OnProjectLoad (void) { + // declares the off-ROM-table CSTF/TMC/TPIU (see the script header) + Project.SetJLinkScript ("./stm32h7s3_trace.JLinkScript"); + Project.SetTraceSource ("Trace Pins"); + Project.SetTracePortWidth (2); + Project.SetSWO (0); + Edit.SysVar (VAR_TRACE_CORE_CLOCK, 300000000); + Project.AddSvdFile ("$(InstallDir)/Config/CPU/Cortex-M7F.svd"); + + Project.SetDevice ("STM32H7S3L8"); + Project.SetHostIF ("USB", ""); + Project.SetTargetIF ("SWD"); + Project.SetTIFSpeed ("4 MHz"); + + File.Open ("../../../../../../examples/cmake-build-stm32h7s3nucleo/device/cdc_msc/cdc_msc.elf"); +} + +/********************************************************************* +* +* AfterTargetReset +* +* Function description +* Event handler routine. +* - Sets the PC register to program reset value. +* - Sets the SP register to program reset value on Cortex-M. +* +********************************************************************** +*/ +void AfterTargetReset (void) { + unsigned int SP; + unsigned int PC; + unsigned int VectorTableAddr; + + VectorTableAddr = Elf.GetBaseAddr(); + + if (VectorTableAddr == 0xFFFFFFFF) { + Util.Log("Project file error: failed to get program base"); + } else { + SP = Target.ReadU32(VectorTableAddr); + Target.SetReg("SP", SP); + + PC = Target.ReadU32(VectorTableAddr + 4); + Target.SetReg("PC", PC); + } +} + +/********************************************************************* +* +* AfterTargetDownload +* +* Function description +* Event handler routine. +* - Sets the PC register to program reset value. +* - Sets the SP register to program reset value on Cortex-M. +* +********************************************************************** +*/ +void AfterTargetDownload (void) { + unsigned int SP; + unsigned int PC; + unsigned int VectorTableAddr; + + VectorTableAddr = Elf.GetBaseAddr(); + + if (VectorTableAddr == 0xFFFFFFFF) { + Util.Log("Project file error: failed to get program base"); + } else { + SP = Target.ReadU32(VectorTableAddr); + Target.SetReg("SP", SP); + + PC = Target.ReadU32(VectorTableAddr + 4); + Target.SetReg("PC", PC); + } +} diff --git a/hw/bsp/stm32h7rs/boards/stm32h7s3nucleo/ozone/stm32h7s3_trace.JLinkScript b/hw/bsp/stm32h7rs/boards/stm32h7s3nucleo/ozone/stm32h7s3_trace.JLinkScript new file mode 100644 index 000000000..5ef4f164d --- /dev/null +++ b/hw/bsp/stm32h7rs/boards/stm32h7s3nucleo/ozone/stm32h7s3_trace.JLinkScript @@ -0,0 +1,11 @@ +/* STM32H7RS pin trace: the CSTF funnel, TMC (ETF) and TPIU are not in the + * core ROM table - declare them or J-Link aborts trace init with "Required + * trace components for pin trace not found!" (addresses per SEGGER's + * NUCLEO-H7S3L8 trace example script; AP0). + */ +int ConfigTargetSettings(void) { + JLINK_ExecCommand("CORESIGHT_SetCSTFBaseAddr = 0x5C013000 ForceUnlock = 1 APIndex = 0"); + JLINK_ExecCommand("CORESIGHT_SetTMCBaseAddr = 0x5C014000 ForceUnlock = 1 APIndex = 0"); + JLINK_ExecCommand("CORESIGHT_SetTPIUBaseAddr = 0x5C015000 ForceUnlock = 1 APIndex = 0"); + return 0; +} diff --git a/hw/bsp/stm32h7rs/family.c b/hw/bsp/stm32h7rs/family.c index b0841c947..385b3c929 100644 --- a/hw/bsp/stm32h7rs/family.c +++ b/hw/bsp/stm32h7rs/family.c @@ -98,17 +98,24 @@ void OTG_HS_IRQHandler(void) { #ifdef TRACE_ETM void trace_etm_init(void) { - // H7 trace pin is PE2 to PE6 - GPIO_InitTypeDef gpio_init; - gpio_init.Pin = GPIO_PIN_2 | GPIO_PIN_3 | GPIO_PIN_4 | GPIO_PIN_5 | GPIO_PIN_6; + // Nucleo-H7S3L8 routes 4-bit trace to the CN1 MIPI20: TRACE_CLK/D0 on + // PE2/PE3, TRACE_D1/D2/D3 on the PG14/PD2/PC12 alternates (MB1737 Table 7). + // No pull: pull-ups degrade the edges at the 100 MHz trace clock (= cpu/3/2) + GPIO_InitTypeDef gpio_init; gpio_init.Mode = GPIO_MODE_AF_PP; - gpio_init.Pull = GPIO_PULLUP; + gpio_init.Pull = GPIO_NOPULL; gpio_init.Speed = GPIO_SPEED_FREQ_VERY_HIGH; gpio_init.Alternate = GPIO_AF0_TRACE; + gpio_init.Pin = GPIO_PIN_2 | GPIO_PIN_3; HAL_GPIO_Init(GPIOE, &gpio_init); + gpio_init.Pin = GPIO_PIN_14; + HAL_GPIO_Init(GPIOG, &gpio_init); + gpio_init.Pin = GPIO_PIN_2; + HAL_GPIO_Init(GPIOD, &gpio_init); + gpio_init.Pin = GPIO_PIN_12; + HAL_GPIO_Init(GPIOC, &gpio_init); - // Enable trace clk, also in D1 and D3 domain - DBGMCU->CR |= DBGMCU_CR_DBG_TRACECKEN | DBGMCU_CR_DBG_CKD1EN | DBGMCU_CR_DBG_CKD3EN; + DBGMCU->CR |= DBGMCU_CR_DBGCKEN | DBGMCU_CR_TRACECLKEN; } #else #define trace_etm_init() diff --git a/hw/bsp/stm32l4/boards/stm32l412nucleo/board.h b/hw/bsp/stm32l4/boards/stm32l412nucleo/board.h index a5250eda9..7f63ec431 100644 --- a/hw/bsp/stm32l4/boards/stm32l412nucleo/board.h +++ b/hw/bsp/stm32l4/boards/stm32l412nucleo/board.h @@ -64,9 +64,10 @@ * AHB Prescaler = 1 * APB1 Prescaler = 1 * APB2 Prescaler = 1 - * MSI Frequency(Hz) = 8000000 - * PLL_M = 1 - * PLL_N = 10 + * MSI Frequency(Hz) = 48000000 + * LSE Frequency(Hz) = 32768 + * PLL_M = 6 + * PLL_N = 20 * PLL_Q = 2 * PLL_R = 2 * VDD(V) = 3.3 @@ -78,29 +79,35 @@ static inline void board_clock_init(void) { RCC_OscInitTypeDef RCC_OscInitStruct = {0}; RCC_ClkInitTypeDef RCC_ClkInitStruct = {0}; - RCC_CRSInitTypeDef RCC_CRSInitStruct = {0}; RCC_PeriphCLKInitTypeDef PeriphClkInitStruct = {0}; /** Configure the main internal regulator output voltage */ HAL_PWREx_ControlVoltageScaling(PWR_REGULATOR_VOLTAGE_SCALE1); + /* HAL clock setup reconfigures its tick while MSI is the reset SYSCLK. */ + HAL_InitTick((1UL << __NVIC_PRIO_BITS) - 1UL); + /** Initializes the RCC Oscillators according to the specified parameters * in the RCC_OscInitTypeDef structure. */ - RCC_OscInitStruct.OscillatorType = RCC_OSCILLATORTYPE_HSI48|RCC_OSCILLATORTYPE_HSI; - RCC_OscInitStruct.HSIState = RCC_HSI_ON; - RCC_OscInitStruct.HSI48State = RCC_HSI48_ON; - RCC_OscInitStruct.HSICalibrationValue = RCC_HSICALIBRATION_DEFAULT; + RCC_OscInitStruct.OscillatorType = RCC_OSCILLATORTYPE_LSE | RCC_OSCILLATORTYPE_MSI; + RCC_OscInitStruct.LSEState = RCC_LSE_ON; + RCC_OscInitStruct.MSIState = RCC_MSI_ON; + RCC_OscInitStruct.MSICalibrationValue = RCC_MSICALIBRATION_DEFAULT; + RCC_OscInitStruct.MSIClockRange = RCC_MSIRANGE_11; RCC_OscInitStruct.PLL.PLLState = RCC_PLL_ON; - RCC_OscInitStruct.PLL.PLLSource = RCC_PLLSOURCE_HSI; - RCC_OscInitStruct.PLL.PLLM = 1; - RCC_OscInitStruct.PLL.PLLN = 10; + RCC_OscInitStruct.PLL.PLLSource = RCC_PLLSOURCE_MSI; + RCC_OscInitStruct.PLL.PLLM = 6; + RCC_OscInitStruct.PLL.PLLN = 20; RCC_OscInitStruct.PLL.PLLQ = RCC_PLLQ_DIV2; RCC_OscInitStruct.PLL.PLLR = RCC_PLLR_DIV2; HAL_RCC_OscConfig(&RCC_OscInitStruct); + /* Stabilize MSI against the on-board 32.768 kHz LSE crystal. */ + HAL_RCCEx_EnableMSIPLLMode(); + /** Initializes the CPU, AHB and APB buses clocks */ RCC_ClkInitStruct.ClockType = RCC_CLOCKTYPE_HCLK|RCC_CLOCKTYPE_SYSCLK @@ -112,24 +119,9 @@ static inline void board_clock_init(void) HAL_RCC_ClockConfig(&RCC_ClkInitStruct, FLASH_LATENCY_4); - /** Enable the SYSCFG APB clock - */ - __HAL_RCC_CRS_CLK_ENABLE(); - - /** Configures CRS - */ - RCC_CRSInitStruct.Prescaler = RCC_CRS_SYNC_DIV1; - RCC_CRSInitStruct.Source = RCC_CRS_SYNC_SOURCE_USB; - RCC_CRSInitStruct.Polarity = RCC_CRS_SYNC_POLARITY_RISING; - RCC_CRSInitStruct.ReloadValue = __HAL_RCC_CRS_RELOADVALUE_CALCULATE(48000000,1000); - RCC_CRSInitStruct.ErrorLimitValue = 34; - RCC_CRSInitStruct.HSI48CalibrationValue = 32; - - HAL_RCCEx_CRSConfig(&RCC_CRSInitStruct); - - /* Select HSI48 output as USB clock source */ + /* Use the same LSE-trimmed MSI source for USB and the CPU PLL. */ PeriphClkInitStruct.PeriphClockSelection = RCC_PERIPHCLK_USB; - PeriphClkInitStruct.UsbClockSelection = RCC_USBCLKSOURCE_HSI48; + PeriphClkInitStruct.UsbClockSelection = RCC_USBCLKSOURCE_MSI; HAL_RCCEx_PeriphCLKConfig(&PeriphClkInitStruct); /* Select PLL output as UART clock source */ diff --git a/hw/bsp/stm32n6/boards/stm32n657nucleo/board.h b/hw/bsp/stm32n6/boards/stm32n657nucleo/board.h index be9ea7a31..873c004d9 100644 --- a/hw/bsp/stm32n6/boards/stm32n657nucleo/board.h +++ b/hw/bsp/stm32n6/boards/stm32n657nucleo/board.h @@ -152,7 +152,13 @@ static void SystemClock_Config(void) { RCC_CLOCKTYPE_PCLK1 | RCC_CLOCKTYPE_PCLK2 | RCC_CLOCKTYPE_PCLK4 | RCC_CLOCKTYPE_PCLK5); RCC_ClkInitStruct.CPUCLKSource = RCC_CPUCLKSOURCE_IC1; RCC_ClkInitStruct.IC1Selection.ClockSelection = RCC_ICCLKSOURCE_PLL1; +#ifdef TRACE_ETM + // 300 MHz CPU -> 37.5 MHz TPIU clock (fixed cpu/8): at 600 MHz the trace + // stream dies with unknown-packet decode errors in the startup burst + RCC_ClkInitStruct.IC1Selection.ClockDivider = 4; +#else RCC_ClkInitStruct.IC1Selection.ClockDivider = 2; +#endif RCC_ClkInitStruct.SYSCLKSource = RCC_SYSCLKSOURCE_IC2_IC6_IC11; RCC_ClkInitStruct.IC2Selection.ClockSelection = RCC_ICCLKSOURCE_PLL1; RCC_ClkInitStruct.IC2Selection.ClockDivider = 3; diff --git a/hw/bsp/stm32n6/boards/stm32n657nucleo/ozone/stm32n657.jdebug b/hw/bsp/stm32n6/boards/stm32n657nucleo/ozone/stm32n657.jdebug new file mode 100644 index 000000000..16eead280 --- /dev/null +++ b/hw/bsp/stm32n6/boards/stm32n657nucleo/ozone/stm32n657.jdebug @@ -0,0 +1,94 @@ +/********************************************************************* +* +* OnProjectLoad +* +* Function description +* Project load routine. Required. +* +* Notes +* Nucleo-N657X0-Q wires 4-bit trace to the CN1 MIPI20 natively (SB36/38/ +* 39/40/41 fitted by default): TRACE_CLK = PB3, D0/D1/D2/D3 = +* PE3/PB0/PB6/PB7 (MB1940 Table 5). No J-Link script needed - the N6 +* trace components sit behind a ROM table J-Link discovers natively. +* +* BOOT1 (JP2) must select Development boot (BOOT1 = 1): the N657 is +* flashless, the app is a RAM image (AXISRAM2) loaded by the debugger, +* and in flash boot the bootROM parks the chip un-attachable. +* Firmware must be built with TRACE_ETM=1 (pin mux + DBGMCU +* DBGCLKEN/TRACECLKEN). Trace clock = cpu/8. TRACE_ETM builds run a 300 MHz core (board.h) -> +* 37.5 MHz TPIU clock: 600 MHz kills the stream in the startup burst. +* +********************************************************************** +*/ +void OnProjectLoad (void) { + Project.SetTraceSource ("Trace Pins"); + Project.SetTracePortWidth (4); + Project.SetSWO (0); + Edit.SysVar (VAR_TRACE_CORE_CLOCK, 300000000); + Project.AddSvdFile ("$(InstallDir)/Config/CPU/Cortex-M55F.svd"); + + Project.SetDevice ("STM32N657X0"); + Project.SetHostIF ("USB", ""); + Project.SetTargetIF ("SWD"); + Project.SetTIFSpeed ("4 MHz"); + + File.Open ("../../../../../../examples/cmake-build-stm32n657nucleo/device/cdc_msc/cdc_msc.elf"); +} + +/********************************************************************* +* +* AfterTargetReset +* +* Function description +* Event handler routine. +* - Sets the PC register to program reset value. +* - Sets the SP register to program reset value on Cortex-M. +* +********************************************************************** +*/ +void AfterTargetReset (void) { + unsigned int SP; + unsigned int PC; + unsigned int VectorTableAddr; + + VectorTableAddr = Elf.GetBaseAddr(); + + if (VectorTableAddr == 0xFFFFFFFF) { + Util.Log("Project file error: failed to get program base"); + } else { + SP = Target.ReadU32(VectorTableAddr); + Target.SetReg("SP", SP); + + PC = Target.ReadU32(VectorTableAddr + 4); + Target.SetReg("PC", PC); + } +} + +/********************************************************************* +* +* AfterTargetDownload +* +* Function description +* Event handler routine. +* - Sets the PC register to program reset value. +* - Sets the SP register to program reset value on Cortex-M. +* +********************************************************************** +*/ +void AfterTargetDownload (void) { + unsigned int SP; + unsigned int PC; + unsigned int VectorTableAddr; + + VectorTableAddr = Elf.GetBaseAddr(); + + if (VectorTableAddr == 0xFFFFFFFF) { + Util.Log("Project file error: failed to get program base"); + } else { + SP = Target.ReadU32(VectorTableAddr); + Target.SetReg("SP", SP); + + PC = Target.ReadU32(VectorTableAddr + 4); + Target.SetReg("PC", PC); + } +} diff --git a/hw/bsp/stm32n6/family.c b/hw/bsp/stm32n6/family.c index 80de20c6a..9d3faa1f2 100644 --- a/hw/bsp/stm32n6/family.c +++ b/hw/bsp/stm32n6/family.c @@ -110,6 +110,27 @@ void USB1_OTG_HS_IRQHandler(void) { tusb_int_handler(0, true); } +#ifdef TRACE_ETM +static void trace_etm_init(void) { + // Nucleo-N657X0-Q routes 4-bit trace to the CN1 MIPI20 natively: + // TRACE_CLK = PB3, D0 = PE3, D1 = PB0, D2 = PB6, D3 = PB7 (MB1940 Table 5) + GPIO_InitTypeDef gpio_init; + gpio_init.Mode = GPIO_MODE_AF_PP; + gpio_init.Pull = GPIO_NOPULL; + gpio_init.Speed = GPIO_SPEED_FREQ_VERY_HIGH; + gpio_init.Alternate = GPIO_AF0_TRACE; + gpio_init.Pin = GPIO_PIN_0 | GPIO_PIN_3 | GPIO_PIN_6 | GPIO_PIN_7; + HAL_GPIO_Init(GPIOB, &gpio_init); + gpio_init.Pin = GPIO_PIN_3; + HAL_GPIO_Init(GPIOE, &gpio_init); + + // trace clock (ck_cpu_tpiu) is a fixed cpu/8 - just enable it + DBGMCU->CR |= DBGMCU_CR_DBGCLKEN | DBGMCU_CR_TRACECLKEN; +} +#else + #define trace_etm_init() +#endif + void board_init(void) { /* Enable BusFault and SecureFault handlers (HardFault is default) */ SCB->SHCSR |= (SCB_SHCSR_BUSFAULTENA_Msk | SCB_SHCSR_SECUREFAULTENA_Msk); @@ -148,6 +169,7 @@ void board_init(void) { for (uint8_t i = 0; i < TU_ARRAY_SIZE(board_pindef); i++) { HAL_GPIO_Init(board_pindef[i].port, &board_pindef[i].pin_init); } + trace_etm_init(); NVIC_SetPriority(UCPD1_IRQn, NVIC_EncodePriority(NVIC_GetPriorityGrouping(),5, 0)); NVIC_EnableIRQ(UCPD1_IRQn); diff --git a/src/class/audio/audio_device.c b/src/class/audio/audio_device.c index 12e82190e..bc4c7e544 100644 --- a/src/class/audio/audio_device.c +++ b/src/class/audio/audio_device.c @@ -1161,9 +1161,6 @@ static bool audiod_set_interface(uint8_t rhport, tusb_control_request_t const *p is_feedback_ep = (desc_ep->bmAttributes.usage == 1); } - //TODO: We need to set EP non busy since this is not taken care of right now in ep_close() - THIS IS A WORKAROUND! - usbd_edpt_clear_stall(rhport, ep_addr); - #if CFG_TUD_AUDIO_ENABLE_EP_IN // For data or data with implicit feedback IN EP if (tu_edpt_dir(ep_addr) == TUSB_DIR_IN && is_data_ep) @@ -1844,7 +1841,7 @@ static bool audiod_calc_tx_packet_sz(audiod_function_t *audio) { static uint16_t audiod_tx_packet_size(const uint16_t *nominal_size, uint16_t data_count, uint16_t fifo_depth, uint16_t fifo_threshold, uint16_t max_depth) { // Flow control need a FIFO size of at least 4*Navg - if (nominal_size[1] && nominal_size[1] <= fifo_depth * 4) { + if (nominal_size[1] && nominal_size[1] * 4 <= fifo_depth) { // Use blackout to prioritize normal size packet static int ctrl_blackout = 0; uint16_t packet_size; diff --git a/src/class/cdc/cdc_host.c b/src/class/cdc/cdc_host.c index 2a452a43c..7a979f649 100644 --- a/src/class/cdc/cdc_host.c +++ b/src/class/cdc/cdc_host.c @@ -714,9 +714,13 @@ bool cdch_xfer_cb(uint8_t daddr, uint8_t ep_addr, xfer_result_t event, uint32_t //--------------------------------------------------------------------+ // Enumeration //--------------------------------------------------------------------+ + static bool open_ep_stream_pair(cdch_interface_t *p_cdc, tusb_desc_endpoint_t const *desc_ep) { for (size_t i = 0; i < 2; i++) { - TU_ASSERT(TUSB_DESC_ENDPOINT == desc_ep->bDescriptorType && TUSB_XFER_BULK == desc_ep->bmAttributes.xfer, 0); + // pin bLength so tu_desc_next() below cannot walk the second endpoint past a caller-checked bound + TU_ASSERT(TUH_VALIDATE_BASIC(sizeof(tusb_desc_endpoint_t) == desc_ep->bLength) && + TUSB_DESC_ENDPOINT == desc_ep->bDescriptorType && TUSB_XFER_BULK == desc_ep->bmAttributes.xfer, + 0); TU_ASSERT(tuh_edpt_open(p_cdc->daddr, desc_ep)); const uint8_t ep_dir = tu_edpt_dir(desc_ep->bEndpointAddress); tu_edpt_stream_t *stream = (ep_dir == TUSB_DIR_IN) ? &p_cdc->stream.rx : &p_cdc->stream.tx; @@ -1012,6 +1016,8 @@ static uint16_t acm_open(uint8_t daddr, const tusb_desc_interface_t *itf_desc, u const uint8_t *p_desc = (const uint8_t *)itf_desc; const uint8_t *desc_end = p_desc + max_len; + TU_ASSERT(TUH_VALIDATE_BASIC(tu_desc_len(p_desc) <= max_len), 0); + cdch_interface_t *p_cdc = make_new_itf(daddr, itf_desc); TU_VERIFY(p_cdc, 0); p_cdc->serial_drid = SERIAL_DRIVER_ACM; @@ -1020,8 +1026,15 @@ static uint16_t acm_open(uint8_t daddr, const tusb_desc_interface_t *itf_desc, u p_desc = tu_desc_next(p_desc); // Communication Functional Descriptors - while ((p_desc < desc_end) && (TUSB_DESC_CS_INTERFACE == tu_desc_type(p_desc))) { - if (CDC_FUNC_DESC_ABSTRACT_CONTROL_MANAGEMENT == cdc_functional_desc_typeof(p_desc)) { + // need the 3-byte header (bLength/bDescriptorType/bDescriptorSubType) in bounds before reading it, and a + // fully contained bLength >= 3 both keeps those reads valid and stops a zero-length descriptor from spinning + while ((p_desc < desc_end) && + TUH_VALIDATE_BASIC((size_t)(desc_end - p_desc) >= 3) && + TUSB_DESC_CS_INTERFACE == tu_desc_type(p_desc) && + TUH_VALIDATE_BASIC(tu_desc_len(p_desc) >= 3) && + TUH_VALIDATE_BASIC(tu_desc_len(p_desc) <= (size_t)(desc_end - p_desc))) { + if (CDC_FUNC_DESC_ABSTRACT_CONTROL_MANAGEMENT == cdc_functional_desc_typeof(p_desc) && + TUH_VALIDATE_BASIC(tu_desc_len(p_desc) >= sizeof(cdc_desc_func_acm_t))) { // save ACM bmCapabilities p_cdc->acm.capability = ((cdc_desc_func_acm_t const *) p_desc)->bmCapabilities; } @@ -1031,23 +1044,29 @@ static uint16_t acm_open(uint8_t daddr, const tusb_desc_interface_t *itf_desc, u // Open notification endpoint of control interface if any if (itf_desc->bNumEndpoints == 1) { + // whole endpoint descriptor must fit: tuh_edpt_open reads the full struct regardless of bLength + TU_ASSERT(TUH_VALIDATE_BASIC((size_t)(desc_end - p_desc) >= sizeof(tusb_desc_endpoint_t)), 0); TU_ASSERT(TUSB_DESC_ENDPOINT == tu_desc_type(p_desc), 0); const tusb_desc_endpoint_t *desc_ep = (const tusb_desc_endpoint_t *)p_desc; TU_ASSERT(tuh_edpt_open(daddr, desc_ep), 0); p_cdc->ep_notif = desc_ep->bEndpointAddress; - p_desc = tu_desc_next(p_desc); + // advance by the fixed struct size, not device-supplied bLength, so p_desc stays inside the checked window + p_desc += sizeof(tusb_desc_endpoint_t); } //------------- Data Interface (if any) -------------// - if (TUSB_DESC_INTERFACE == tu_desc_type(p_desc)) { + if (TUH_VALIDATE_BASIC((size_t)(desc_end - p_desc) >= sizeof(tusb_desc_interface_t)) && + TUSB_DESC_INTERFACE == tu_desc_type(p_desc)) { const tusb_desc_interface_t *data_itf = (const tusb_desc_interface_t *)p_desc; if (data_itf->bInterfaceClass == TUSB_CLASS_CDC_DATA) { - p_desc = tu_desc_next(p_desc); // next to endpoint descriptor + p_desc += sizeof(tusb_desc_interface_t); // fixed struct size to endpoint descriptor, not device bLength - // data endpoints expected to be in pairs + // open_ep_stream_pair consumes exactly two endpoints; require that count and that both fit before reading them + TU_ASSERT(TUH_VALIDATE_BASIC(data_itf->bNumEndpoints == 2), 0); + TU_ASSERT(TUH_VALIDATE_BASIC((size_t)(desc_end - p_desc) >= 2 * sizeof(tusb_desc_endpoint_t)), 0); TU_ASSERT(open_ep_stream_pair(p_cdc, (const tusb_desc_endpoint_t *)p_desc), 0); - p_desc += data_itf->bNumEndpoints * sizeof(tusb_desc_endpoint_t); + p_desc += 2 * sizeof(tusb_desc_endpoint_t); } } @@ -1189,6 +1208,7 @@ static uint16_t ftdi_open(uint8_t daddr, const tusb_desc_interface_t *itf_desc, TU_VERIFY(itf_desc->bInterfaceSubClass == 0xff && itf_desc->bInterfaceProtocol == 0xff && itf_desc->bNumEndpoints == 2, 0); + TU_VERIFY(TUH_VALIDATE_BASIC(itf_desc->bLength == sizeof(tusb_desc_interface_t)), 0); const uint16_t drv_len = (uint16_t)(sizeof(tusb_desc_interface_t) + itf_desc->bNumEndpoints * sizeof(tusb_desc_endpoint_t)); TU_VERIFY(drv_len <= max_len, 0); @@ -1574,6 +1594,7 @@ enum { static uint16_t cp210x_open(uint8_t daddr, const tusb_desc_interface_t *itf_desc, uint16_t max_len) { // CP210x Interface includes 1 vendor interface + 2 bulk endpoints TU_VERIFY(itf_desc->bInterfaceSubClass == 0 && itf_desc->bInterfaceProtocol == 0 && itf_desc->bNumEndpoints == 2, 0); + TU_VERIFY(TUH_VALIDATE_BASIC(itf_desc->bLength == sizeof(tusb_desc_interface_t)), 0); const uint16_t drv_len = (uint16_t)(sizeof(tusb_desc_interface_t) + itf_desc->bNumEndpoints * sizeof(tusb_desc_endpoint_t)); TU_VERIFY(drv_len <= max_len, 0); @@ -1745,6 +1766,7 @@ enum { static uint16_t ch34x_open(uint8_t daddr, const tusb_desc_interface_t *itf_desc, uint16_t max_len) { // CH34x Interface includes 1 vendor interface + 2 bulk + 1 interrupt endpoints TU_VERIFY(itf_desc->bNumEndpoints == 3, 0); + TU_VERIFY(TUH_VALIDATE_BASIC(itf_desc->bLength == sizeof(tusb_desc_interface_t)), 0); const uint16_t drv_len = (uint16_t)(sizeof(tusb_desc_interface_t) + itf_desc->bNumEndpoints * sizeof(tusb_desc_endpoint_t)); TU_VERIFY(drv_len <= max_len, 0); @@ -2081,6 +2103,7 @@ enum { static uint16_t pl2303_open(uint8_t daddr, const tusb_desc_interface_t *itf_desc, uint16_t max_len) { // PL2303 Interface includes 1 vendor interface + 1 interrupt endpoints + 2 bulk TU_VERIFY(itf_desc->bNumEndpoints == 3, 0); + TU_VERIFY(TUH_VALIDATE_BASIC(itf_desc->bLength == sizeof(tusb_desc_interface_t)), 0); const uint16_t drv_len = (uint16_t)(sizeof(tusb_desc_interface_t) + itf_desc->bNumEndpoints * sizeof(tusb_desc_endpoint_t)); TU_VERIFY(drv_len <= max_len, 0); diff --git a/src/class/hid/hid.h b/src/class/hid/hid.h index 50a206ced..a6d1dba15 100644 --- a/src/class/hid/hid.h +++ b/src/class/hid/hid.h @@ -368,7 +368,7 @@ typedef enum /// @} //--------------------------------------------------------------------+ -// HID KEYCODE +// HID KEYCODE - defined by HID Usage Table: Keyboard/Keypad Page (0x07) //--------------------------------------------------------------------+ #define HID_KEY_NONE 0x00 #define HID_KEY_A 0x04 @@ -772,11 +772,12 @@ enum { HID_USAGE_PAGE_KEYBOARD = 0x07, HID_USAGE_PAGE_LED = 0x08, HID_USAGE_PAGE_BUTTON = 0x09, - HID_USAGE_PAGE_ORDINAL = 0x0a, - HID_USAGE_PAGE_TELEPHONY = 0x0b, - HID_USAGE_PAGE_CONSUMER = 0x0c, - HID_USAGE_PAGE_DIGITIZER = 0x0d, - HID_USAGE_PAGE_PID = 0x0f, + HID_USAGE_PAGE_ORDINAL = 0x0A, + HID_USAGE_PAGE_TELEPHONY = 0x0B, + HID_USAGE_PAGE_CONSUMER = 0x0C, + HID_USAGE_PAGE_DIGITIZER = 0x0D, + HID_USAGE_PAGE_HAPTIC = 0x0E, + HID_USAGE_PAGE_PID = 0x0F, HID_USAGE_PAGE_UNICODE = 0x10, HID_USAGE_PAGE_SOC = 0x11, HID_USAGE_PAGE_EYE_AND_HEAD_TRACKERS = 0x12, @@ -786,528 +787,1091 @@ enum { HID_USAGE_PAGE_SENSORS = 0x20, // 0x21 - 0x3f is reserved HID_USAGE_PAGE_MEDICAL_INSTRUMENT = 0x40, + HID_USAGE_PAGE_BRAILLE_DISPLAY = 0x41, HID_USAGE_PAGE_LIGHTING_AND_ILLUMINATION = 0x59, HID_USAGE_PAGE_MONITOR = 0x80, // 0x80 - 0x83 HID_USAGE_PAGE_POWER = 0x84, HID_USAGE_PAGE_BATTERY = 0x85, // 0x86 - 0x87 is reserved for Power Device - HID_USAGE_PAGE_BARCODE_SCANNER = 0x8c, - HID_USAGE_PAGE_SCALE = 0x8d, - HID_USAGE_PAGE_MSR = 0x8e, + HID_USAGE_PAGE_BARCODE_SCANNER = 0x8C, + HID_USAGE_PAGE_SCALE = 0x8D, + HID_USAGE_PAGE_MSR = 0x8E, HID_USAGE_PAGE_CAMERA = 0x90, HID_USAGE_PAGE_ARCADE = 0x91, + HID_USAGE_PAGE_GAMING = 0x92, // Gaming Standards Association (GSA) HID usage page HID_USAGE_PAGE_FIDO = 0xF1D0, // FIDO alliance HID usage page HID_USAGE_PAGE_VENDOR = 0xFF00 // 0xFF00 - 0xFFFF }; /// HID Usage Table - Table 6: Generic Desktop Page enum { - HID_USAGE_DESKTOP_POINTER = 0x01, - HID_USAGE_DESKTOP_MOUSE = 0x02, - HID_USAGE_DESKTOP_JOYSTICK = 0x04, - HID_USAGE_DESKTOP_GAMEPAD = 0x05, - HID_USAGE_DESKTOP_KEYBOARD = 0x06, - HID_USAGE_DESKTOP_KEYPAD = 0x07, - HID_USAGE_DESKTOP_MULTI_AXIS_CONTROLLER = 0x08, - HID_USAGE_DESKTOP_TABLET_PC_SYSTEM = 0x09, - HID_USAGE_DESKTOP_X = 0x30, - HID_USAGE_DESKTOP_Y = 0x31, - HID_USAGE_DESKTOP_Z = 0x32, - HID_USAGE_DESKTOP_RX = 0x33, - HID_USAGE_DESKTOP_RY = 0x34, - HID_USAGE_DESKTOP_RZ = 0x35, - HID_USAGE_DESKTOP_SLIDER = 0x36, - HID_USAGE_DESKTOP_DIAL = 0x37, - HID_USAGE_DESKTOP_WHEEL = 0x38, - HID_USAGE_DESKTOP_HAT_SWITCH = 0x39, - HID_USAGE_DESKTOP_COUNTED_BUFFER = 0x3a, - HID_USAGE_DESKTOP_BYTE_COUNT = 0x3b, - HID_USAGE_DESKTOP_MOTION_WAKEUP = 0x3c, - HID_USAGE_DESKTOP_START = 0x3d, - HID_USAGE_DESKTOP_SELECT = 0x3e, - HID_USAGE_DESKTOP_VX = 0x40, - HID_USAGE_DESKTOP_VY = 0x41, - HID_USAGE_DESKTOP_VZ = 0x42, - HID_USAGE_DESKTOP_VBRX = 0x43, - HID_USAGE_DESKTOP_VBRY = 0x44, - HID_USAGE_DESKTOP_VBRZ = 0x45, - HID_USAGE_DESKTOP_VNO = 0x46, - HID_USAGE_DESKTOP_FEATURE_NOTIFICATION = 0x47, - HID_USAGE_DESKTOP_RESOLUTION_MULTIPLIER = 0x48, - HID_USAGE_DESKTOP_SYSTEM_CONTROL = 0x80, - HID_USAGE_DESKTOP_SYSTEM_POWER_DOWN = 0x81, - HID_USAGE_DESKTOP_SYSTEM_SLEEP = 0x82, - HID_USAGE_DESKTOP_SYSTEM_WAKE_UP = 0x83, - HID_USAGE_DESKTOP_SYSTEM_CONTEXT_MENU = 0x84, - HID_USAGE_DESKTOP_SYSTEM_MAIN_MENU = 0x85, - HID_USAGE_DESKTOP_SYSTEM_APP_MENU = 0x86, - HID_USAGE_DESKTOP_SYSTEM_MENU_HELP = 0x87, - HID_USAGE_DESKTOP_SYSTEM_MENU_EXIT = 0x88, - HID_USAGE_DESKTOP_SYSTEM_MENU_SELECT = 0x89, - HID_USAGE_DESKTOP_SYSTEM_MENU_RIGHT = 0x8A, - HID_USAGE_DESKTOP_SYSTEM_MENU_LEFT = 0x8B, - HID_USAGE_DESKTOP_SYSTEM_MENU_UP = 0x8C, - HID_USAGE_DESKTOP_SYSTEM_MENU_DOWN = 0x8D, - HID_USAGE_DESKTOP_SYSTEM_COLD_RESTART = 0x8E, - HID_USAGE_DESKTOP_SYSTEM_WARM_RESTART = 0x8F, - HID_USAGE_DESKTOP_DPAD_UP = 0x90, - HID_USAGE_DESKTOP_DPAD_DOWN = 0x91, - HID_USAGE_DESKTOP_DPAD_RIGHT = 0x92, - HID_USAGE_DESKTOP_DPAD_LEFT = 0x93, - HID_USAGE_DESKTOP_SYSTEM_DOCK = 0xA0, - HID_USAGE_DESKTOP_SYSTEM_UNDOCK = 0xA1, - HID_USAGE_DESKTOP_SYSTEM_SETUP = 0xA2, - HID_USAGE_DESKTOP_SYSTEM_BREAK = 0xA3, - HID_USAGE_DESKTOP_SYSTEM_DEBUGGER_BREAK = 0xA4, - HID_USAGE_DESKTOP_APPLICATION_BREAK = 0xA5, - HID_USAGE_DESKTOP_APPLICATION_DEBUGGER_BREAK = 0xA6, - HID_USAGE_DESKTOP_SYSTEM_SPEAKER_MUTE = 0xA7, - HID_USAGE_DESKTOP_SYSTEM_HIBERNATE = 0xA8, - HID_USAGE_DESKTOP_SYSTEM_DISPLAY_INVERT = 0xB0, - HID_USAGE_DESKTOP_SYSTEM_DISPLAY_INTERNAL = 0xB1, - HID_USAGE_DESKTOP_SYSTEM_DISPLAY_EXTERNAL = 0xB2, - HID_USAGE_DESKTOP_SYSTEM_DISPLAY_BOTH = 0xB3, - HID_USAGE_DESKTOP_SYSTEM_DISPLAY_DUAL = 0xB4, - HID_USAGE_DESKTOP_SYSTEM_DISPLAY_TOGGLE_INT_EXT = 0xB5, - HID_USAGE_DESKTOP_SYSTEM_DISPLAY_SWAP_PRIMARY_SECONDARY = 0xB6, - HID_USAGE_DESKTOP_SYSTEM_DISPLAY_LCD_AUTOSCALE = 0xB7 + HID_USAGE_DESKTOP_POINTER = 0x01, // CP + HID_USAGE_DESKTOP_MOUSE = 0x02, // CA + // 03 Reserved + + HID_USAGE_DESKTOP_JOYSTICK = 0x04, // CA + HID_USAGE_DESKTOP_GAMEPAD = 0x05, // CA + HID_USAGE_DESKTOP_KEYBOARD = 0x06, // CA + HID_USAGE_DESKTOP_KEYPAD = 0x07, // CA + HID_USAGE_DESKTOP_MULTI_AXIS_CONTROLLER = 0x08, // CA + HID_USAGE_DESKTOP_TABLET_PC_SYSTEM = 0x09, // CA + HID_USAGE_DESKTOP_WATER_COOLING = 0x0A, // CA + HID_USAGE_DESKTOP_COMPUTER_CHASSIS = 0x0B, // CA + HID_USAGE_DESKTOP_WIRELESS_RADIO = 0x0C, // CA + HID_USAGE_DESKTOP_PORTABLE_DEVICE = 0x0D, // CA + HID_USAGE_DESKTOP_SYSTEM_MULTI_AXIS_CONTROLLER = 0x0E, // CA + HID_USAGE_DESKTOP_SPATIAL_CONTROLLER = 0x0F, // CA + HID_USAGE_DESKTOP_ASSISTIVE = 0x10, // CA + HID_USAGE_DESKTOP_DEVICE_DOCK = 0x11, // CA + HID_USAGE_DESKTOP_DOCKABLE_DEVICE = 0x12, // CA + HID_USAGE_DESKTOP_CALL_STATE_MANAGEMENT = 0x13, // CA + // 14-2F Reserved + + HID_USAGE_DESKTOP_X = 0x30, // DV + HID_USAGE_DESKTOP_Y = 0x31, // DV + HID_USAGE_DESKTOP_Z = 0x32, // DV + HID_USAGE_DESKTOP_RX = 0x33, // DV + HID_USAGE_DESKTOP_RY = 0x34, // DV + HID_USAGE_DESKTOP_RZ = 0x35, // DV + HID_USAGE_DESKTOP_SLIDER = 0x36, // DV + HID_USAGE_DESKTOP_DIAL = 0x37, // DV + HID_USAGE_DESKTOP_WHEEL = 0x38, // DV + HID_USAGE_DESKTOP_HAT_SWITCH = 0x39, // DV + HID_USAGE_DESKTOP_COUNTED_BUFFER = 0x3A, // CL + HID_USAGE_DESKTOP_BYTE_COUNT = 0x3B, // DV + HID_USAGE_DESKTOP_MOTION_WAKEUP = 0x3C, // OSC/DF + HID_USAGE_DESKTOP_START = 0x3D, // OOC + HID_USAGE_DESKTOP_SELECT = 0x3E, // OOC + // 3F Reserved + + HID_USAGE_DESKTOP_VX = 0x40, // DV + HID_USAGE_DESKTOP_VY = 0x41, // DV + HID_USAGE_DESKTOP_VZ = 0x42, // DV + HID_USAGE_DESKTOP_VBRX = 0x43, // DV + HID_USAGE_DESKTOP_VBRY = 0x44, // DV + HID_USAGE_DESKTOP_VBRZ = 0x45, // DV + HID_USAGE_DESKTOP_VNO = 0x46, // DV + HID_USAGE_DESKTOP_FEATURE_NOTIFICATION = 0x47, // DV/DF + HID_USAGE_DESKTOP_RESOLUTION_MULTIPLIER = 0x48, // DV + HID_USAGE_DESKTOP_QX = 0x49, // DV + HID_USAGE_DESKTOP_QY = 0x4A, // DV + HID_USAGE_DESKTOP_QZ = 0x4B, // DV + HID_USAGE_DESKTOP_QW = 0x4C, // DV + // 4D-7F Reserved + + HID_USAGE_DESKTOP_SYSTEM_CONTROL = 0x80, // CA + HID_USAGE_DESKTOP_SYSTEM_POWER_DOWN = 0x81, // OSC + HID_USAGE_DESKTOP_SYSTEM_SLEEP = 0x82, // OSC + HID_USAGE_DESKTOP_SYSTEM_WAKE_UP = 0x83, // OSC + HID_USAGE_DESKTOP_SYSTEM_CONTEXT_MENU = 0x84, // OSC + HID_USAGE_DESKTOP_SYSTEM_MAIN_MENU = 0x85, // OSC + HID_USAGE_DESKTOP_SYSTEM_APP_MENU = 0x86, // OSC + HID_USAGE_DESKTOP_SYSTEM_MENU_HELP = 0x87, // OSC + HID_USAGE_DESKTOP_SYSTEM_MENU_EXIT = 0x88, // OSC + HID_USAGE_DESKTOP_SYSTEM_MENU_SELECT = 0x89, // OSC + HID_USAGE_DESKTOP_SYSTEM_MENU_RIGHT = 0x8A, // RTC + HID_USAGE_DESKTOP_SYSTEM_MENU_LEFT = 0x8B, // RTC + HID_USAGE_DESKTOP_SYSTEM_MENU_UP = 0x8C, // RTC + HID_USAGE_DESKTOP_SYSTEM_MENU_DOWN = 0x8D, // RTC + HID_USAGE_DESKTOP_SYSTEM_COLD_RESTART = 0x8E, // OSC + HID_USAGE_DESKTOP_SYSTEM_WARM_RESTART = 0x8F, // OSC + HID_USAGE_DESKTOP_DPAD_UP = 0x90, // OOC + HID_USAGE_DESKTOP_DPAD_DOWN = 0x91, // OOC + HID_USAGE_DESKTOP_DPAD_RIGHT = 0x92, // OOC + HID_USAGE_DESKTOP_DPAD_LEFT = 0x93, // OOC + HID_USAGE_DESKTOP_INDEX_TRIGGER = 0x94, // MC/DV + HID_USAGE_DESKTOP_PALM_TRIGGER = 0x95, // MC/DV + HID_USAGE_DESKTOP_THUMBSTICK = 0x96, // CP + HID_USAGE_DESKTOP_SYSTEM_FUNCTION_SHIFT = 0x97, // MC + HID_USAGE_DESKTOP_SYSTEM_FUNCTION_SHIFT_LOCK = 0x98, // OOC + HID_USAGE_DESKTOP_SYSTEM_FUNCTION_SHIFT_LOCK_INDICATOR = 0x99, // DV + HID_USAGE_DESKTOP_SYSTEM_DISMISS_NOTIFICATION = 0x9A, // OSC + HID_USAGE_DESKTOP_SYSTEM_DO_NOT_DISTURB = 0x9B, // OOC + // 9C-9F Reserved + + HID_USAGE_DESKTOP_SYSTEM_DOCK = 0xA0, // OSC + HID_USAGE_DESKTOP_SYSTEM_UNDOCK = 0xA1, // OSC + HID_USAGE_DESKTOP_SYSTEM_SETUP = 0xA2, // OSC + HID_USAGE_DESKTOP_SYSTEM_BREAK = 0xA3, // OSC + HID_USAGE_DESKTOP_SYSTEM_DEBUGGER_BREAK = 0xA4, // OSC + HID_USAGE_DESKTOP_APPLICATION_BREAK = 0xA5, // OSC + HID_USAGE_DESKTOP_APPLICATION_DEBUGGER_BREAK = 0xA6, // OSC + HID_USAGE_DESKTOP_SYSTEM_SPEAKER_MUTE = 0xA7, // OSC + HID_USAGE_DESKTOP_SYSTEM_HIBERNATE = 0xA8, // OSC + HID_USAGE_DESKTOP_SYSTEM_MICROPHONE_MUTE = 0xA9, // OOC + HID_USAGE_DESKTOP_SYSTEM_ACCESSIBILITY_BINDING = 0xAA, // OOC + // AB-AF Reserved + + HID_USAGE_DESKTOP_SYSTEM_DISPLAY_INVERT = 0xB0, // OSC + HID_USAGE_DESKTOP_SYSTEM_DISPLAY_INTERNAL = 0xB1, // OSC + HID_USAGE_DESKTOP_SYSTEM_DISPLAY_EXTERNAL = 0xB2, // OSC + HID_USAGE_DESKTOP_SYSTEM_DISPLAY_BOTH = 0xB3, // OSC + HID_USAGE_DESKTOP_SYSTEM_DISPLAY_DUAL = 0xB4, // OSC + HID_USAGE_DESKTOP_SYSTEM_DISPLAY_TOGGLE_INT_EXT = 0xB5, // OSC + HID_USAGE_DESKTOP_SYSTEM_DISPLAY_SWAP_PRIMARY_SECONDARY = 0xB6, // OSC + HID_USAGE_DESKTOP_SYSTEM_DISPLAY_LCD_AUTOSCALE = 0xB7, // OSC + // B8-BF Reserved + + HID_USAGE_DESKTOP_SENSOR_ZONE = 0xC0, // CL + HID_USAGE_DESKTOP_RPM = 0xC1, // DV + HID_USAGE_DESKTOP_COOLANT_LEVEL = 0xC2, // DV + HID_USAGE_DESKTOP_COOLANT_CRITICAL_LEVEL = 0xC3, // SV + HID_USAGE_DESKTOP_COOLANT_PUMP = 0xC4, // US + HID_USAGE_DESKTOP_CHASSIS_ENCLOSURE = 0xC5, // CL + HID_USAGE_DESKTOP_WIRELESS_RADIO_BUTTON = 0xC6, // OOC + HID_USAGE_DESKTOP_WIRELESS_RADIO_LED = 0xC7, // OOC + HID_USAGE_DESKTOP_WIRELESS_RADIO_SLIDER_SWITCH = 0xC8, // OOC + HID_USAGE_DESKTOP_SYSTEM_DISPLAY_ROTATION_LOCK_BUTTON = 0xC9, // OOC + HID_USAGE_DESKTOP_SYSTEM_DISPLAY_ROTATION_LOCK_SLIDER_SWITCH = 0xCA, // OOC + HID_USAGE_DESKTOP_CONTROL_ENABLE = 0xCB, // DF + // CC-CF Reserved + + HID_USAGE_DESKTOP_DOCKABLE_DEVICE_UNIQUE_ID = 0xD0, // DV + HID_USAGE_DESKTOP_DOCKABLE_DEVICE_VENDOR_ID = 0xD1, // DV + HID_USAGE_DESKTOP_DOCKABLE_DEVICE_PRIMARY_USAGE_PAGE = 0xD2, // DV + HID_USAGE_DESKTOP_DOCKABLE_DEVICE_PRIMARY_USAGE_ID = 0xD3, // DV + HID_USAGE_DESKTOP_DOCKABLE_DEVICE_DOCKING_STATE = 0xD4, // DF + HID_USAGE_DESKTOP_DOCKABLE_DEVICE_DISPLAY_OCCLUSION = 0xD5, // CL + HID_USAGE_DESKTOP_DOCKABLE_DEVICE_OBJECT_TYPE = 0xD6, // DV + // D7-DF Reserved + + HID_USAGE_DESKTOP_CALL_ACTIVE_LED = 0xE0, // OOC + HID_USAGE_DESKTOP_CALL_MUTE_TOGGLE = 0xE1, // OSC + HID_USAGE_DESKTOP_CALL_MUTE_LED = 0xE2 // OOC + // E3-FFFF Reserved +}; + +/// HID Usage Table: Simulation Controls Page (0x02) +enum { + HID_USAGE_SIMULATION_CONTROLS_FLIGHT_SIMULATION_DEVICE = 0x01, // CA + HID_USAGE_SIMULATION_CONTROLS_AUTOMOBILE_SIMULATION_DEVICE = 0x02, // CA + HID_USAGE_SIMULATION_CONTROLS_TANK_SIMULATION_DEVICE = 0x03, // CA + HID_USAGE_SIMULATION_CONTROLS_SPACESHIP_SIMULATION_DEVICE = 0x04, // CA + HID_USAGE_SIMULATION_CONTROLS_SUBMARINE_SIMULATION_DEVICE = 0x05, // CA + HID_USAGE_SIMULATION_CONTROLS_SAILING_SIMULATION_DEVICE = 0x06, // CA + HID_USAGE_SIMULATION_CONTROLS_MOTORCYCLE_SIMULATION_DEVICE = 0x07, // CA + HID_USAGE_SIMULATION_CONTROLS_SPORTS_SIMULATION_DEVICE = 0x08, // CA + HID_USAGE_SIMULATION_CONTROLS_AIRPLANE_SIMULATION_DEVICE = 0x09, // CA + HID_USAGE_SIMULATION_CONTROLS_HELICOPTER_SIMULATION_DEVICE = 0x0A, // CA + HID_USAGE_SIMULATION_CONTROLS_MAGIC_CARPET_SIMULATION_DEVICE = 0x0B, // CA + HID_USAGE_SIMULATION_CONTROLS_BICYCLE_SIMULATION_DEVICE = 0x0C, // CA + // 0D-1F Reserved + + HID_USAGE_SIMULATION_CONTROLS_FLIGHT_CONTROL_STICK = 0x20, // CA + HID_USAGE_SIMULATION_CONTROLS_FLIGHT_STICK = 0x21, // CA + HID_USAGE_SIMULATION_CONTROLS_CYCLIC_CONTROL = 0x22, // CP + HID_USAGE_SIMULATION_CONTROLS_CYCLIC_TRIM = 0x23, // CP + HID_USAGE_SIMULATION_CONTROLS_FLIGHT_YOKE = 0x24, // CA + HID_USAGE_SIMULATION_CONTROLS_TRACK_CONTROL = 0x25, // CP + // 26-AF Reserved + + HID_USAGE_SIMULATION_CONTROLS_AILERON = 0xB0, // DV + HID_USAGE_SIMULATION_CONTROLS_AILERON_TRIM = 0xB1, // DV + HID_USAGE_SIMULATION_CONTROLS_ANTI_TORQUE_CONTROL = 0xB2, // DV + HID_USAGE_SIMULATION_CONTROLS_AUTOPILOT_ENABLE = 0xB3, // OOC + HID_USAGE_SIMULATION_CONTROLS_CHAFF_RELEASE = 0xB4, // OSC + HID_USAGE_SIMULATION_CONTROLS_COLLECTIVE_CONTROL = 0xB5, // DV + HID_USAGE_SIMULATION_CONTROLS_DIVE_BRAKE = 0xB6, // DV + HID_USAGE_SIMULATION_CONTROLS_ELECTRONIC_COUNTERMEASURES = 0xB7, // OOC + HID_USAGE_SIMULATION_CONTROLS_ELEVATOR = 0xB8, // DV + HID_USAGE_SIMULATION_CONTROLS_ELEVATOR_TRIM = 0xB9, // DV + HID_USAGE_SIMULATION_CONTROLS_RUDDER = 0xBA, // DV + HID_USAGE_SIMULATION_CONTROLS_THROTTLE = 0xBB, // DV + HID_USAGE_SIMULATION_CONTROLS_FLIGHT_COMMUNICATIONS = 0xBC, // OOC + HID_USAGE_SIMULATION_CONTROLS_FLARE_RELEASE = 0xBD, // OSC + HID_USAGE_SIMULATION_CONTROLS_LANDING_GEAR = 0xBE, // OOC + HID_USAGE_SIMULATION_CONTROLS_TOE_BRAKE = 0xBF, // DV + HID_USAGE_SIMULATION_CONTROLS_TRIGGER = 0xC0, // MC + HID_USAGE_SIMULATION_CONTROLS_WEAPONS_ARM = 0xC1, // OOC + HID_USAGE_SIMULATION_CONTROLS_WEAPONS_SELECT = 0xC2, // OSC + HID_USAGE_SIMULATION_CONTROLS_WING_FLAPS = 0xC3, // DV + HID_USAGE_SIMULATION_CONTROLS_ACCELERATOR = 0xC4, // DV + HID_USAGE_SIMULATION_CONTROLS_BRAKE = 0xC5, // DV + HID_USAGE_SIMULATION_CONTROLS_CLUTCH = 0xC6, // DV + HID_USAGE_SIMULATION_CONTROLS_SHIFTER = 0xC7, // DV + HID_USAGE_SIMULATION_CONTROLS_STEERING = 0xC8, // DV + HID_USAGE_SIMULATION_CONTROLS_TURRET_DIRECTION = 0xC9, // DV + HID_USAGE_SIMULATION_CONTROLS_BARREL_ELEVATION = 0xCA, // DV + HID_USAGE_SIMULATION_CONTROLS_DIVE_PLANE = 0xCB, // DV + HID_USAGE_SIMULATION_CONTROLS_BALLAST = 0xCC, // DV + HID_USAGE_SIMULATION_CONTROLS_BICYCLE_CRANK = 0xCD, // DV + HID_USAGE_SIMULATION_CONTROLS_HANDLE_BARS = 0xCE, // DV + HID_USAGE_SIMULATION_CONTROLS_FRONT_BRAKE = 0xCF, // DV + HID_USAGE_SIMULATION_CONTROLS_REAR_BRAKE = 0xD0, // DV + // D1-FFFF Reserved +}; + +/// HID Usage Table: VR Controls Page (0x03) +enum { + HID_USAGE_VR_CONTROLS_BELT = 0x01, // CA + HID_USAGE_VR_CONTROLS_BODY_SUIT = 0x02, // CA + HID_USAGE_VR_CONTROLS_FLEXOR = 0x03, // CP + HID_USAGE_VR_CONTROLS_GLOVE = 0x04, // CA + HID_USAGE_VR_CONTROLS_HEAD_TRACKER = 0x05, // CP + HID_USAGE_VR_CONTROLS_HEAD_MOUNTED_DISPLAY = 0x06, // CA + HID_USAGE_VR_CONTROLS_HAND_TRACKER = 0x07, // CA + HID_USAGE_VR_CONTROLS_OCULOMETER = 0x08, // CA + HID_USAGE_VR_CONTROLS_VEST = 0x09, // CA + HID_USAGE_VR_CONTROLS_ANIMATRONIC_DEVICE = 0x0A, // CA + // 0B-1F Reserved + + HID_USAGE_VR_CONTROLS_STEREO_ENABLE = 0x20, // OOC + HID_USAGE_VR_CONTROLS_DISPLAY_ENABLE = 0x21 // OOC + // 22-FFFF Reserved +}; + +/// HID Usage Table: Sports Controls Page (0x04) +enum { + HID_USAGE_SPORTS_CONTROLS_BASEBALL_BAT = 0x01, // CA + HID_USAGE_SPORTS_CONTROLS_GOLF_CLUB = 0x02, // CA + HID_USAGE_SPORTS_CONTROLS_ROWING_MACHINE = 0x03, // CA + HID_USAGE_SPORTS_CONTROLS_TREADMILL = 0x04, // CA + // 05-2F Reserved + + HID_USAGE_SPORTS_CONTROLS_OAR = 0x30, // DV + HID_USAGE_SPORTS_CONTROLS_SLOPE = 0x31, // DV + HID_USAGE_SPORTS_CONTROLS_RATE = 0x32, // DV + HID_USAGE_SPORTS_CONTROLS_STICK_SPEED = 0x33, // DV + HID_USAGE_SPORTS_CONTROLS_STICK_FACE_ANGLE = 0x34, // DV + HID_USAGE_SPORTS_CONTROLS_STICK_HEEL_TOE = 0x35, // DV + HID_USAGE_SPORTS_CONTROLS_STICK_FOLLOW_THROUGH = 0x36, // DV + HID_USAGE_SPORTS_CONTROLS_STICK_TEMPO = 0x37, // DV + HID_USAGE_SPORTS_CONTROLS_STICK_TYPE = 0x38, // NAry + HID_USAGE_SPORTS_CONTROLS_STICK_HEIGHT = 0x39, // DV + // 3A-4F Reserved + + HID_USAGE_SPORTS_CONTROLS_PUTTER = 0x50, // Sel + HID_USAGE_SPORTS_CONTROLS_1_IRON = 0x51, // Sel + HID_USAGE_SPORTS_CONTROLS_2_IRON = 0x52, // Sel + HID_USAGE_SPORTS_CONTROLS_3_IRON = 0x53, // Sel + HID_USAGE_SPORTS_CONTROLS_4_IRON = 0x54, // Sel + HID_USAGE_SPORTS_CONTROLS_5_IRON = 0x55, // Sel + HID_USAGE_SPORTS_CONTROLS_6_IRON = 0x56, // Sel + HID_USAGE_SPORTS_CONTROLS_7_IRON = 0x57, // Sel + HID_USAGE_SPORTS_CONTROLS_8_IRON = 0x58, // Sel + HID_USAGE_SPORTS_CONTROLS_9_IRON = 0x59, // Sel + HID_USAGE_SPORTS_CONTROLS_10_IRON = 0x5A, // Sel + HID_USAGE_SPORTS_CONTROLS_11_IRON = 0x5B, // Sel + HID_USAGE_SPORTS_CONTROLS_SAND_WEDGE = 0x5C, // Sel + HID_USAGE_SPORTS_CONTROLS_LOFT_WEDGE = 0x5D, // Sel + HID_USAGE_SPORTS_CONTROLS_POWER_WEDGE = 0x5E, // Sel + HID_USAGE_SPORTS_CONTROLS_1_WOOD = 0x5F, // Sel + HID_USAGE_SPORTS_CONTROLS_3_WOOD = 0x60, // Sel + HID_USAGE_SPORTS_CONTROLS_5_WOOD = 0x61, // Sel + HID_USAGE_SPORTS_CONTROLS_7_WOOD = 0x62, // Sel + HID_USAGE_SPORTS_CONTROLS_9_WOOD = 0x63, // Sel + // 64-FFFF Reserved +}; + +/// HID Usage Table: Game Controls Page (0x05) +enum { + HID_USAGE_GAME_CONTROLS_3D_GAME_CONTROLLER = 0x01, // CA + HID_USAGE_GAME_CONTROLS_PINBALL_DEVICE = 0x02, // CA + HID_USAGE_GAME_CONTROLS_GUN_DEVICE = 0x03, // CA + // 04-1F Reserved + + HID_USAGE_GAME_CONTROLS_POINT_OF_VIEW = 0x20, // CP + HID_USAGE_GAME_CONTROLS_TURN_RIGHT_LEFT = 0x21, // DV + HID_USAGE_GAME_CONTROLS_PITCH_FORWARD_BACKWARD = 0x22, // DV + HID_USAGE_GAME_CONTROLS_ROLL_RIGHT_LEFT = 0x23, // DV + HID_USAGE_GAME_CONTROLS_MOVE_RIGHT_LEFT = 0x24, // DV + HID_USAGE_GAME_CONTROLS_MOVE_FORWARD_BACKWARD = 0x25, // DV + HID_USAGE_GAME_CONTROLS_MOVE_UP_DOWN = 0x26, // DV + HID_USAGE_GAME_CONTROLS_LEAN_RIGHT_LEFT = 0x27, // DV + HID_USAGE_GAME_CONTROLS_LEAN_FORWARD_BACKWARD = 0x28, // DV + HID_USAGE_GAME_CONTROLS_HEIGHT_OF_POV = 0x29, // DV + HID_USAGE_GAME_CONTROLS_FLIPPER = 0x2A, // MC + HID_USAGE_GAME_CONTROLS_SECONDARY_FLIPPER = 0x2B, // MC + HID_USAGE_GAME_CONTROLS_BUMP = 0x2C, // MC + HID_USAGE_GAME_CONTROLS_NEW_GAME = 0x2D, // OSC + HID_USAGE_GAME_CONTROLS_SHOOT_BALL = 0x2E, // OSC + HID_USAGE_GAME_CONTROLS_PLAYER = 0x2F, // OSC + HID_USAGE_GAME_CONTROLS_GUN_BOLT = 0x30, // OOC + HID_USAGE_GAME_CONTROLS_GUN_CLIP = 0x31, // OOC + HID_USAGE_GAME_CONTROLS_GUN_SELECTOR = 0x32, // NAry + HID_USAGE_GAME_CONTROLS_GUN_SINGLE_SHOT = 0x33, // Sel + HID_USAGE_GAME_CONTROLS_GUN_BURST = 0x34, // Sel + HID_USAGE_GAME_CONTROLS_GUN_AUTOMATIC = 0x35, // Sel + HID_USAGE_GAME_CONTROLS_GUN_SAFETY = 0x36, // OOC + HID_USAGE_GAME_CONTROLS_GAMEPAD_FIRE_JUMP = 0x37, // CL + // 38 Reserved + + HID_USAGE_GAME_CONTROLS_GAMEPAD_TRIGGER = 0x39, // CL + HID_USAGE_GAME_CONTROLS_FORM_FITTING_GAMEPAD = 0x3A // SF + // 3B-FFFF Reserved +}; + +/// HID Usage Table: Generic Device Controls Page (0x06) +enum { + HID_USAGE_GENERIC_DEVICE_CONTROLS_BACKGROUND_NONUSER_CONTROLS = 0x01, // CA + // 02-1F Reserved + + HID_USAGE_GENERIC_DEVICE_CONTROLS_BATTERY_STRENGTH = 0x20, // DV + HID_USAGE_GENERIC_DEVICE_CONTROLS_WIRELESS_CHANNEL = 0x21, // DV + HID_USAGE_GENERIC_DEVICE_CONTROLS_WIRELESS_ID = 0x22, // DV + HID_USAGE_GENERIC_DEVICE_CONTROLS_DISCOVER_WIRELESS_CONTROL = 0x23, // OSC + HID_USAGE_GENERIC_DEVICE_CONTROLS_SECURITY_CODE_CHARACTER_ENTERED = 0x24, // OSC + HID_USAGE_GENERIC_DEVICE_CONTROLS_SECURITY_CODE_CHARACTER_ERASED = 0x25, // OSC + HID_USAGE_GENERIC_DEVICE_CONTROLS_SECURITY_CODE_CLEARED = 0x26, // OSC + HID_USAGE_GENERIC_DEVICE_CONTROLS_SEQUENCE_ID = 0x27, // DV + HID_USAGE_GENERIC_DEVICE_CONTROLS_SEQUENCE_ID_RESET = 0x28, // DF + HID_USAGE_GENERIC_DEVICE_CONTROLS_RF_SIGNAL_STRENGTH = 0x29, // DV + HID_USAGE_GENERIC_DEVICE_CONTROLS_SOFTWARE_VERSION = 0x2A, // CL + HID_USAGE_GENERIC_DEVICE_CONTROLS_PROTOCOL_VERSION = 0x2B, // CL + HID_USAGE_GENERIC_DEVICE_CONTROLS_HARDWARE_VERSION = 0x2C, // CL + HID_USAGE_GENERIC_DEVICE_CONTROLS_MAJOR = 0x2D, // SV + HID_USAGE_GENERIC_DEVICE_CONTROLS_MINOR = 0x2E, // SV + HID_USAGE_GENERIC_DEVICE_CONTROLS_REVISION = 0x2F, // SV + HID_USAGE_GENERIC_DEVICE_CONTROLS_HANDEDNESS = 0x30, // NAry + HID_USAGE_GENERIC_DEVICE_CONTROLS_EITHER_HAND = 0x31, // Sel + HID_USAGE_GENERIC_DEVICE_CONTROLS_LEFT_HAND = 0x32, // Sel + HID_USAGE_GENERIC_DEVICE_CONTROLS_RIGHT_HAND = 0x33, // Sel + HID_USAGE_GENERIC_DEVICE_CONTROLS_BOTH_HANDS = 0x34, // Sel + // 35-3F Reserved + + HID_USAGE_GENERIC_DEVICE_CONTROLS_GRIP_POSE_OFFSET = 0x40, // CP + HID_USAGE_GENERIC_DEVICE_CONTROLS_POINTER_POSE_OFFSET = 0x41 // CP + // 42-FFFF Reserved +}; + +/// HID Usage Table: Keyboard/Keypad Page (0x07) +/// Defined above + +/// HID Usage Table: LED Page (0x08) +enum { + HID_USAGE_LED_NUM_LOCK = 0x01, // OOC + HID_USAGE_LED_CAPS_LOCK = 0x02, // OOC + HID_USAGE_LED_SCROLL_LOCK = 0x03, // OOC + HID_USAGE_LED_COMPOSE = 0x04, // OOC + HID_USAGE_LED_KANA = 0x05, // OOC + HID_USAGE_LED_POWER = 0x06, // OOC + HID_USAGE_LED_SHIFT = 0x07, // OOC + HID_USAGE_LED_DO_NOT_SHIFT = 0x08, // OOC + HID_USAGE_LED_MUTE = 0x09, // OOC + HID_USAGE_LED_TONE_ENABLE = 0x0A, // OOC + HID_USAGE_LED_HIGH_CUT_FILTER = 0x0B, // OOC + HID_USAGE_LED_LOW_CUT_FILTER = 0x0C, // OOC + HID_USAGE_LED_EQUALIZER_ENABLE = 0x0D, // OOC + HID_USAGE_LED_SOUND_FIELD_ON = 0x0E, // OOC + HID_USAGE_LED_SURROUND_ON = 0x0F, // OOC + HID_USAGE_LED_REPEAT = 0x10, // OOC + HID_USAGE_LED_STEREO = 0x11, // OOC + HID_USAGE_LED_SAMPLING_RATE_DETECT = 0x12, // OOC + HID_USAGE_LED_SPINNING = 0x13, // OOC + HID_USAGE_LED_CAV = 0x14, // OOC + HID_USAGE_LED_CLV = 0x15, // OOC + HID_USAGE_LED_RECORDING_FORMAT_DETECT = 0x16, // OOC + HID_USAGE_LED_OFF_HOOK = 0x17, // OOC + HID_USAGE_LED_RING = 0x18, // OOC + HID_USAGE_LED_MESSAGE_WAITING = 0x19, // OOC + HID_USAGE_LED_DATA_MODE = 0x1A, // OOC + HID_USAGE_LED_BATTERY_OPERATION = 0x1B, // OOC + HID_USAGE_LED_BATTERY_OK = 0x1C, // OOC + HID_USAGE_LED_BATTERY_LOW = 0x1D, // OOC + HID_USAGE_LED_SPEAKER = 0x1E, // OOC + HID_USAGE_LED_HEADSET = 0x1F, // OOC + HID_USAGE_LED_HOLD = 0x20, // OOC + HID_USAGE_LED_MICROPHONE = 0x21, // OOC + HID_USAGE_LED_COVERAGE = 0x22, // OOC + HID_USAGE_LED_NIGHT_MODE = 0x23, // OOC + HID_USAGE_LED_SEND_CALLS = 0x24, // OOC + HID_USAGE_LED_CALL_PICKUPS = 0x25, // OOC + HID_USAGE_LED_CONFERENCE = 0x26, // OOC + HID_USAGE_LED_STANDBY = 0x27, // OOC + HID_USAGE_LED_CAMERA_ON = 0x28, // OOC + HID_USAGE_LED_CAMERA_OFF = 0x29, // OOC + HID_USAGE_LED_ON_LINE = 0x2A, // OOC + HID_USAGE_LED_OFF_LINE = 0x2B, // OOC + HID_USAGE_LED_BUSY = 0x2C, // OOC + HID_USAGE_LED_READY = 0x2D, // OOC + HID_USAGE_LED_PAPER_OUT = 0x2E, // OOC + HID_USAGE_LED_PAPER_JAM = 0x2F, // OOC + HID_USAGE_LED_REMOTE = 0x30, // OOC + HID_USAGE_LED_FORWARD = 0x31, // OOC + HID_USAGE_LED_REVERSE = 0x32, // OOC + HID_USAGE_LED_STOP = 0x33, // OOC + HID_USAGE_LED_REWIND = 0x34, // OOC + HID_USAGE_LED_FAST_FORWARD = 0x35, // OOC + HID_USAGE_LED_PLAY = 0x36, // OOC + HID_USAGE_LED_PAUSE = 0x37, // OOC + HID_USAGE_LED_RECORD = 0x38, // OOC + HID_USAGE_LED_ERROR = 0x39, // OOC + HID_USAGE_LED_USAGE_SELECTED_INDICATOR = 0x3A, // US + HID_USAGE_LED_USAGE_IN_USE_INDICATOR = 0x3B, // US + HID_USAGE_LED_USAGE_MULTI_MODE_INDICATOR = 0x3C, // UM + HID_USAGE_LED_INDICATOR_ON = 0x3D, // Sel + HID_USAGE_LED_INDICATOR_FLASH = 0x3E, // Sel + HID_USAGE_LED_INDICATOR_SLOW_BLINK = 0x3F, // Sel + HID_USAGE_LED_INDICATOR_FAST_BLINK = 0x40, // Sel + HID_USAGE_LED_INDICATOR_OFF = 0x41, // Sel + HID_USAGE_LED_FLASH_ON_TIME = 0x42, // DV + HID_USAGE_LED_SLOW_BLINK_ON_TIME = 0x43, // DV + HID_USAGE_LED_SLOW_BLINK_OFF_TIME = 0x44, // DV + HID_USAGE_LED_FAST_BLINK_ON_TIME = 0x45, // DV + HID_USAGE_LED_FAST_BLINK_OFF_TIME = 0x46, // DV + HID_USAGE_LED_USAGE_INDICATOR_COLOR = 0x47, // UM + HID_USAGE_LED_INDICATOR_RED = 0x48, // Sel + HID_USAGE_LED_INDICATOR_GREEN = 0x49, // Sel + HID_USAGE_LED_INDICATOR_AMBER = 0x4A, // Sel + HID_USAGE_LED_GENERIC_INDICATOR = 0x4B, // OOC + HID_USAGE_LED_SYSTEM_SUSPEND = 0x4C, // OOC + HID_USAGE_LED_EXTERNAL_POWER_CONNECTED = 0x4D, // OOC + HID_USAGE_LED_INDICATOR_BLUE = 0x4E, // Sel + HID_USAGE_LED_INDICATOR_ORANGE = 0x4F, // Sel + HID_USAGE_LED_GOOD_STATUS = 0x50, // OOC + HID_USAGE_LED_WARNING_STATUS = 0x51, // OOC + HID_USAGE_LED_RGB_LED = 0x52, // CL + HID_USAGE_LED_RED_LED_CHANNEL = 0x53, // DV + HID_USAGE_LED_BLUE_LED_CHANNEL = 0x54, // DV + HID_USAGE_LED_GREEN_LED_CHANNEL = 0x55, // DV + HID_USAGE_LED_LED_INTENSITY = 0x56, // DV + HID_USAGE_LED_SYSTEM_MICROPHONE_MUTE = 0x57, // OOC + // 58-5F Reserved + + HID_USAGE_LED_PLAYER_INDICATOR = 0x60, // NAry + HID_USAGE_LED_PLAYER_1 = 0x61, // Sel + HID_USAGE_LED_PLAYER_2 = 0x62, // Sel + HID_USAGE_LED_PLAYER_3 = 0x63, // Sel + HID_USAGE_LED_PLAYER_4 = 0x64, // Sel + HID_USAGE_LED_PLAYER_5 = 0x65, // Sel + HID_USAGE_LED_PLAYER_6 = 0x66, // Sel + HID_USAGE_LED_PLAYER_7 = 0x67, // Sel + HID_USAGE_LED_PLAYER_8 = 0x68, // Sel + // 69-FFFF Reserved +}; + +/// HID Usage Table: Button Page (0x09) +/// Intentionally skipped + +/// HID Usage Table: Ordinal Page (0x0A) +/// Intentionally skipped + +/// HID Usage Table: Telephony Device Page (0x0B) +enum { + HID_USAGE_TELEPHONY_PHONE = 0x0001, // CA + HID_USAGE_TELEPHONY_ANSWERING_MACHINE = 0x0002, // CA + HID_USAGE_TELEPHONY_MESSAGE_CONTROLS = 0x0003, // CL + HID_USAGE_TELEPHONY_HANDSET = 0x0004, // CL + HID_USAGE_TELEPHONY_HEADSET = 0x0005, // CL/CA + HID_USAGE_TELEPHONY_TELEPHONY_KEY_PAD = 0x0006, // NAry + HID_USAGE_TELEPHONY_PROGRAMMABLE_BUTTON = 0x0007, // NAry + // 08-1F Reserved + + HID_USAGE_TELEPHONY_HOOK_SWITCH = 0x0020, // OOC + HID_USAGE_TELEPHONY_FLASH = 0x0021, // MC + HID_USAGE_TELEPHONY_FEATURE = 0x0022, // OSC + HID_USAGE_TELEPHONY_HOLD = 0x0023, // OOC + HID_USAGE_TELEPHONY_REDIAL = 0x0024, // OSC + HID_USAGE_TELEPHONY_TRANSFER = 0x0025, // OSC + HID_USAGE_TELEPHONY_DROP = 0x0026, // OSC + HID_USAGE_TELEPHONY_PARK = 0x0027, // OOC + HID_USAGE_TELEPHONY_FORWARD_CALLS = 0x0028, // OOC + HID_USAGE_TELEPHONY_ALTERNATE_FUNCTION = 0x0029, // MC + HID_USAGE_TELEPHONY_LINE = 0x002A, // OSC/NAry + HID_USAGE_TELEPHONY_SPEAKER_PHONE = 0x002B, // OOC + HID_USAGE_TELEPHONY_CONFERENCE = 0x002C, // OOC + HID_USAGE_TELEPHONY_RING_ENABLE = 0x002D, // OOC + HID_USAGE_TELEPHONY_RING_SELECT = 0x002E, // OSC + HID_USAGE_TELEPHONY_PHONE_MUTE = 0x002F, // OOC + HID_USAGE_TELEPHONY_CALLER_ID = 0x0030, // MC + HID_USAGE_TELEPHONY_SEND = 0x0031, // OOC + // 32-4F Reserved + + HID_USAGE_TELEPHONY_SPEED_DIAL = 0x0050, // OSC + HID_USAGE_TELEPHONY_STORE_NUMBER = 0x0051, // OSC + HID_USAGE_TELEPHONY_RECALL_NUMBER = 0x0052, // OSC + HID_USAGE_TELEPHONY_PHONE_DIRECTORY = 0x0053, // OOC + // 54-6F Reserved + + HID_USAGE_TELEPHONY_VOICE_MAIL = 0x0070, // OOC + HID_USAGE_TELEPHONY_SCREEN_CALLS = 0x0071, // OOC + HID_USAGE_TELEPHONY_DO_NOT_DISTURB = 0x0072, // OOC + HID_USAGE_TELEPHONY_MESSAGE = 0x0073, // OSC + HID_USAGE_TELEPHONY_ANSWER_ON_OFF = 0x0074, // OOC + // 75-8F Reserved + + HID_USAGE_TELEPHONY_INSIDE_DIAL_TONE = 0x0090, // MC + HID_USAGE_TELEPHONY_OUTSIDE_DIAL_TONE = 0x0091, // MC + HID_USAGE_TELEPHONY_INSIDE_RING_TONE = 0x0092, // MC + HID_USAGE_TELEPHONY_OUTSIDE_RING_TONE = 0x0093, // MC + HID_USAGE_TELEPHONY_PRIORITY_RING_TONE = 0x0094, // MC + HID_USAGE_TELEPHONY_INSIDE_RINGBACK = 0x0095, // MC + HID_USAGE_TELEPHONY_PRIORITY_RINGBACK = 0x0096, // MC + HID_USAGE_TELEPHONY_LINE_BUSY_TONE = 0x0097, // MC + HID_USAGE_TELEPHONY_REORDER_TONE = 0x0098, // MC + HID_USAGE_TELEPHONY_CALL_WAITING_TONE = 0x0099, // MC + HID_USAGE_TELEPHONY_CONFIRMATION_TONE_1 = 0x009A, // MC + HID_USAGE_TELEPHONY_CONFIRMATION_TONE_2 = 0x009B, // MC + HID_USAGE_TELEPHONY_TONES_OFF = 0x009C, // OOC + HID_USAGE_TELEPHONY_OUTSIDE_RINGBACK = 0x009D, // MC + HID_USAGE_TELEPHONY_RINGER = 0x009E, // OOC + // 9F-AF Reserved + + HID_USAGE_TELEPHONY_PHONE_KEY_0 = 0x00B0, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_1 = 0x00B1, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_2 = 0x00B2, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_3 = 0x00B3, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_4 = 0x00B4, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_5 = 0x00B5, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_6 = 0x00B6, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_7 = 0x00B7, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_8 = 0x00B8, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_9 = 0x00B9, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_STAR = 0x00BA, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_POUND = 0x00BB, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_A = 0x00BC, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_B = 0x00BD, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_C = 0x00BE, // Sel + HID_USAGE_TELEPHONY_PHONE_KEY_D = 0x00BF, // Sel + HID_USAGE_TELEPHONY_PHONE_CALL_HISTORY_KEY = 0x00C0, // Sel + HID_USAGE_TELEPHONY_PHONE_CALLER_ID_KEY = 0x00C1, // Sel + HID_USAGE_TELEPHONY_PHONE_SETTINGS_KEY = 0x00C2, // Sel + // C3-EF Reserved + + HID_USAGE_TELEPHONY_HOST_CONTROL = 0x00F0, // OOC + HID_USAGE_TELEPHONY_HOST_AVAILABLE = 0x00F1, // OOC + HID_USAGE_TELEPHONY_HOST_CALL_ACTIVE = 0x00F2, // OOC + HID_USAGE_TELEPHONY_ACTIVATE_HANDSET_AUDIO = 0x00F3, // OOC + HID_USAGE_TELEPHONY_RING_TYPE = 0x00F4, // NAry + HID_USAGE_TELEPHONY_REDIALABLE_PHONE_NUMBER = 0x00F5, // OOC + // F6-F7 Reserved + + HID_USAGE_TELEPHONY_STOP_RING_TONE = 0x00F8, // Sel + HID_USAGE_TELEHONY_PSTN_RING_TONE = 0x00F9, // Sel + HID_USAGE_TELEPHONY_HOST_RING_TONE = 0x00FA, // Sel + HID_USAGE_TELEPHONY_ALERT_SOUND_ERROR = 0x00FB, // Sel + HID_USAGE_TELEPHONY_ALERT_SOUND_CONFIRM = 0x00FC, // Sel + HID_USAGE_TELEPHONY_ALERT_SOUND_NOTIFICATION = 0x00FD, // Sel + HID_USAGE_TELEPHONY_SILENT_RING = 0x00FE, // Sel + // FF-107 Reserved + + HID_USAGE_TELEPHONY_EMAIL_MESSAGE_WAITING = 0x0108, // OOC + HID_USAGE_TELEPHONY_VOICEMAIL_MESSAGE_WAITING = 0x0109, // OOC + HID_USAGE_TELEPHONY_HOST_HOLD = 0x010A, // OOC + // 10B-10F Reserved + + HID_USAGE_TELEPHONY_INCOMING_CALL_HISTORY_COUNT = 0x0110, // DV + HID_USAGE_TELEPHONY_OUTGOING_CALL_HISTORY_COUNT = 0x0111, // DV + HID_USAGE_TELEPHONY_INCOMING_CALL_HISTORY = 0x0112, // CL + HID_USAGE_TELEPHONY_OUTGOING_CALL_HISTORY = 0x0113, // CL + HID_USAGE_TELEPHONY_PHONE_LOCALE = 0x0114, // DV + // 115-13F Reserved + + HID_USAGE_TELEPHONY_PHONE_TIME_SECOND = 0x0140, // DV + HID_USAGE_TELEPHONY_PHONE_TIME_MINUTE = 0x0141, // DV + HID_USAGE_TELEPHONY_PHONE_TIME_HOUR = 0x0142, // DV + HID_USAGE_TELEPHONY_PHONE_DATE_DAY = 0x0143, // DV + HID_USAGE_TELEPHONY_PHONE_DATE_MONTH = 0x0144, // DV + HID_USAGE_TELEPHONY_PHONE_DATE_YEAR = 0x0145, // DV + HID_USAGE_TELEPHONY_HANDSET_NICKNAME = 0x0146, // DV + HID_USAGE_TELEPHONY_ADDRESS_BOOK_ID = 0x0147, // DV + // 148-149 Reserved + + HID_USAGE_TELEPHONY_CALL_DURATION = 0x014A, // DV + HID_USAGE_TELEPHONY_DUAL_MODE_PHONE = 0x014B, // CA + // 14C-FFFF Reserved }; /// HID Usage Table: Consumer Page (0x0C) -/// Only contains controls that supported by Windows (whole list is too long) enum { - HID_USAGE_CONSUMER_UNASSIGNED = 0x0000, + HID_USAGE_CONSUMER_UNASSIGNED = 0x0000, // Generic Control - HID_USAGE_CONSUMER_CONTROL = 0x0001, - HID_USAGE_CONSUMER_NUMERIC_KEY_PAD = 0x0002, - HID_USAGE_CONSUMER_PROGRAMMABLE_BUTTONS = 0x0003, - HID_USAGE_CONSUMER_MICROPHONE = 0x0004, - HID_USAGE_CONSUMER_HEADPHONE = 0x0005, - HID_USAGE_CONSUMER_GRAPHIC_EQUALIZER = 0x0006, - // 07-1F Reserved + HID_USAGE_CONSUMER_CONTROL = 0x0001, // CA + HID_USAGE_CONSUMER_NUMERIC_KEY_PAD = 0x0002, // NAry + HID_USAGE_CONSUMER_PROGRAMMABLE_BUTTONS = 0x0003, // NAry + HID_USAGE_CONSUMER_MICROPHONE = 0x0004, // CA + HID_USAGE_CONSUMER_HEADPHONE = 0x0005, // CA + HID_USAGE_CONSUMER_GRAPHIC_EQUALIZER = 0x0006, // CA + HID_USAGE_CONSUMER_KEYBOARD_BACKLIGHT = 0x0007, // CA + // 08-1F Reserved - HID_USAGE_CONSUMER_PLUS_10 = 0x0020, - HID_USAGE_CONSUMER_PLUS_100 = 0x0021, - HID_USAGE_CONSUMER_AM_PM = 0x0022, + HID_USAGE_CONSUMER_PLUS_10 = 0x0020, // OSC + HID_USAGE_CONSUMER_PLUS_100 = 0x0021, // OSC + HID_USAGE_CONSUMER_AM_PM = 0x0022, // OSC // 23-3F Reserved // Power Control - HID_USAGE_CONSUMER_POWER = 0x0030, - HID_USAGE_CONSUMER_RESET = 0x0031, - HID_USAGE_CONSUMER_SLEEP = 0x0032, + HID_USAGE_CONSUMER_POWER = 0x0030, // OOC + HID_USAGE_CONSUMER_RESET = 0x0031, // OSC + HID_USAGE_CONSUMER_SLEEP = 0x0032, // OSC - HID_USAGE_CONSUMER_SLEEP_AFTER = 0x0033, - HID_USAGE_CONSUMER_SLEEP_MODE = 0x0034, - HID_USAGE_CONSUMER_ILLUMINATION = 0x0035, - HID_USAGE_CONSUMER_FUNCTION_BUTTONS = 0x0036, + HID_USAGE_CONSUMER_SLEEP_AFTER = 0x0033, // OSC + HID_USAGE_CONSUMER_SLEEP_MODE = 0x0034, // RTC + HID_USAGE_CONSUMER_ILLUMINATION = 0x0035, // OOC + HID_USAGE_CONSUMER_FUNCTION_BUTTONS = 0x0036, // NAry // 37-3F Reserved - HID_USAGE_CONSUMER_MENU = 0x0040, - HID_USAGE_CONSUMER_MENU_PICK = 0x0041, - HID_USAGE_CONSUMER_MENU_UP = 0x0042, - HID_USAGE_CONSUMER_MENU_DOWN = 0x0043, - HID_USAGE_CONSUMER_MENU_LEFT = 0x0044, - HID_USAGE_CONSUMER_MENU_RIGHT = 0x0045, - HID_USAGE_CONSUMER_MENU_ESCAPE = 0x0046, - HID_USAGE_CONSUMER_MENU_VALUE_INCREASE = 0x0047, - HID_USAGE_CONSUMER_MENU_VALUE_DECREASE = 0x0048, + HID_USAGE_CONSUMER_MENU = 0x0040, // OOC + HID_USAGE_CONSUMER_MENU_PICK = 0x0041, // OSC + HID_USAGE_CONSUMER_MENU_UP = 0x0042, // OSC + HID_USAGE_CONSUMER_MENU_DOWN = 0x0043, // OSC + HID_USAGE_CONSUMER_MENU_LEFT = 0x0044, // OSC + HID_USAGE_CONSUMER_MENU_RIGHT = 0x0045, // OSC + HID_USAGE_CONSUMER_MENU_ESCAPE = 0x0046, // OSC + HID_USAGE_CONSUMER_MENU_VALUE_INCREASE = 0x0047, // OSC + HID_USAGE_CONSUMER_MENU_VALUE_DECREASE = 0x0048, // OSC // 49-5F Reserved - HID_USAGE_CONSUMER_DATA_ON_SCREEN = 0x0060, - HID_USAGE_CONSUMER_CLOSED_CAPTION = 0x0061, - HID_USAGE_CONSUMER_CLOSED_CAPTION_SELECT = 0x0062, - HID_USAGE_CONSUMER_VCR_TV = 0x0063, - HID_USAGE_CONSUMER_BROADCAST_MODE = 0x0064, - HID_USAGE_CONSUMER_SNAPSHOT = 0x0065, - HID_USAGE_CONSUMER_STILL = 0x0066, - - // 67-7F Reserved - // Screen Brightness - HID_USAGE_CONSUMER_BRIGHTNESS_INCREMENT = 0x006F, - HID_USAGE_CONSUMER_BRIGHTNESS_DECREMENT = 0x0070, - - // These HID usages operate only on mobile systems (battery powered) and - // require Windows 8 (build 8302 or greater). - HID_USAGE_CONSUMER_WIRELESS_RADIO_CONTROLS = 0x000C, - HID_USAGE_CONSUMER_WIRELESS_RADIO_BUTTONS = 0x00C6, - HID_USAGE_CONSUMER_WIRELESS_RADIO_LED = 0x00C7, - HID_USAGE_CONSUMER_WIRELESS_RADIO_SLIDER_SWITCH = 0x00C8, - - HID_USAGE_CONSUMER_SELECTION = 0x0080, - HID_USAGE_CONSUMER_ASSIGN_SELECTION = 0x0081, - HID_USAGE_CONSUMER_MODE_STEP = 0x0082, - HID_USAGE_CONSUMER_RECALL_LAST = 0x0083, - HID_USAGE_CONSUMER_ENTER_CHANNEL = 0x0084, - HID_USAGE_CONSUMER_ORDER_MOVIE = 0x0085, - HID_USAGE_CONSUMER_CHANNEL = 0x0086, - HID_USAGE_CONSUMER_MEDIA_SELECTION = 0x0087, - HID_USAGE_CONSUMER_MEDIA_SELECT_COMPUTER = 0x0088, - HID_USAGE_CONSUMER_MEDIA_SELECT_TV = 0x0089, - HID_USAGE_CONSUMER_MEDIA_SELECT_WWW = 0x008A, - HID_USAGE_CONSUMER_MEDIA_SELECT_DVD = 0x008B, - HID_USAGE_CONSUMER_MEDIA_SELECT_TELEPHONE = 0x008C, - HID_USAGE_CONSUMER_MEDIA_SELECT_PROGRAM_GUIDE = 0x008D, - HID_USAGE_CONSUMER_MEDIA_SELECT_VIDEO_PHONE = 0x008E, - HID_USAGE_CONSUMER_MEDIA_SELECT_GAMES = 0x008F, - HID_USAGE_CONSUMER_MEDIA_SELECT_MESSAGES = 0x0090, - HID_USAGE_CONSUMER_MEDIA_SELECT_CD = 0x0091, - HID_USAGE_CONSUMER_MEDIA_SELECT_VCR = 0x0092, - HID_USAGE_CONSUMER_MEDIA_SELECT_TUNER = 0x0093, - HID_USAGE_CONSUMER_QUIT = 0x0094, - HID_USAGE_CONSUMER_HELP = 0x0095, - HID_USAGE_CONSUMER_MEDIA_SELECT_TAPE = 0x0096, - HID_USAGE_CONSUMER_MEDIA_SELECT_CABLE = 0x0097, - HID_USAGE_CONSUMER_MEDIA_SELECT_SATELLITE = 0x0098, - HID_USAGE_CONSUMER_MEDIA_SELECT_SECURITY = 0x0099, - HID_USAGE_CONSUMER_MEDIA_SELECT_HOME = 0x009A, - HID_USAGE_CONSUMER_MEDIA_SELECT_CALL = 0x009B, - HID_USAGE_CONSUMER_CHANNEL_INCREMENT = 0x009C, - HID_USAGE_CONSUMER_CHANNEL_DECREMENT = 0x009D, - HID_USAGE_CONSUMER_MEDIA_SELECT_SAP = 0x009E, + HID_USAGE_CONSUMER_DATA_ON_SCREEN = 0x0060, // OOC + HID_USAGE_CONSUMER_CLOSED_CAPTION = 0x0061, // OOC + HID_USAGE_CONSUMER_CLOSED_CAPTION_SELECT = 0x0062, // OSC + HID_USAGE_CONSUMER_VCR_TV = 0x0063, // OOC + HID_USAGE_CONSUMER_BROADCAST_MODE = 0x0064, // OSC + HID_USAGE_CONSUMER_SNAPSHOT = 0x0065, // OSC + HID_USAGE_CONSUMER_STILL = 0x0066, // OSC + HID_USAGE_CONSUMER_PICTURE_IN_PICTURE_TOGGLE = 0x0067, // OSC + HID_USAGE_CONSUMER_PICTURE_IN_PICTURE_SWAP = 0x0068, // OSC + HID_USAGE_CONSUMER_RED_MENU_BUTTON = 0x0069, // MC + HID_USAGE_CONSUMER_GREEN_MENU_BUTTON = 0x006A, // MC + HID_USAGE_CONSUMER_BLUE_MENU_BUTTON = 0x006B, // MC + HID_USAGE_CONSUMER_YELLOW_MENU_BUTTON = 0x006C, // MC + HID_USAGE_CONSUMER_ASPECT = 0x006D, // OSC + HID_USAGE_CONSUMER_3D_MODE_SELECT = 0x006E, // OSC + HID_USAGE_CONSUMER_DISPLAY_BRIGHTNESS_INCREMENT = 0x006F, // RTC + HID_USAGE_CONSUMER_DISPLAY_BRIGHTNESS_DECREMENT = 0x0070, // RTC + HID_USAGE_CONSUMER_DISPLAY_BRIGHTNESS = 0x0071, // LC + HID_USAGE_CONSUMER_DISPLAY_BACKLIGHT_TOGGLE = 0x0072, // OOC + HID_USAGE_CONSUMER_DISPLAY_SET_BRIGHTNESS_TO_MINIMUM = 0x0073, // OSC + HID_USAGE_CONSUMER_DISPLAY_SET_BRIGHTNESS_TO_MAXIMUM = 0x0074, // OSC + HID_USAGE_CONSUMER_DISPLAY_SET_AUTO_BRIGHTNESS = 0x0075, // OOC + HID_USAGE_CONSUMER_CAMERA_ACCESS_ENABLED = 0x0076, // OOC + HID_USAGE_CONSUMER_CAMERA_ACCESS_DISABLED = 0x0077, // OOC + HID_USAGE_CONSUMER_CAMERA_ACCESS_TOGGLE = 0x0078, // OOC + HID_USAGE_CONSUMER_KEYBOARD_BRIGHTNESS_INCREMENT = 0x0079, // OSC + HID_USAGE_CONSUMER_KEYBOARD_BRIGHTNESS_DECREMENT = 0x007A, // OSC + HID_USAGE_CONSUMER_KEYBOARD_BACKLIGHT_SET_LEVEL = 0x007B, // LC + HID_USAGE_CONSUMER_KEYBOARD_BACKLIGHT_OOC = 0x007C, // OOC + HID_USAGE_CONSUMER_KEYBOARD_BACKLIGHT_SET_MINIMUM = 0x007D, // OSC + HID_USAGE_CONSUMER_KEYBOARD_BACKLIGHT_SET_MAXIMUM = 0x007E, // OSC + HID_USAGE_CONSUMER_KEYBOARD_BACKLIGHT_AUTO = 0x007F, // OOC + HID_USAGE_CONSUMER_SELECTION = 0x0080, // NAry + HID_USAGE_CONSUMER_ASSIGN_SELECTION = 0x0081, // OSC + HID_USAGE_CONSUMER_MODE_STEP = 0x0082, // OSC + HID_USAGE_CONSUMER_RECALL_LAST = 0x0083, // OSC + HID_USAGE_CONSUMER_ENTER_CHANNEL = 0x0084, // OSC + HID_USAGE_CONSUMER_ORDER_MOVIE = 0x0085, // OSC + HID_USAGE_CONSUMER_CHANNEL = 0x0086, // LC + HID_USAGE_CONSUMER_MEDIA_SELECTION = 0x0087, // NAry + HID_USAGE_CONSUMER_MEDIA_SELECT_COMPUTER = 0x0088, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_TV = 0x0089, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_WWW = 0x008A, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_DVD = 0x008B, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_TELEPHONE = 0x008C, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_PROGRAM_GUIDE = 0x008D, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_VIDEO_PHONE = 0x008E, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_GAMES = 0x008F, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_MESSAGES = 0x0090, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_CD = 0x0091, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_VCR = 0x0092, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_TUNER = 0x0093, // Sel + HID_USAGE_CONSUMER_QUIT = 0x0094, // OSC + HID_USAGE_CONSUMER_HELP = 0x0095, // OOC + HID_USAGE_CONSUMER_MEDIA_SELECT_TAPE = 0x0096, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_CABLE = 0x0097, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_SATELLITE = 0x0098, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_SECURITY = 0x0099, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_HOME = 0x009A, // Sel + HID_USAGE_CONSUMER_MEDIA_SELECT_CALL = 0x009B, // Sel + HID_USAGE_CONSUMER_CHANNEL_INCREMENT = 0x009C, // OSC + HID_USAGE_CONSUMER_CHANNEL_DECREMENT = 0x009D, // OSC + HID_USAGE_CONSUMER_MEDIA_SELECT_SAP = 0x009E, // Sel // 9F Reserved - HID_USAGE_CONSUMER_VCR_PLUS = 0x00A0, - HID_USAGE_CONSUMER_ONCE = 0x00A1, - HID_USAGE_CONSUMER_DAILY = 0x00A2, - HID_USAGE_CONSUMER_WEEKLY = 0x00A3, - HID_USAGE_CONSUMER_MONTHLY = 0x00A4, + HID_USAGE_CONSUMER_VCR_PLUS = 0x00A0, // OSC + HID_USAGE_CONSUMER_ONCE = 0x00A1, // OSC + HID_USAGE_CONSUMER_DAILY = 0x00A2, // OSC + HID_USAGE_CONSUMER_WEEKLY = 0x00A3, // OSC + HID_USAGE_CONSUMER_MONTHLY = 0x00A4, // OSC // A5-AF Reserved - HID_USAGE_CONSUMER_PLAY = 0x00B0, - HID_USAGE_CONSUMER_PAUSE = 0x00B1, - HID_USAGE_CONSUMER_RECORD = 0x00B2, - HID_USAGE_CONSUMER_FAST_FORWARD = 0x00B3, - HID_USAGE_CONSUMER_REWIND = 0x00B4, - HID_USAGE_CONSUMER_SCAN_NEXT_TRACK = 0x00B5, - HID_USAGE_CONSUMER_SCAN_PREVIOUS_TRACK = 0x00B6, - HID_USAGE_CONSUMER_STOP = 0x00B7, - HID_USAGE_CONSUMER_EJECT = 0x00B8, - HID_USAGE_CONSUMER_RANDOM_PLAY = 0x00B9, - HID_USAGE_CONSUMER_SELECT_DISC = 0x00BA, - HID_USAGE_CONSUMER_ENTER_DISC = 0x00BB, - HID_USAGE_CONSUMER_REPEAT = 0x00BC, - HID_USAGE_CONSUMER_TRACKING = 0x00BD, - HID_USAGE_CONSUMER_TRACK_NORMAL = 0x00BE, - HID_USAGE_CONSUMER_SLOW_TRACKING = 0x00BF, - HID_USAGE_CONSUMER_FRAME_FORWARD = 0x00C0, - HID_USAGE_CONSUMER_FRAME_BACK = 0x00C1, - HID_USAGE_CONSUMER_MARK = 0x00C2, - HID_USAGE_CONSUMER_CLEAR_MARK = 0x00C3, - HID_USAGE_CONSUMER_REPEAT_FROM_MARK = 0x00C4, - HID_USAGE_CONSUMER_RETURN_TO_MARK = 0x00C5, - HID_USAGE_CONSUMER_SEARCH_MARK_FORWARD = 0x00C6, - HID_USAGE_CONSUMER_SEARCH_MARK_BACKWARDS = 0x00C7, - HID_USAGE_CONSUMER_COUNTER_RESET = 0x00C8, - HID_USAGE_CONSUMER_SHOW_COUNTER = 0x00C9, - HID_USAGE_CONSUMER_TRACKING_INCREMENT = 0x00CA, - HID_USAGE_CONSUMER_TRACKING_DECREMENT = 0x00CB, - HID_USAGE_CONSUMER_STOP_EJECT = 0x00CC, + HID_USAGE_CONSUMER_PLAY = 0x00B0, // OOC + HID_USAGE_CONSUMER_PAUSE = 0x00B1, // OOC + HID_USAGE_CONSUMER_RECORD = 0x00B2, // OOC + HID_USAGE_CONSUMER_FAST_FORWARD = 0x00B3, // OOC + HID_USAGE_CONSUMER_REWIND = 0x00B4, // OOC + HID_USAGE_CONSUMER_SCAN_NEXT_TRACK = 0x00B5, // OSC + HID_USAGE_CONSUMER_SCAN_PREVIOUS_TRACK = 0x00B6, // OSC + HID_USAGE_CONSUMER_STOP = 0x00B7, // OSC + HID_USAGE_CONSUMER_EJECT = 0x00B8, // OSC + HID_USAGE_CONSUMER_RANDOM_PLAY = 0x00B9, // OOC + HID_USAGE_CONSUMER_SELECT_DISC = 0x00BA, // NAry + HID_USAGE_CONSUMER_ENTER_DISC = 0x00BB, // MC + HID_USAGE_CONSUMER_REPEAT = 0x00BC, // OSC + HID_USAGE_CONSUMER_TRACKING = 0x00BD, // LC + HID_USAGE_CONSUMER_TRACK_NORMAL = 0x00BE, // OSC + HID_USAGE_CONSUMER_SLOW_TRACKING = 0x00BF, // LC + HID_USAGE_CONSUMER_FRAME_FORWARD = 0x00C0, // RTC + HID_USAGE_CONSUMER_FRAME_BACK = 0x00C1, // RTC + HID_USAGE_CONSUMER_MARK = 0x00C2, // OSC + HID_USAGE_CONSUMER_CLEAR_MARK = 0x00C3, // OSC + HID_USAGE_CONSUMER_REPEAT_FROM_MARK = 0x00C4, // OOC + HID_USAGE_CONSUMER_RETURN_TO_MARK = 0x00C5, // OSC + HID_USAGE_CONSUMER_SEARCH_MARK_FORWARD = 0x00C6, // OSC + HID_USAGE_CONSUMER_SEARCH_MARK_BACKWARDS = 0x00C7, // OSC + HID_USAGE_CONSUMER_COUNTER_RESET = 0x00C8, // OSC - // Media Control - HID_USAGE_CONSUMER_PLAY_PAUSE = 0x00CD, + // These HID usages operate only on mobile systems (battery powered) and + // require Windows 8 (build 8302 or greater). + HID_USAGE_CONSUMER_WIRELESS_RADIO_CONTROLS = 0x000C, + HID_USAGE_CONSUMER_WIRELESS_RADIO_BUTTONS = 0x00C6, + HID_USAGE_CONSUMER_WIRELESS_RADIO_LED = 0x00C7, + HID_USAGE_CONSUMER_WIRELESS_RADIO_SLIDER_SWITCH = 0x00C8, - HID_USAGE_CONSUMER_PLAY_SKIP = 0x00CE, - // CF-DF Reserved - HID_USAGE_CONSUMER_VOLUME = 0x00E0, - HID_USAGE_CONSUMER_BALANCE = 0x00E1, - HID_USAGE_CONSUMER_MUTE = 0x00E2, - HID_USAGE_CONSUMER_BASS = 0x00E3, - HID_USAGE_CONSUMER_TREBLE = 0x00E4, - HID_USAGE_CONSUMER_BASS_BOOST = 0x00E5, - HID_USAGE_CONSUMER_SURROUND_MODE = 0x00E6, - HID_USAGE_CONSUMER_LOUDNESS = 0x00E7, - HID_USAGE_CONSUMER_MPX = 0x00E8, - HID_USAGE_CONSUMER_VOLUME_INCREMENT = 0x00E9, - HID_USAGE_CONSUMER_VOLUME_DECREMENT = 0x00EA, + HID_USAGE_CONSUMER_SHOW_COUNTER = 0x00C9, // OSC + HID_USAGE_CONSUMER_TRACKING_INCREMENT = 0x00CA, // RTC + HID_USAGE_CONSUMER_TRACKING_DECREMENT = 0x00CB, // RTC + HID_USAGE_CONSUMER_STOP_EJECT = 0x00CC, // OSC + HID_USAGE_CONSUMER_PLAY_PAUSE = 0x00CD, // OSC + HID_USAGE_CONSUMER_PLAY_SKIP = 0x00CE, // OSC + HID_USAGE_CONSUMER_VOICE_COMMAND = 0x00CF, // OSC + HID_USAGE_CONSUMER_INVOKE_CAPTURE_INTERFACE = 0x00D0, // Sel + HID_USAGE_CONSUMER_START_OR_STOP_GAME_RECORDING = 0x00D1, // Sel + HID_USAGE_CONSUMER_HISTORICAL_GAME_CAPTURE = 0x00D2, // Sel + HID_USAGE_CONSUMER_CAPTURE_GAME_SCREENSHOT = 0x00D3, // Sel + HID_USAGE_CONSUMER_SHOW_OR_HIDE_RECORDING_INDICATOR = 0x00D4, // Sel + HID_USAGE_CONSUMER_START_OR_STOP_MICROPHONE_CAPTURE = 0x00D5, // Sel + HID_USAGE_CONSUMER_START_OR_STOP_CAMERA_CAPTURE = 0x00D6, // Sel + HID_USAGE_CONSUMER_START_OR_STOP_GAME_BROADCAST = 0x00D7, // Sel + HID_USAGE_CONSUMER_START_OR_STOP_VOICE_DICTATION_SESSION = 0x00D8, // OOC + HID_USAGE_CONSUMER_INVOKE_DISMISS_EMOJI_PICKER = 0x00D9, // OOC + // DA-DF Reserved + HID_USAGE_CONSUMER_VOLUME = 0x00E0, // LC + HID_USAGE_CONSUMER_BALANCE = 0x00E1, // LC + HID_USAGE_CONSUMER_MUTE = 0x00E2, // OOC + HID_USAGE_CONSUMER_BASS = 0x00E3, // LC + HID_USAGE_CONSUMER_TREBLE = 0x00E4, // LC + HID_USAGE_CONSUMER_BASS_BOOST = 0x00E5, // OOC + HID_USAGE_CONSUMER_SURROUND_MODE = 0x00E6, // OSC + HID_USAGE_CONSUMER_LOUDNESS = 0x00E7, // OOC + HID_USAGE_CONSUMER_MPX = 0x00E8, // OOC + HID_USAGE_CONSUMER_VOLUME_INCREMENT = 0x00E9, // RTC + HID_USAGE_CONSUMER_VOLUME_DECREMENT = 0x00EA, // RTC // EB-EF Reserved - HID_USAGE_CONSUMER_SPEED_SELECT = 0x00F0, - HID_USAGE_CONSUMER_PLAYBACK_SPEED = 0x00F1, - HID_USAGE_CONSUMER_STANDARD_PLAY = 0x00F2, - HID_USAGE_CONSUMER_LONG_PLAY = 0x00F3, - HID_USAGE_CONSUMER_EXTENDED_PLAY = 0x00F4, - HID_USAGE_CONSUMER_SLOW = 0x00F5, + HID_USAGE_CONSUMER_SPEED_SELECT = 0x00F0, // OSC + HID_USAGE_CONSUMER_PLAYBACK_SPEED = 0x00F1, // NAry + HID_USAGE_CONSUMER_STANDARD_PLAY = 0x00F2, // Sel + HID_USAGE_CONSUMER_LONG_PLAY = 0x00F3, // Sel + HID_USAGE_CONSUMER_EXTENDED_PLAY = 0x00F4, // Sel + HID_USAGE_CONSUMER_SLOW = 0x00F5, // OSC // F6-FF Reserved - HID_USAGE_CONSUMER_FAN_ENABLE = 0x0100, - HID_USAGE_CONSUMER_FAN_SPEED = 0x0101, - HID_USAGE_CONSUMER_LIGHT_ENABLE = 0x0102, - HID_USAGE_CONSUMER_LIGHT_ILLUMINATION_LEVEL = 0x0103, - HID_USAGE_CONSUMER_CLIMATE_CONTROL_ENABLE = 0x0104, - HID_USAGE_CONSUMER_ROOM_TEMPERATURE = 0x0105, - HID_USAGE_CONSUMER_SECURITY_ENABLE = 0x0106, - HID_USAGE_CONSUMER_FIRE_ALARM = 0x0107, - HID_USAGE_CONSUMER_POLICE_ALARM = 0x0108, - HID_USAGE_CONSUMER_PROXIMITY = 0x0109, - HID_USAGE_CONSUMER_MOTION = 0x010A, - HID_USAGE_CONSUMER_DURESS_ALARM = 0x010B, - HID_USAGE_CONSUMER_HOLDUP_ALARM = 0x010C, - HID_USAGE_CONSUMER_MEDICAL_ALARM = 0x010D, + HID_USAGE_CONSUMER_FAN_ENABLE = 0x0100, // OOC + HID_USAGE_CONSUMER_FAN_SPEED = 0x0101, // LC + HID_USAGE_CONSUMER_LIGHT_ENABLE = 0x0102, // OOC + HID_USAGE_CONSUMER_LIGHT_ILLUMINATION_LEVEL = 0x0103, // LC + HID_USAGE_CONSUMER_CLIMATE_CONTROL_ENABLE = 0x0104, // OOC + HID_USAGE_CONSUMER_ROOM_TEMPERATURE = 0x0105, // LC + HID_USAGE_CONSUMER_SECURITY_ENABLE = 0x0106, // OOC + HID_USAGE_CONSUMER_FIRE_ALARM = 0x0107, // OSC + HID_USAGE_CONSUMER_POLICE_ALARM = 0x0108, // OSC + HID_USAGE_CONSUMER_PROXIMITY = 0x0109, // LC + HID_USAGE_CONSUMER_MOTION = 0x010A, // OSC + HID_USAGE_CONSUMER_DURESS_ALARM = 0x010B, // OSC + HID_USAGE_CONSUMER_HOLDUP_ALARM = 0x010C, // OSC + HID_USAGE_CONSUMER_MEDICAL_ALARM = 0x010D, // OSC // 10E-14F Reserved - HID_USAGE_CONSUMER_BALANCE_RIGHT = 0x0150, - HID_USAGE_CONSUMER_BALANCE_LEFT = 0x0151, - HID_USAGE_CONSUMER_BASS_INCREMENT = 0x0152, - HID_USAGE_CONSUMER_BASS_DECREMENT = 0x0153, - HID_USAGE_CONSUMER_TREBLE_INCREMENT = 0x0154, - HID_USAGE_CONSUMER_TREBLE_DECREMENT = 0x0155, + HID_USAGE_CONSUMER_BALANCE_RIGHT = 0x0150, // RTC + HID_USAGE_CONSUMER_BALANCE_LEFT = 0x0151, // RTC + HID_USAGE_CONSUMER_BASS_INCREMENT = 0x0152, // RTC + HID_USAGE_CONSUMER_BASS_DECREMENT = 0x0153, // RTC + HID_USAGE_CONSUMER_TREBLE_INCREMENT = 0x0154, // RTC + HID_USAGE_CONSUMER_TREBLE_DECREMENT = 0x0155, // RTC // 156-15F Reserved - HID_USAGE_CONSUMER_SPEAKER_SYSTEM = 0x0160, - HID_USAGE_CONSUMER_CHANNEL_LEFT = 0x0161, - HID_USAGE_CONSUMER_CHANNEL_RIGHT = 0x0162, - HID_USAGE_CONSUMER_CHANNEL_CENTER = 0x0163, - HID_USAGE_CONSUMER_CHANNEL_FRONT = 0x0164, - HID_USAGE_CONSUMER_CHANNEL_CENTER_FRONT = 0x0165, - HID_USAGE_CONSUMER_CHANNEL_SIDE = 0x0166, - HID_USAGE_CONSUMER_CHANNEL_SURROUND = 0x0167, - HID_USAGE_CONSUMER_CHANNEL_LOW_FREQUENCY = 0x0168, - // Enhancement - // CL 15.12.1 - HID_USAGE_CONSUMER_CHANNEL_TOP = 0x0169, - HID_USAGE_CONSUMER_CHANNEL_UNKNOWN = 0x016A, + HID_USAGE_CONSUMER_SPEAKER_SYSTEM = 0x0160, // CL + HID_USAGE_CONSUMER_CHANNEL_LEFT = 0x0161, // CL + HID_USAGE_CONSUMER_CHANNEL_RIGHT = 0x0162, // CL + HID_USAGE_CONSUMER_CHANNEL_CENTER = 0x0163, // CL + HID_USAGE_CONSUMER_CHANNEL_FRONT = 0x0164, // CL + HID_USAGE_CONSUMER_CHANNEL_CENTER_FRONT = 0x0165, // CL + HID_USAGE_CONSUMER_CHANNEL_SIDE = 0x0166, // CL + HID_USAGE_CONSUMER_CHANNEL_SURROUND = 0x0167, // CL + HID_USAGE_CONSUMER_CHANNEL_LOW_FREQUENCY = 0x0168, // CL + HID_USAGE_CONSUMER_CHANNEL_TOP = 0x0169, // CL + HID_USAGE_CONSUMER_CHANNEL_UNKNOWN = 0x016A, // CL // 16B-16F Reserved - HID_USAGE_CONSUMER_SUB_CHANNEL = 0x0170, - HID_USAGE_CONSUMER_SUB_CHANNEL_INCREMENT = 0x0171, - HID_USAGE_CONSUMER_SUB_CHANNEL_DECREMENT = 0x0172, - HID_USAGE_CONSUMER_ALTERNATE_AUDIO_INCREMENT = 0x0173, - HID_USAGE_CONSUMER_ALTERNATE_AUDIO_DECREMENT = 0x0174, + HID_USAGE_CONSUMER_SUB_CHANNEL = 0x0170, // LC + HID_USAGE_CONSUMER_SUB_CHANNEL_INCREMENT = 0x0171, // OSC + HID_USAGE_CONSUMER_SUB_CHANNEL_DECREMENT = 0x0172, // OSC + HID_USAGE_CONSUMER_ALTERNATE_AUDIO_INCREMENT = 0x0173, // OSC + HID_USAGE_CONSUMER_ALTERNATE_AUDIO_DECREMENT = 0x0174, // OSC // 175-17F Reserved - HID_USAGE_CONSUMER_APPLICATION_LAUNCH_BUTTONS = 0x0180, - HID_USAGE_CONSUMER_AL_LAUNCH_BUTTON_CONFIGURATION = 0x0181, - // Tool - // Sel 15.15 - HID_USAGE_CONSUMER_AL_PROGRAMMABLE_BUTTON = 0x0182, - // Configuration - // Sel 15.15 - HID_USAGE_CONSUMER_AL_CONSUMER_CONTROL_CONFIGURATION = 0x0183, - // Configuration - // Sel 15.15 - HID_USAGE_CONSUMER_AL_WORD_PROCESSOR = 0x0184, - HID_USAGE_CONSUMER_AL_TEXT_EDITOR = 0x0185, - HID_USAGE_CONSUMER_AL_SPREADSHEET = 0x0186, - HID_USAGE_CONSUMER_AL_GRAPHICS_EDITOR = 0x0187, - HID_USAGE_CONSUMER_AL_PRESENTATION_APP = 0x0188, - HID_USAGE_CONSUMER_AL_DATABASE_APP = 0x0189, - HID_USAGE_CONSUMER_AL_EMAIL_READER = 0x018A, - HID_USAGE_CONSUMER_AL_NEWSREADER = 0x018B, - HID_USAGE_CONSUMER_AL_VOICEMAIL = 0x018C, - HID_USAGE_CONSUMER_AL_CONTACTS_ADDRESS_BOOK = 0x018D, - HID_USAGE_CONSUMER_AL_CALENDAR_SCHEDULE = 0x018E, - HID_USAGE_CONSUMER_AL_TASK_PROJECT_MANAGER = 0x018F, - HID_USAGE_CONSUMER_AL_LOG_JOURNAL_TIMECARD = 0x0190, - HID_USAGE_CONSUMER_AL_CHECKBOOK_FINANCE = 0x0191, - HID_USAGE_CONSUMER_AL_CALCULATOR = 0x0192, - HID_USAGE_CONSUMER_AL_A_V_CAPTURE_PLAYBACK = 0x0193, - HID_USAGE_CONSUMER_AL_LOCAL_MACHINE_BROWSER = 0x0194, - HID_USAGE_CONSUMER_AL_LAN_WAN_BROWSER = 0x0195, - HID_USAGE_CONSUMER_AL_INTERNET_BROWSER = 0x0196, - HID_USAGE_CONSUMER_AL_REMOTE_NETWORKING_ISP = 0x0197, - // Connect - // Sel 15.15 - HID_USAGE_CONSUMER_AL_NETWORK_CONFERENCE = 0x0198, - HID_USAGE_CONSUMER_AL_NETWORK_CHAT = 0x0199, - HID_USAGE_CONSUMER_AL_TELEPHONY_DIALER = 0x019A, - HID_USAGE_CONSUMER_AL_LOGON = 0x019B, - HID_USAGE_CONSUMER_AL_LOGOFF = 0x019C, - HID_USAGE_CONSUMER_AL_LOGON_LOGOFF = 0x019D, - HID_USAGE_CONSUMER_AL_TERMINAL_LOCK_SCREENSAVER = 0x019E, - HID_USAGE_CONSUMER_AL_CONTROL_PANEL = 0x019F, - HID_USAGE_CONSUMER_AL_COMMAND_LINE_PROCESSOR_RUN = 0x01A0, - HID_USAGE_CONSUMER_AL_PROCESS_TASK_MANAGER = 0x01A1, - HID_USAGE_CONSUMER_AL_SELECT_TASK_APPLICATION = 0x01A2, - HID_USAGE_CONSUMER_AL_NEXT_TASK_APPLICATION = 0x01A3, - HID_USAGE_CONSUMER_AL_PREVIOUS_TASK_APPLICATION = 0x01A4, - HID_USAGE_CONSUMER_AL_PREEMPTIVE_HALT = 0x01A5, - // Task_Application - // Sel 15.15 - HID_USAGE_CONSUMER_AL_INTEGRATED_HELP_CENTER = 0x01A6, - HID_USAGE_CONSUMER_AL_DOCUMENTS = 0x01A7, - HID_USAGE_CONSUMER_AL_THESAURUS = 0x01A8, - HID_USAGE_CONSUMER_AL_DICTIONARY = 0x01A9, - HID_USAGE_CONSUMER_AL_DESKTOP = 0x01AA, - HID_USAGE_CONSUMER_AL_SPELL_CHECK = 0x01AB, - HID_USAGE_CONSUMER_AL_GRAMMAR_CHECK = 0x01AC, - HID_USAGE_CONSUMER_AL_WIRELESS_STATUS = 0x01AD, - HID_USAGE_CONSUMER_AL_KEYBOARD_LAYOUT = 0x01AE, - HID_USAGE_CONSUMER_AL_VIRUS_PROTECTION = 0x01AF, - HID_USAGE_CONSUMER_AL_ENCRYPTION = 0x01B0, - HID_USAGE_CONSUMER_AL_SCREEN_SAVER = 0x01B1, - HID_USAGE_CONSUMER_AL_ALARMS = 0x01B2, - HID_USAGE_CONSUMER_AL_CLOCK = 0x01B3, - HID_USAGE_CONSUMER_AL_FILE_BROWSER = 0x01B4, - HID_USAGE_CONSUMER_AL_POWER_STATUS = 0x01B5, - HID_USAGE_CONSUMER_AL_IMAGE_BROWSER = 0x01B6, - HID_USAGE_CONSUMER_AL_AUDIO_BROWSER = 0x01B7, - HID_USAGE_CONSUMER_AL_MOVIE_BROWSER = 0x01B8, - HID_USAGE_CONSUMER_AL_DIGITAL_RIGHTS_MANAGER = 0x01B9, - HID_USAGE_CONSUMER_AL_DIGITAL_WALLET = 0x01BA, + HID_USAGE_CONSUMER_APPLICATION_LAUNCH_BUTTONS = 0x0180, // NAry + HID_USAGE_CONSUMER_AL_LAUNCH_BUTTON_CONFIGURATION = 0x0181, // Sel + HID_USAGE_CONSUMER_AL_PROGRAMMABLE_BUTTON = 0x0182, // Sel + HID_USAGE_CONSUMER_AL_CONSUMER_CONTROL_CONFIGURATION = 0x0183, // Sel + HID_USAGE_CONSUMER_AL_WORD_PROCESSOR = 0x0184, // Sel + HID_USAGE_CONSUMER_AL_TEXT_EDITOR = 0x0185, // Sel + HID_USAGE_CONSUMER_AL_SPREADSHEET = 0x0186, // Sel + HID_USAGE_CONSUMER_AL_GRAPHICS_EDITOR = 0x0187, // Sel + HID_USAGE_CONSUMER_AL_PRESENTATION_APP = 0x0188, // Sel + HID_USAGE_CONSUMER_AL_DATABASE_APP = 0x0189, // Sel + HID_USAGE_CONSUMER_AL_EMAIL_READER = 0x018A, // Sel + HID_USAGE_CONSUMER_AL_NEWSREADER = 0x018B, // Sel + HID_USAGE_CONSUMER_AL_VOICEMAIL = 0x018C, // Sel + HID_USAGE_CONSUMER_AL_CONTACTS_ADDRESS_BOOK = 0x018D, // Sel + HID_USAGE_CONSUMER_AL_CALENDAR_SCHEDULE = 0x018E, // Sel + HID_USAGE_CONSUMER_AL_TASK_PROJECT_MANAGER = 0x018F, // Sel + HID_USAGE_CONSUMER_AL_LOG_JOURNAL_TIMECARD = 0x0190, // Sel + HID_USAGE_CONSUMER_AL_CHECKBOOK_FINANCE = 0x0191, // Sel + HID_USAGE_CONSUMER_AL_CALCULATOR = 0x0192, // Sel + HID_USAGE_CONSUMER_AL_A_V_CAPTURE_PLAYBACK = 0x0193, // Sel + HID_USAGE_CONSUMER_AL_LOCAL_MACHINE_BROWSER = 0x0194, // Sel + HID_USAGE_CONSUMER_AL_LAN_WAN_BROWSER = 0x0195, // Sel + HID_USAGE_CONSUMER_AL_INTERNET_BROWSER = 0x0196, // Sel + HID_USAGE_CONSUMER_AL_REMOTE_NETWORKING_ISP = 0x0197, // Sel + HID_USAGE_CONSUMER_AL_NETWORK_CONFERENCE = 0x0198, // Sel + HID_USAGE_CONSUMER_AL_NETWORK_CHAT = 0x0199, // Sel + HID_USAGE_CONSUMER_AL_TELEPHONY_DIALER = 0x019A, // Sel + HID_USAGE_CONSUMER_AL_LOGON = 0x019B, // Sel + HID_USAGE_CONSUMER_AL_LOGOFF = 0x019C, // Sel + HID_USAGE_CONSUMER_AL_LOGON_LOGOFF = 0x019D, // Sel + HID_USAGE_CONSUMER_AL_TERMINAL_LOCK_SCREENSAVER = 0x019E, // Sel + HID_USAGE_CONSUMER_AL_CONTROL_PANEL = 0x019F, // Sel + HID_USAGE_CONSUMER_AL_COMMAND_LINE_PROCESSOR_RUN = 0x01A0, // Sel + HID_USAGE_CONSUMER_AL_PROCESS_TASK_MANAGER = 0x01A1, // Sel + HID_USAGE_CONSUMER_AL_SELECT_TASK_APPLICATION = 0x01A2, // Sel + HID_USAGE_CONSUMER_AL_NEXT_TASK_APPLICATION = 0x01A3, // Sel + HID_USAGE_CONSUMER_AL_PREVIOUS_TASK_APPLICATION = 0x01A4, // Sel + HID_USAGE_CONSUMER_AL_PREEMPTIVE_HALT = 0x01A5, // Sel + HID_USAGE_CONSUMER_AL_INTEGRATED_HELP_CENTER = 0x01A6, // Sel + HID_USAGE_CONSUMER_AL_DOCUMENTS = 0x01A7, // Sel + HID_USAGE_CONSUMER_AL_THESAURUS = 0x01A8, // Sel + HID_USAGE_CONSUMER_AL_DICTIONARY = 0x01A9, // Sel + HID_USAGE_CONSUMER_AL_DESKTOP = 0x01AA, // Sel + HID_USAGE_CONSUMER_AL_SPELL_CHECK = 0x01AB, // Sel + HID_USAGE_CONSUMER_AL_GRAMMAR_CHECK = 0x01AC, // Sel + HID_USAGE_CONSUMER_AL_WIRELESS_STATUS = 0x01AD, // Sel + HID_USAGE_CONSUMER_AL_KEYBOARD_LAYOUT = 0x01AE, // Sel + HID_USAGE_CONSUMER_AL_VIRUS_PROTECTION = 0x01AF, // Sel + HID_USAGE_CONSUMER_AL_ENCRYPTION = 0x01B0, // Sel + HID_USAGE_CONSUMER_AL_SCREEN_SAVER = 0x01B1, // Sel + HID_USAGE_CONSUMER_AL_ALARMS = 0x01B2, // Sel + HID_USAGE_CONSUMER_AL_CLOCK = 0x01B3, // Sel + HID_USAGE_CONSUMER_AL_FILE_BROWSER = 0x01B4, // Sel + HID_USAGE_CONSUMER_AL_POWER_STATUS = 0x01B5, // Sel + HID_USAGE_CONSUMER_AL_IMAGE_BROWSER = 0x01B6, // Sel + HID_USAGE_CONSUMER_AL_AUDIO_BROWSER = 0x01B7, // Sel + HID_USAGE_CONSUMER_AL_MOVIE_BROWSER = 0x01B8, // Sel + HID_USAGE_CONSUMER_AL_DIGITAL_RIGHTS_MANAGER = 0x01B9, // Sel + HID_USAGE_CONSUMER_AL_DIGITAL_WALLET = 0x01BA, // Sel // 1BB Reserved - HID_USAGE_CONSUMER_AL_INSTANT_MESSAGING = 0x01BC, - HID_USAGE_CONSUMER_AL_OEM_FEATURES_TIPS_TUTORIAL = 0x01BD, - // Browser - // Sel 15.15 - HID_USAGE_CONSUMER_AL_OEM_HELP = 0x01BE, - HID_USAGE_CONSUMER_AL_ONLINE_COMMUNITY = 0x01BF, - HID_USAGE_CONSUMER_AL_ENTERTAINMENT_CONTENT = 0x01C0, - // Browser - // Sel 15.15 - HID_USAGE_CONSUMER_AL_ONLINE_SHOPPING_BROWSER = 0x01C1, - HID_USAGE_CONSUMER_AL_SMARTCARD_INFORMATION_HELP = 0x01C2, - HID_USAGE_CONSUMER_AL_MARKET_MONITOR_FINANCE = 0x01C3, - // Browser - // Sel 15.15 - HID_USAGE_CONSUMER_AL_CUSTOMIZED_CORPORATE_NEWS = 0x01C4, - // Browser - // Sel 15.15 - HID_USAGE_CONSUMER_AL_ONLINE_ACTIVITY_BROWSER = 0x01C5, - HID_USAGE_CONSUMER_AL_RESEARCH_SEARCH_BROWSER = 0x01C6, - HID_USAGE_CONSUMER_AL_AUDIO_PLAYER = 0x01C7, - // 1C8-1FF Reserved - HID_USAGE_CONSUMER_GENERIC_GUI_APPLICATION = 0x0200, - // ' Controls - // ' - HID_USAGE_CONSUMER_AC_NEW = 0x0201, - HID_USAGE_CONSUMER_AC_OPEN = 0x0202, - HID_USAGE_CONSUMER_AC_CLOSE = 0x0203, - HID_USAGE_CONSUMER_AC_EXIT = 0x0204, - HID_USAGE_CONSUMER_AC_MAXIMIZE = 0x0205, - HID_USAGE_CONSUMER_AC_MINIMIZE = 0x0206, - HID_USAGE_CONSUMER_AC_SAVE = 0x0207, - HID_USAGE_CONSUMER_AC_PRINT = 0x0208, - HID_USAGE_CONSUMER_AC_PROPERTIES = 0x0209, - HID_USAGE_CONSUMER_AC_UNDO = 0x021A, - HID_USAGE_CONSUMER_AC_COPY = 0x021B, - HID_USAGE_CONSUMER_AC_CUT = 0x021C, - HID_USAGE_CONSUMER_AC_PASTE = 0x021D, - HID_USAGE_CONSUMER_AC_SELECT_ALL = 0x021E, - HID_USAGE_CONSUMER_AC_FIND = 0x021F, - HID_USAGE_CONSUMER_AC_FIND_AND_REPLACE = 0x0220, - // Browser/Explorer Specific - HID_USAGE_CONSUMER_AC_SEARCH = 0x0221, - HID_USAGE_CONSUMER_AC_GO_TO = 0x0222, - HID_USAGE_CONSUMER_AC_HOME = 0x0223, - HID_USAGE_CONSUMER_AC_BACK = 0x0224, - HID_USAGE_CONSUMER_AC_FORWARD = 0x0225, - HID_USAGE_CONSUMER_AC_STOP = 0x0226, - HID_USAGE_CONSUMER_AC_REFRESH = 0x0227, - HID_USAGE_CONSUMER_AC_PREVIOUS_LINK = 0x0228, - HID_USAGE_CONSUMER_AC_NEXT_LINK = 0x0229, - HID_USAGE_CONSUMER_AC_BOOKMARKS = 0x022A, - HID_USAGE_CONSUMER_AC_HISTORY = 0x022B, - HID_USAGE_CONSUMER_AC_SUBSCRIPTIONS = 0x022C, - HID_USAGE_CONSUMER_AC_ZOOM_IN = 0x022D, - HID_USAGE_CONSUMER_AC_ZOOM_OUT = 0x022E, - HID_USAGE_CONSUMER_AC_ZOOM = 0x022F, - HID_USAGE_CONSUMER_AC_FULL_SCREEN_VIEW = 0x0230, - HID_USAGE_CONSUMER_AC_NORMAL_VIEW = 0x0231, - HID_USAGE_CONSUMER_AC_VIEW_TOGGLE = 0x0232, - HID_USAGE_CONSUMER_AC_SCROLL_UP = 0x0233, - HID_USAGE_CONSUMER_AC_SCROLL_DOWN = 0x0234, - HID_USAGE_CONSUMER_AC_SCROLL = 0x0235, - HID_USAGE_CONSUMER_AC_PAN_LEFT = 0x0236, - HID_USAGE_CONSUMER_AC_PAN_RIGHT = 0x0237, - // Mouse Horizontal scroll - HID_USAGE_CONSUMER_AC_PAN = 0x0238, - HID_USAGE_CONSUMER_AC_NEW_WINDOW = 0x0239, - HID_USAGE_CONSUMER_AC_TILE_HORIZONTALLY = 0x023A, - HID_USAGE_CONSUMER_AC_TILE_VERTICALLY = 0x023B, - HID_USAGE_CONSUMER_AC_FORMAT = 0x023C, - HID_USAGE_CONSUMER_AC_EDIT = 0x023D, - HID_USAGE_CONSUMER_AC_BOLD = 0x023E, - HID_USAGE_CONSUMER_AC_ITALICS = 0x023F, - HID_USAGE_CONSUMER_AC_UNDERLINE = 0x0240, - HID_USAGE_CONSUMER_AC_STRIKETHROUGH = 0x0241, - HID_USAGE_CONSUMER_AC_SUBSCRIPT = 0x0242, - HID_USAGE_CONSUMER_AC_SUPERSCRIPT = 0x0243, - HID_USAGE_CONSUMER_AC_ALL_CAPS = 0x0244, - HID_USAGE_CONSUMER_AC_ROTATE = 0x0245, - HID_USAGE_CONSUMER_AC_RESIZE = 0x0246, - HID_USAGE_CONSUMER_AC_FLIP_HORIZONTAL = 0x0247, - HID_USAGE_CONSUMER_AC_FLIP_VERTICAL = 0x0248, - HID_USAGE_CONSUMER_AC_MIRROR_HORIZONTAL = 0x0249, - HID_USAGE_CONSUMER_AC_MIRROR_VERTICAL = 0x024A, - HID_USAGE_CONSUMER_AC_FONT_SELECT = 0x024B, - HID_USAGE_CONSUMER_AC_FONT_COLOR = 0x024C, - HID_USAGE_CONSUMER_AC_FONT_SIZE = 0x024D, - HID_USAGE_CONSUMER_AC_JUSTIFY_LEFT = 0x024E, - HID_USAGE_CONSUMER_AC_JUSTIFY_CENTER_H = 0x024F, - HID_USAGE_CONSUMER_AC_JUSTIFY_RIGHT = 0x0250, - HID_USAGE_CONSUMER_AC_JUSTIFY_BLOCK_H = 0x0251, - HID_USAGE_CONSUMER_AC_JUSTIFY_TOP = 0x0252, - HID_USAGE_CONSUMER_AC_JUSTIFY_CENTER_V = 0x0253, - HID_USAGE_CONSUMER_AC_JUSTIFY_BOTTOM = 0x0254, - HID_USAGE_CONSUMER_AC_JUSTIFY_BLOCK_V = 0x0255, - HID_USAGE_CONSUMER_AC_INDENT_DECREASE = 0x0256, - HID_USAGE_CONSUMER_AC_INDENT_INCREASE = 0x0257, - HID_USAGE_CONSUMER_AC_NUMBERED_LIST = 0x0258, - HID_USAGE_CONSUMER_AC_RESTART_NUMBERING = 0x0259, - HID_USAGE_CONSUMER_AC_BULLETED_LIST = 0x025A, - HID_USAGE_CONSUMER_AC_PROMOTE = 0x025B, - HID_USAGE_CONSUMER_AC_DEMOTE = 0x025C, - HID_USAGE_CONSUMER_AC_YES = 0x025D, - HID_USAGE_CONSUMER_AC_NO = 0x025E, - HID_USAGE_CONSUMER_AC_CANCEL = 0x025F, - HID_USAGE_CONSUMER_AC_CATALOG = 0x0260, - HID_USAGE_CONSUMER_AC_BUY_CHECKOUT = 0x0261, - HID_USAGE_CONSUMER_AC_ADD_TO_CART = 0x0262, - HID_USAGE_CONSUMER_AC_EXPAND = 0x0263, - HID_USAGE_CONSUMER_AC_EXPAND_ALL = 0x0264, - HID_USAGE_CONSUMER_AC_COLLAPSE = 0x0265, - HID_USAGE_CONSUMER_AC_COLLAPSE_ALL = 0x0266, - HID_USAGE_CONSUMER_AC_PRINT_PREVIEW = 0x0267, - HID_USAGE_CONSUMER_AC_PASTE_SPECIAL = 0x0268, - HID_USAGE_CONSUMER_AC_INSERT_MODE = 0x0269, - HID_USAGE_CONSUMER_AC_DELETE = 0x026A, - HID_USAGE_CONSUMER_AC_LOCK = 0x026B, - HID_USAGE_CONSUMER_AC_UNLOCK = 0x026C, - HID_USAGE_CONSUMER_AC_PROTECT = 0x026D, - HID_USAGE_CONSUMER_AC_UNPROTECT = 0x026E, - HID_USAGE_CONSUMER_AC_ATTACH_COMMENT = 0x026F, - HID_USAGE_CONSUMER_AC_DELETE_COMMENT = 0x0270, - HID_USAGE_CONSUMER_AC_VIEW_COMMENT = 0x0271, - HID_USAGE_CONSUMER_AC_SELECT_WORD = 0x0272, - HID_USAGE_CONSUMER_AC_SELECT_SENTENCE = 0x0273, - HID_USAGE_CONSUMER_AC_SELECT_PARAGRAPH = 0x0274, - HID_USAGE_CONSUMER_AC_SELECT_COLUMN = 0x0275, - HID_USAGE_CONSUMER_AC_SELECT_ROW = 0x0276, - HID_USAGE_CONSUMER_AC_SELECT_TABLE = 0x0277, - HID_USAGE_CONSUMER_AC_SELECT_OBJECT = 0x0278, - HID_USAGE_CONSUMER_AC_REDO_REPEAT = 0x0279, - HID_USAGE_CONSUMER_AC_SORT = 0x027A, - HID_USAGE_CONSUMER_AC_SORT_ASCENDING = 0x027B, - HID_USAGE_CONSUMER_AC_SORT_DESCENDING = 0x027C, - HID_USAGE_CONSUMER_AC_FILTER = 0x027D, - HID_USAGE_CONSUMER_AC_SET_CLOCK = 0x027E, - HID_USAGE_CONSUMER_AC_VIEW_CLOCK = 0x027F, - HID_USAGE_CONSUMER_AC_SELECT_TIME_ZONE = 0x0280, - HID_USAGE_CONSUMER_AC_EDIT_TIME_ZONES = 0x0281, - HID_USAGE_CONSUMER_AC_SET_ALARM = 0x0282, - HID_USAGE_CONSUMER_AC_CLEAR_ALARM = 0x0283, - HID_USAGE_CONSUMER_AC_SNOOZE_ALARM = 0x0284, - HID_USAGE_CONSUMER_AC_RESET_ALARM = 0x0285, - HID_USAGE_CONSUMER_AC_SYNCHRONIZE = 0x0286, - HID_USAGE_CONSUMER_AC_SEND_RECEIVE = 0x0287, - HID_USAGE_CONSUMER_AC_SEND_TO = 0x0288, - HID_USAGE_CONSUMER_AC_REPLY = 0x0289, - HID_USAGE_CONSUMER_AC_REPLY_ALL = 0x028A, - HID_USAGE_CONSUMER_AC_FORWARD_MSG = 0x028B, - HID_USAGE_CONSUMER_AC_SEND = 0x028C, - HID_USAGE_CONSUMER_AC_ATTACH_FILE = 0x028D, - HID_USAGE_CONSUMER_AC_UPLOAD = 0x028E, - HID_USAGE_CONSUMER_AC_DOWNLOAD_SAVE_TARGET_AS = 0x028F, - HID_USAGE_CONSUMER_AC_SET_BORDERS = 0x0290, - HID_USAGE_CONSUMER_AC_INSERT_ROW = 0x0291, - HID_USAGE_CONSUMER_AC_INSERT_COLUMN = 0x0292, - HID_USAGE_CONSUMER_AC_INSERT_FILE = 0x0293, - HID_USAGE_CONSUMER_AC_INSERT_PICTURE = 0x0294, - HID_USAGE_CONSUMER_AC_INSERT_OBJECT = 0x0295, - HID_USAGE_CONSUMER_AC_INSERT_SYMBOL = 0x0296, - HID_USAGE_CONSUMER_AC_SAVE_AND_CLOSE = 0x0297, - HID_USAGE_CONSUMER_AC_RENAME = 0x0298, - HID_USAGE_CONSUMER_AC_MERGE = 0x0299, - HID_USAGE_CONSUMER_AC_SPLIT = 0x029A, - HID_USAGE_CONSUMER_AC_DISRIBUTE_HORIZONTALLY = 0x029B, - HID_USAGE_CONSUMER_AC_DISTRIBUTE_VERTICALLY = 0x029C, - // 29D-FFFF Reserved + HID_USAGE_CONSUMER_AL_INSTANT_MESSAGING = 0x01BC, // Sel + HID_USAGE_CONSUMER_AL_OEM_FEATURES_TIPS_TUTORIAL = 0x01BD, // Sel + HID_USAGE_CONSUMER_AL_OEM_HELP = 0x01BE, // Sel + HID_USAGE_CONSUMER_AL_ONLINE_COMMUNITY = 0x01BF, // Sel + HID_USAGE_CONSUMER_AL_ENTERTAINMENT_CONTENT = 0x01C0, // Sel + HID_USAGE_CONSUMER_AL_ONLINE_SHOPPING_BROWSER = 0x01C1, // Sel + HID_USAGE_CONSUMER_AL_SMARTCARD_INFORMATION_HELP = 0x01C2, // Sel + HID_USAGE_CONSUMER_AL_MARKET_MONITOR_FINANCE = 0x01C3, // Sel + HID_USAGE_CONSUMER_AL_CUSTOMIZED_CORPORATE_NEWS = 0x01C4, // Sel + HID_USAGE_CONSUMER_AL_ONLINE_ACTIVITY_BROWSER = 0x01C5, // Sel + HID_USAGE_CONSUMER_AL_RESEARCH_SEARCH_BROWSER = 0x01C6, // Sel + HID_USAGE_CONSUMER_AL_AUDIO_PLAYER = 0x01C7, // Sel + HID_USAGE_CONSUMER_AL_MESSAGE_STATUS = 0x01C8, // Sel + HID_USAGE_CONSUMER_AL_CONTACT_SYNC = 0x01C9, // Sel + HID_USAGE_CONSUMER_AL_NAVIGATION = 0x01CA, // Sel + HID_USAGE_CONSUMER_AL_CONTEXT_AWARE_DESKTOP_ASSISTANT = 0x01CB, // Sel + // 1CC-1FF Reserved + HID_USAGE_CONSUMER_GENERIC_GUI_APPLICATION = 0x0200, // NAry + HID_USAGE_CONSUMER_AC_NEW = 0x0201, // Sel + HID_USAGE_CONSUMER_AC_OPEN = 0x0202, // Sel + HID_USAGE_CONSUMER_AC_CLOSE = 0x0203, // Sel + HID_USAGE_CONSUMER_AC_EXIT = 0x0204, // Sel + HID_USAGE_CONSUMER_AC_MAXIMIZE = 0x0205, // Sel + HID_USAGE_CONSUMER_AC_MINIMIZE = 0x0206, // Sel + HID_USAGE_CONSUMER_AC_SAVE = 0x0207, // Sel + HID_USAGE_CONSUMER_AC_PRINT = 0x0208, // Sel + HID_USAGE_CONSUMER_AC_PROPERTIES = 0x0209, // Sel + // 20A-219 Reserved + HID_USAGE_CONSUMER_AC_UNDO = 0x021A, // Sel + HID_USAGE_CONSUMER_AC_COPY = 0x021B, // Sel + HID_USAGE_CONSUMER_AC_CUT = 0x021C, // Sel + HID_USAGE_CONSUMER_AC_PASTE = 0x021D, // Sel + HID_USAGE_CONSUMER_AC_SELECT_ALL = 0x021E, // Sel + HID_USAGE_CONSUMER_AC_FIND = 0x021F, // Sel + HID_USAGE_CONSUMER_AC_FIND_AND_REPLACE = 0x0220, // Sel + HID_USAGE_CONSUMER_AC_SEARCH = 0x0221, // Sel + HID_USAGE_CONSUMER_AC_GO_TO = 0x0222, // Sel + HID_USAGE_CONSUMER_AC_HOME = 0x0223, // Sel + HID_USAGE_CONSUMER_AC_BACK = 0x0224, // Sel + HID_USAGE_CONSUMER_AC_FORWARD = 0x0225, // Sel + HID_USAGE_CONSUMER_AC_STOP = 0x0226, // Sel + HID_USAGE_CONSUMER_AC_REFRESH = 0x0227, // Sel + HID_USAGE_CONSUMER_AC_PREVIOUS_LINK = 0x0228, // Sel + HID_USAGE_CONSUMER_AC_NEXT_LINK = 0x0229, // Sel + HID_USAGE_CONSUMER_AC_BOOKMARKS = 0x022A, // Sel + HID_USAGE_CONSUMER_AC_HISTORY = 0x022B, // Sel + HID_USAGE_CONSUMER_AC_SUBSCRIPTIONS = 0x022C, // Sel + HID_USAGE_CONSUMER_AC_ZOOM_IN = 0x022D, // Sel + HID_USAGE_CONSUMER_AC_ZOOM_OUT = 0x022E, // Sel + HID_USAGE_CONSUMER_AC_ZOOM = 0x022F, // LC + HID_USAGE_CONSUMER_AC_FULL_SCREEN_VIEW = 0x0230, // Sel + HID_USAGE_CONSUMER_AC_NORMAL_VIEW = 0x0231, // Sel + HID_USAGE_CONSUMER_AC_VIEW_TOGGLE = 0x0232, // Sel + HID_USAGE_CONSUMER_AC_SCROLL_UP = 0x0233, // Sel + HID_USAGE_CONSUMER_AC_SCROLL_DOWN = 0x0234, // Sel + HID_USAGE_CONSUMER_AC_SCROLL = 0x0235, // LC + HID_USAGE_CONSUMER_AC_PAN_LEFT = 0x0236, // Sel + HID_USAGE_CONSUMER_AC_PAN_RIGHT = 0x0237, // Sel + HID_USAGE_CONSUMER_AC_PAN = 0x0238, // LC + HID_USAGE_CONSUMER_AC_NEW_WINDOW = 0x0239, // Sel + HID_USAGE_CONSUMER_AC_TILE_HORIZONTALLY = 0x023A, // Sel + HID_USAGE_CONSUMER_AC_TILE_VERTICALLY = 0x023B, // Sel + HID_USAGE_CONSUMER_AC_FORMAT = 0x023C, // Sel + HID_USAGE_CONSUMER_AC_EDIT = 0x023D, // Sel + HID_USAGE_CONSUMER_AC_BOLD = 0x023E, // Sel + HID_USAGE_CONSUMER_AC_ITALICS = 0x023F, // Sel + HID_USAGE_CONSUMER_AC_UNDERLINE = 0x0240, // Sel + HID_USAGE_CONSUMER_AC_STRIKETHROUGH = 0x0241, // Sel + HID_USAGE_CONSUMER_AC_SUBSCRIPT = 0x0242, // Sel + HID_USAGE_CONSUMER_AC_SUPERSCRIPT = 0x0243, // Sel + HID_USAGE_CONSUMER_AC_ALL_CAPS = 0x0244, // Sel + HID_USAGE_CONSUMER_AC_ROTATE = 0x0245, // Sel + HID_USAGE_CONSUMER_AC_RESIZE = 0x0246, // Sel + HID_USAGE_CONSUMER_AC_FLIP_HORIZONTAL = 0x0247, // Sel + HID_USAGE_CONSUMER_AC_FLIP_VERTICAL = 0x0248, // Sel + HID_USAGE_CONSUMER_AC_MIRROR_HORIZONTAL = 0x0249, // Sel + HID_USAGE_CONSUMER_AC_MIRROR_VERTICAL = 0x024A, // Sel + HID_USAGE_CONSUMER_AC_FONT_SELECT = 0x024B, // Sel + HID_USAGE_CONSUMER_AC_FONT_COLOR = 0x024C, // Sel + HID_USAGE_CONSUMER_AC_FONT_SIZE = 0x024D, // Sel + HID_USAGE_CONSUMER_AC_JUSTIFY_LEFT = 0x024E, // Sel + HID_USAGE_CONSUMER_AC_JUSTIFY_CENTER_H = 0x024F, // Sel + HID_USAGE_CONSUMER_AC_JUSTIFY_RIGHT = 0x0250, // Sel + HID_USAGE_CONSUMER_AC_JUSTIFY_BLOCK_H = 0x0251, // Sel + HID_USAGE_CONSUMER_AC_JUSTIFY_TOP = 0x0252, // Sel + HID_USAGE_CONSUMER_AC_JUSTIFY_CENTER_V = 0x0253, // Sel + HID_USAGE_CONSUMER_AC_JUSTIFY_BOTTOM = 0x0254, // Sel + HID_USAGE_CONSUMER_AC_JUSTIFY_BLOCK_V = 0x0255, // Sel + HID_USAGE_CONSUMER_AC_INDENT_DECREASE = 0x0256, // Sel + HID_USAGE_CONSUMER_AC_INDENT_INCREASE = 0x0257, // Sel + HID_USAGE_CONSUMER_AC_NUMBERED_LIST = 0x0258, // Sel + HID_USAGE_CONSUMER_AC_RESTART_NUMBERING = 0x0259, // Sel + HID_USAGE_CONSUMER_AC_BULLETED_LIST = 0x025A, // Sel + HID_USAGE_CONSUMER_AC_PROMOTE = 0x025B, // Sel + HID_USAGE_CONSUMER_AC_DEMOTE = 0x025C, // Sel + HID_USAGE_CONSUMER_AC_YES = 0x025D, // Sel + HID_USAGE_CONSUMER_AC_NO = 0x025E, // Sel + HID_USAGE_CONSUMER_AC_CANCEL = 0x025F, // Sel + HID_USAGE_CONSUMER_AC_CATALOG = 0x0260, // Sel + HID_USAGE_CONSUMER_AC_BUY_CHECKOUT = 0x0261, // Sel + HID_USAGE_CONSUMER_AC_ADD_TO_CART = 0x0262, // Sel + HID_USAGE_CONSUMER_AC_EXPAND = 0x0263, // Sel + HID_USAGE_CONSUMER_AC_EXPAND_ALL = 0x0264, // Sel + HID_USAGE_CONSUMER_AC_COLLAPSE = 0x0265, // Sel + HID_USAGE_CONSUMER_AC_COLLAPSE_ALL = 0x0266, // Sel + HID_USAGE_CONSUMER_AC_PRINT_PREVIEW = 0x0267, // Sel + HID_USAGE_CONSUMER_AC_PASTE_SPECIAL = 0x0268, // Sel + HID_USAGE_CONSUMER_AC_INSERT_MODE = 0x0269, // Sel + HID_USAGE_CONSUMER_AC_DELETE = 0x026A, // Sel + HID_USAGE_CONSUMER_AC_LOCK = 0x026B, // Sel + HID_USAGE_CONSUMER_AC_UNLOCK = 0x026C, // Sel + HID_USAGE_CONSUMER_AC_PROTECT = 0x026D, // Sel + HID_USAGE_CONSUMER_AC_UNPROTECT = 0x026E, // Sel + HID_USAGE_CONSUMER_AC_ATTACH_COMMENT = 0x026F, // Sel + HID_USAGE_CONSUMER_AC_DELETE_COMMENT = 0x0270, // Sel + HID_USAGE_CONSUMER_AC_VIEW_COMMENT = 0x0271, // Sel + HID_USAGE_CONSUMER_AC_SELECT_WORD = 0x0272, // Sel + HID_USAGE_CONSUMER_AC_SELECT_SENTENCE = 0x0273, // Sel + HID_USAGE_CONSUMER_AC_SELECT_PARAGRAPH = 0x0274, // Sel + HID_USAGE_CONSUMER_AC_SELECT_COLUMN = 0x0275, // Sel + HID_USAGE_CONSUMER_AC_SELECT_ROW = 0x0276, // Sel + HID_USAGE_CONSUMER_AC_SELECT_TABLE = 0x0277, // Sel + HID_USAGE_CONSUMER_AC_SELECT_OBJECT = 0x0278, // Sel + HID_USAGE_CONSUMER_AC_REDO_REPEAT = 0x0279, // Sel + HID_USAGE_CONSUMER_AC_SORT = 0x027A, // Sel + HID_USAGE_CONSUMER_AC_SORT_ASCENDING = 0x027B, // Sel + HID_USAGE_CONSUMER_AC_SORT_DESCENDING = 0x027C, // Sel + HID_USAGE_CONSUMER_AC_FILTER = 0x027D, // Sel + HID_USAGE_CONSUMER_AC_SET_CLOCK = 0x027E, // Sel + HID_USAGE_CONSUMER_AC_VIEW_CLOCK = 0x027F, // Sel + HID_USAGE_CONSUMER_AC_SELECT_TIME_ZONE = 0x0280, // Sel + HID_USAGE_CONSUMER_AC_EDIT_TIME_ZONES = 0x0281, // Sel + HID_USAGE_CONSUMER_AC_SET_ALARM = 0x0282, // Sel + HID_USAGE_CONSUMER_AC_CLEAR_ALARM = 0x0283, // Sel + HID_USAGE_CONSUMER_AC_SNOOZE_ALARM = 0x0284, // Sel + HID_USAGE_CONSUMER_AC_RESET_ALARM = 0x0285, // Sel + HID_USAGE_CONSUMER_AC_SYNCHRONIZE = 0x0286, // Sel + HID_USAGE_CONSUMER_AC_SEND_RECEIVE = 0x0287, // Sel + HID_USAGE_CONSUMER_AC_SEND_TO = 0x0288, // Sel + HID_USAGE_CONSUMER_AC_REPLY = 0x0289, // Sel + HID_USAGE_CONSUMER_AC_REPLY_ALL = 0x028A, // Sel + HID_USAGE_CONSUMER_AC_FORWARD_MSG = 0x028B, // Sel + HID_USAGE_CONSUMER_AC_SEND = 0x028C, // Sel + HID_USAGE_CONSUMER_AC_ATTACH_FILE = 0x028D, // Sel + HID_USAGE_CONSUMER_AC_UPLOAD = 0x028E, // Sel + HID_USAGE_CONSUMER_AC_DOWNLOAD_SAVE_TARGET_AS = 0x028F, // Sel + HID_USAGE_CONSUMER_AC_SET_BORDERS = 0x0290, // Sel + HID_USAGE_CONSUMER_AC_INSERT_ROW = 0x0291, // Sel + HID_USAGE_CONSUMER_AC_INSERT_COLUMN = 0x0292, // Sel + HID_USAGE_CONSUMER_AC_INSERT_FILE = 0x0293, // Sel + HID_USAGE_CONSUMER_AC_INSERT_PICTURE = 0x0294, // Sel + HID_USAGE_CONSUMER_AC_INSERT_OBJECT = 0x0295, // Sel + HID_USAGE_CONSUMER_AC_INSERT_SYMBOL = 0x0296, // Sel + HID_USAGE_CONSUMER_AC_SAVE_AND_CLOSE = 0x0297, // Sel + HID_USAGE_CONSUMER_AC_RENAME = 0x0298, // Sel + HID_USAGE_CONSUMER_AC_MERGE = 0x0299, // Sel + HID_USAGE_CONSUMER_AC_SPLIT = 0x029A, // Sel + HID_USAGE_CONSUMER_AC_DISRIBUTE_HORIZONTALLY = 0x029B, // Sel + HID_USAGE_CONSUMER_AC_DISTRIBUTE_VERTICALLY = 0x029C, // Sel + HID_USAGE_CONSUMER_AC_NEXT_KEYBOARD_LAYOUT_SELECT = 0x029D, // Sel + HID_USAGE_CONSUMER_AC_NAVIGATION_GUIDANCE = 0x029E, // Sel + HID_USAGE_CONSUMER_AC_DESKTOP_SHOW_ALL_WINDOWS = 0x029F, // Sel + HID_USAGE_CONSUMER_AC_SOFT_KEY_LEFT = 0x02A0, // Sel + HID_USAGE_CONSUMER_AC_SOFT_KEY_RIGHT = 0x02A1, // Sel + HID_USAGE_CONSUMER_AC_DESKTOP_SHOW_ALL_APPLICATIONS = 0x02A2, // Sel + // 2A3-2AF Reserved + HID_USAGE_CONSUMER_AC_IDLE_KEEP_ALIVE = 0x02B0, // Sel + // 2B1-2BF Reserved + HID_USAGE_CONSUMER_EXTENDED_KEYBOARD_ATTRIBUTES_COLLECTION = 0x02C0, // CL + HID_USAGE_CONSUMER_KEYBOARD_FORM_FACTOR = 0x02C1, // SV + HID_USAGE_CONSUMER_KEYBOARD_KEY_TYPE = 0x02C2, // SV + HID_USAGE_CONSUMER_KEYBOARD_PHYSICAL_LAYOUT = 0x02C3, // SV + HID_USAGE_CONSUMER_VENDOR_SPECIFIC_KEYBOARD_PHYSICAL_LAYOUT = 0x02C4, // SV + HID_USAGE_CONSUMER_KEYBOARD_IETF_LANGUAGE_TAG_INDEX = 0x02C5, // SV + HID_USAGE_CONSUMER_IMPLEMENTED_KEYBOARD_INPUT_ASSIST_CONTROLS = 0x02C6, // SV + HID_USAGE_CONSUMER_KEYBOARD_INPUT_ASSIST_PREVIOUS = 0x02C7, // Sel + HID_USAGE_CONSUMER_KEYBOARD_INPUT_ASSIST_NEXT = 0x02C8, // Sel + HID_USAGE_CONSUMER_KEYBOARD_INPUT_ASSIST_PREVIOUS_GROUP = 0x02C9, // Sel + HID_USAGE_CONSUMER_KEYBOARD_INPUT_ASSIST_NEXT_GROUP = 0x02CA, // Sel + HID_USAGE_CONSUMER_KEYBOARD_INPUT_ASSIST_ACCEPT = 0x02CB, // Sel + HID_USAGE_CONSUMER_KEYBOARD_INPUT_ASSIST_CANCEL = 0x02CC, // Sel + // 2CD-2CF Reserved + HID_USAGE_CONSUMER_PRIVACY_SCREEN_TOGGLE = 0x02D0, // OOC + HID_USAGE_CONSUMER_PRIVACY_SCREEN_LEVEL_DECREMENT = 0x02D1, // RTC + HID_USAGE_CONSUMER_PRIVACY_SCREEN_LEVEL_INCREMENT = 0x02D2, // RTC + HID_USAGE_CONSUMER_PRIVACY_SCREEN_LEVEL_MINIMUM = 0x02D3, // OSC + HID_USAGE_CONSUMER_PRIVACY_SCREEN_LEVEL_MAXIMUM = 0x02D4, // OSC + // 2D5-4FF Reserved + HID_USAGE_CONSUMER_CONTACT_EDITED = 0x0500, // OOC + HID_USAGE_CONSUMER_CONTACT_ADDED = 0x0501, // OOC + HID_USAGE_CONSUMER_CONTACT_RECORD_ACTIVE = 0x0502, // OOC + HID_USAGE_CONSUMER_CONTACT_INDEX = 0x0503, // DV + HID_USAGE_CONSUMER_CONTACT_NICKNAME = 0x0504, // DV + HID_USAGE_CONSUMER_CONTACT_FIRST_NAME = 0x0505, // DV + HID_USAGE_CONSUMER_CONTACT_LAST_NAME = 0x0506, // DV + HID_USAGE_CONSUMER_CONTACT_FULL_NAME = 0x0507, // DV + HID_USAGE_CONSUMER_CONTACT_PHONE_NUMBER_PERSONAL = 0x0508, // DV + HID_USAGE_CONSUMER_CONTACT_PHONE_NUMBER_BUSINESS = 0x0509, // DV + HID_USAGE_CONSUMER_CONTACT_PHONE_NUMBER_MOBILE = 0x050A, // DV + HID_USAGE_CONSUMER_CONTACT_PHONE_NUMBER_PAGER = 0x050B, // DV + HID_USAGE_CONSUMER_CONTACT_PHONE_NUMBER_FAX = 0x050C, // DV + HID_USAGE_CONSUMER_CONTACT_PHONE_NUMBER_OTHER = 0x050D, // DV + HID_USAGE_CONSUMER_CONTACT_EMAIL_PERSONAL = 0x050E, // DV + HID_USAGE_CONSUMER_CONTACT_EMAIL_BUSINESS = 0x050F, // DV + HID_USAGE_CONSUMER_CONTACT_EMAIL_OTHER = 0x0510, // DV + HID_USAGE_CONSUMER_CONTACT_EMAIL_MAIN = 0x0511, // DV + HID_USAGE_CONSUMER_CONTACT_SPEED_DIAL_NUMBER = 0x0512, // DV + HID_USAGE_CONSUMER_CONTACT_STATUS_FLAG = 0x0513, // DV + HID_USAGE_CONSUMER_CONTACT_MISC = 0x0514, // DV + HID_USAGE_CONSUMER_KEYBOARD_BRIGHTNESS_NEXT = 0x0515, // OSC + HID_USAGE_CONSUMER_KEYBOARD_BRIGHTNESS_PREVIOUS = 0x0516, // OSC + HID_USAGE_CONSUMER_KEYBOARD_BACKLIGHT_LEVEL_SUGGESTION = 0x0517, // SV + // 518-FFFF Reserved + // For Backwards compatibility to prevent current builds from breaking + HID_USAGE_CONSUMER_BRIGHTNESS_INCREMENT = HID_USAGE_CONSUMER_DISPLAY_BRIGHTNESS_INCREMENT, // RTC + HID_USAGE_CONSUMER_BRIGHTNESS_DECREMENT = HID_USAGE_CONSUMER_DISPLAY_BRIGHTNESS_DECREMENT, // RTC }; /// HID Usage Table: Digitizer Page (0x0D) @@ -1371,6 +1935,7 @@ enum { HID_USAGE_DIGITIZER_SURFACE_SWITCH = 0x57, // DF HID_USAGE_DIGITIZER_BUTTON_SWITCH = 0x58, // DF HID_USAGE_DIGITIZER_PAD_TYPE = 0x59, // SF + HID_USAGE_DIGITIZER_SECONDARY_BARREL_SWITCH = 0x5A, // MC HID_USAGE_DIGITIZER_TRANSDUCER_SERIAL_NUMBER = 0x5B, // SV HID_USAGE_DIGITIZER_PREFERRED_COLOR = 0x5C, // DV HID_USAGE_DIGITIZER_PREFERRED_COLOR_LOCKED = 0x5D, // MC @@ -1431,151 +1996,500 @@ enum { // Reserved (0xB1 - 0xFFFF) }; +/// HID Usage Table: Haptics Page (0x0E) +enum{ + HID_USAGE_HAPTICS_SIMPLE_HAPTIC_CONTROLLER = 0x0001, // CA/CL + // Reserved (0x0002 - 0x000F) + HID_USAGE_HAPTICS_WAVEFORM_LIST = 0x0010, // NAry + HID_USAGE_HAPTICS_DURATION_LIST = 0x0011, // NAry + // Reserved (0x0012 - 0x001F) + HID_USAGE_HAPTICS_AUTO_TRIGGER = 0x0020, // DV + HID_USAGE_HAPTICS_MANUAL_TRIGGER = 0x0021, // DV + HID_USAGE_HAPTICS_AUTO_TRIGGER_ASSOCIATED_CONTROL = 0x0022, // SV + HID_USAGE_HAPTICS_INTENSITY = 0x0023, // DV + HID_USAGE_HAPTICS_REPEAT_COUNT = 0x0024, // DV + HID_USAGE_HAPTICS_RETRIGGER_PERIOD = 0x0025, // DV + HID_USAGE_HAPTICS_WAVEFORM_VENDOR_PAGE = 0x0026, // SV + HID_USAGE_HAPTICS_WAVEFORM_VENDOR_ID = 0x0027, // SV + HID_USAGE_HAPTICS_WAVEFORM_CUTOFF_TIME = 0x0028, // SV + // Reserved (0x0029 - 0x1000) + HID_USAGE_HAPTICS_WAVEFORM_NONE = 0x1001, // SV + HID_USAGE_HAPTICS_WAVEFORM_STOP = 0x1002, // SV + HID_USAGE_HAPTICS_WAVEFORM_CLICK = 0x1003, // SV + HID_USAGE_HAPTICS_WAVEFORM_BUZZ_CONTINUOUS = 0x1004, // SV + HID_USAGE_HAPTICS_WAVEFORM_RUMBLE_CONTINUOUS = 0x1005, // SV + HID_USAGE_HAPTICS_WAVEFORM_PRESS = 0x1006, // SV + HID_USAGE_HAPTICS_WAVEFORM_RELEASE = 0x1007, // SV + HID_USAGE_HAPTICS_WAVEFORM_HOVER = 0x1008, // SV + HID_USAGE_HAPTICS_WAVEFORM_SUCCESS = 0x1009, // SV + HID_USAGE_HAPTICS_WAVEFORM_ERROR = 0x100A, // SV + HID_USAGE_HAPTICS_WAVEFORM_INK_CONTINUOUS = 0x100B, // SV + HID_USAGE_HAPTICS_WAVEFORM_PENCIL_CONTINUOUS = 0x100C, // SV + HID_USAGE_HAPTICS_WAVEFORM_MARKER_CONTINUOUS = 0x100D, // SV + HID_USAGE_HAPTICS_WAVEFORM_CHISEL_MARKER_CONTINUOUS = 0x100E, // SV + HID_USAGE_HAPTICS_WAVEFORM_BRUSH_CONTINUOUS = 0x100F, // SV + HID_USAGE_HAPTICS_WAVEFORM_ERASER_CONTINUOUS = 0x1010, // SV + HID_USAGE_HAPTICS_WAVEFORM_SPARKLE_CONTINUOUS = 0x1011, // SV + // Reserved (0x1012 - 0xFFFF) +}; + /// HID Usage Table: Physical Input Device Page (0x0F) enum { HID_USAGE_PID_UNDEFINED = 0x00, - HID_USAGE_PID_PHYSICAL_INPUT_DEVICE = 0x01, - HID_USAGE_PID_NORMAL = 0x20, - HID_USAGE_PID_SET_EFFECT_REPORT = 0x21, - HID_USAGE_PID_EFFECT_PARAMETER_BLOCK_INDEX = 0x22, - HID_USAGE_PID_PARAMETER_BLOCK_OFFSET = 0x23, - HID_USAGE_PID_ROM_FLAG = 0x24, - HID_USAGE_PID_EFFECT_TYPE = 0x25, - HID_USAGE_PID_ET_CONSTANTFORCE = 0x26, - HID_USAGE_PID_ET_RAMP = 0x27, - HID_USAGE_PID_ET_CUSTOMFORCE = 0x28, - HID_USAGE_PID_ET_SQUARE = 0x30, - HID_USAGE_PID_ET_SINE = 0x31, - HID_USAGE_PID_ET_TRIANGLE = 0x32, - HID_USAGE_PID_ET_SAWTOOTH_UP = 0x33, - HID_USAGE_PID_ET_SAWTOOTH_DOWN = 0x34, - HID_USAGE_PID_ET_SPRING = 0x40, - HID_USAGE_PID_ET_DAMPER = 0x41, - HID_USAGE_PID_ET_INERTIA = 0x42, - HID_USAGE_PID_ET_FRICTION = 0x43, - HID_USAGE_PID_DURATION = 0x50, - HID_USAGE_PID_SAMPLE_PERIOD = 0x51, - HID_USAGE_PID_GAIN = 0x52, - HID_USAGE_PID_TRIGGER_BUTTON = 0x53, - HID_USAGE_PID_TRIGGER_REPEAT_INTERVAL = 0x54, - HID_USAGE_PID_AXES_ENABLE = 0x55, - HID_USAGE_PID_DIRECTION_ENABLE = 0x56, - HID_USAGE_PID_DIRECTION = 0x57, - HID_USAGE_PID_TYPE_SPECIFIC_BLOCK_OFFSET = 0x58, - HID_USAGE_PID_BLOCK_TYPE = 0x59, - HID_USAGE_PID_SET_ENVELOPE_REPORT = 0x5a, - HID_USAGE_PID_ATTACK_LEVEL = 0x5b, - HID_USAGE_PID_ATTACK_TIME = 0x5c, - HID_USAGE_PID_FADE_LEVEL = 0x5d, - HID_USAGE_PID_FADE_TIME = 0x5e, - HID_USAGE_PID_SET_CONDITION_REPORT = 0x5f, - HID_USAGE_PID_CENTERPOINT_OFFSET = 0x60, - HID_USAGE_PID_POSITIVE_COEFFICIENT = 0x61, - HID_USAGE_PID_NEGATIVE_COEFFICIENT = 0x62, - HID_USAGE_PID_POSITIVE_SATURATION = 0x63, - HID_USAGE_PID_NEGATIVE_SATURATION = 0x64, - HID_USAGE_PID_DEAD_BAND = 0x65, - HID_USAGE_PID_DOWNLOAD_FORCE_SAMPLE = 0x66, - HID_USAGE_PID_ISOCH_CUSTOMFORCE_ENABLE = 0x67, - HID_USAGE_PID_CUSTOMFORCE_DATA_REPORT = 0x68, - HID_USAGE_PID_CUSTOMFORCE_DATA = 0x69, - HID_USAGE_PID_CUSTOMFORCE_VENDOR_DEFINED_DATA = 0x6a, - HID_USAGE_PID_SET_CUSTOMFORCE_REPORT = 0x6b, - HID_USAGE_PID_CUSTOMFORCE_DATA_OFFSET = 0x6c, - HID_USAGE_PID_SAMPLE_COUNT = 0x6d, - HID_USAGE_PID_SET_PERIODIC_REPORT = 0x6e, - HID_USAGE_PID_OFFSET = 0x6f, - HID_USAGE_PID_MAGNITUDE = 0x70, - HID_USAGE_PID_PHASE = 0x71, - HID_USAGE_PID_PERIOD = 0x72, - HID_USAGE_PID_SET_CONSTANTFORCE_REPORT = 0x73, - HID_USAGE_PID_SET_RAMPFORCE_REPORT = 0x74, - HID_USAGE_PID_RAMP_START = 0x75, - HID_USAGE_PID_RAMP_END = 0x76, - HID_USAGE_PID_EFFECT_OPERATION_REPORT = 0x77, - HID_USAGE_PID_EFFECT_OPERATION = 0x78, - HID_USAGE_PID_OP_EFFECT_START = 0x79, - HID_USAGE_PID_OP_EFFECT_START_SOLO = 0x7a, - HID_USAGE_PID_OP_EFFECT_STOP = 0x7b, - HID_USAGE_PID_LOOP_COUNT = 0x7c, - HID_USAGE_PID_DEVICE_GAIN_REPORT = 0x7d, - HID_USAGE_PID_DEVICE_GAIN = 0x7e, - HID_USAGE_PID_PARAMETER_BLOCK_POOLS_REPORT = 0x7f, - HID_USAGE_PID_RAM_POOL_SIZE = 0x80, - HID_USAGE_PID_ROM_POOL_SIZE = 0x81, - HID_USAGE_PID_ROM_EFFECT_BLOCK_COUNT = 0x82, - HID_USAGE_PID_SIMULTANEOUS_EFFECTS_MAX = 0x83, - HID_USAGE_PID_POOL_ALIGNMENT = 0x84, - HID_USAGE_PID_PARAMETER_BLOCK_MOVE_REPORT = 0x85, - HID_USAGE_PID_MOVE_SOURCE = 0x86, - HID_USAGE_PID_MOVE_DESTINATION = 0x87, - HID_USAGE_PID_MOVE_LENGTH = 0x88, - HID_USAGE_PID_EFFECT_PARAMETER_BLOCK_LOAD_REPORT = 0x89, - HID_USAGE_PID_EFFECT_PARAMETER_BLOCK_LOAD_STATUS = 0x8b, - HID_USAGE_PID_BLOCK_LOAD_SUCCESS = 0x8c, - HID_USAGE_PID_BLOCK_LOAD_FULL = 0x8d, - HID_USAGE_PID_BLOCK_LOAD_ERROR = 0x8e, - HID_USAGE_PID_BLOCK_HANDLE = 0x8f, - HID_USAGE_PID_EFFECT_PARAMETER_BLOCK_FREE_REPORT = 0x90, - HID_USAGE_PID_TYPE_SPECIFIC_BLOCK_HANDLE = 0x91, - HID_USAGE_PID_PID_STATE_REPORT = 0x92, - HID_USAGE_PID_EFFECT_PLAYING = 0x94, - HID_USAGE_PID_PID_DEVICE_CONTROL_REPORT = 0x95, - HID_USAGE_PID_PID_DEVICE_CONTROL = 0x96, - HID_USAGE_PID_DC_ENABLE_ACTUATORS = 0x97, - HID_USAGE_PID_DC_DISABLE_ACTUATORS = 0x98, - HID_USAGE_PID_DC_STOP_ALL_EFFECTS = 0x99, - HID_USAGE_PID_DC_RESET = 0x9a, - HID_USAGE_PID_DC_PAUSE = 0x9b, - HID_USAGE_PID_DC_CONTINUE = 0x9c, - HID_USAGE_PID_DEVICE_PAUSED = 0x9f, - HID_USAGE_PID_ACTUATORS_ENABLED = 0xa0, - HID_USAGE_PID_SAFETY_SWITCH = 0xa4, - HID_USAGE_PID_ACTUATOR_OVERRIDE_SWITCH = 0xa5, - HID_USAGE_PID_ACTUATOR_POWER = 0xa6, - HID_USAGE_PID_START_DELAY = 0xa7, - HID_USAGE_PID_PARAMETER_BLOCK_SIZE = 0xa8, - HID_USAGE_PID_DEVICEMANAGED_POOL = 0xa9, - HID_USAGE_PID_SHARED_PARAMETER_BLOCKS = 0xaa, - HID_USAGE_PID_CREATE_NEW_EFFECT_PARAMETER_BLOCK_REPORT = 0xab, - HID_USAGE_PID_RAM_POOL_AVAILABLE = 0xac, + HID_USAGE_PID_PHYSICAL_INPUT_DEVICE = 0x01, // CA + // Reserved (0x02 - 0x1F) + HID_USAGE_PID_NORMAL = 0x20, // DV + HID_USAGE_PID_SET_EFFECT_REPORT = 0x21, // CL + HID_USAGE_PID_EFFECT_PARAMETER_BLOCK_INDEX = 0x22, // DV + HID_USAGE_PID_PARAMETER_BLOCK_OFFSET = 0x23, // DV + HID_USAGE_PID_ROM_FLAG = 0x24, // DF + HID_USAGE_PID_EFFECT_TYPE = 0x25, // NAry + HID_USAGE_PID_ET_CONSTANTFORCE = 0x26, // Sel + HID_USAGE_PID_ET_RAMP = 0x27, // Sel + HID_USAGE_PID_ET_CUSTOMFORCE = 0x28, // Sel + // Reserved (0x29 - 0x2F) + HID_USAGE_PID_ET_SQUARE = 0x30, // Sel + HID_USAGE_PID_ET_SINE = 0x31, // Sel + HID_USAGE_PID_ET_TRIANGLE = 0x32, // Sel + HID_USAGE_PID_ET_SAWTOOTH_UP = 0x33, // Sel + HID_USAGE_PID_ET_SAWTOOTH_DOWN = 0x34, // Sel + // Reserved (0x35 - 0x3F) + HID_USAGE_PID_ET_SPRING = 0x40, // Sel + HID_USAGE_PID_ET_DAMPER = 0x41, // Sel + HID_USAGE_PID_ET_INERTIA = 0x42, // Sel + HID_USAGE_PID_ET_FRICTION = 0x43, // Sel + // Reserved (0x44 - 0x4F) + HID_USAGE_PID_DURATION = 0x50, // DV + HID_USAGE_PID_SAMPLE_PERIOD = 0x51, // DV + HID_USAGE_PID_GAIN = 0x52, // DV + HID_USAGE_PID_TRIGGER_BUTTON = 0x53, // DV + HID_USAGE_PID_TRIGGER_REPEAT_INTERVAL = 0x54, // DV + HID_USAGE_PID_AXES_ENABLE = 0x55, // US + HID_USAGE_PID_DIRECTION_ENABLE = 0x56, // DF + HID_USAGE_PID_DIRECTION = 0x57, // CL + HID_USAGE_PID_TYPE_SPECIFIC_BLOCK_OFFSET = 0x58, // CL + HID_USAGE_PID_BLOCK_TYPE = 0x59, // NAry + HID_USAGE_PID_SET_ENVELOPE_REPORT = 0x5A, // CL/SV + HID_USAGE_PID_ATTACK_LEVEL = 0x5B, // DV + HID_USAGE_PID_ATTACK_TIME = 0x5C, // DV + HID_USAGE_PID_FADE_LEVEL = 0x5D, // DV + HID_USAGE_PID_FADE_TIME = 0x5E, // DV + HID_USAGE_PID_SET_CONDITION_REPORT = 0x5F, // CL/SV + HID_USAGE_PID_CENTERPOINT_OFFSET = 0x60, // DV + HID_USAGE_PID_POSITIVE_COEFFICIENT = 0x61, // DV + HID_USAGE_PID_NEGATIVE_COEFFICIENT = 0x62, // DV + HID_USAGE_PID_POSITIVE_SATURATION = 0x63, // DV + HID_USAGE_PID_NEGATIVE_SATURATION = 0x64, // DV + HID_USAGE_PID_DEAD_BAND = 0x65, // DV + HID_USAGE_PID_DOWNLOAD_FORCE_SAMPLE = 0x66, // CL + HID_USAGE_PID_ISOCH_CUSTOMFORCE_ENABLE = 0x67, // DF + HID_USAGE_PID_CUSTOMFORCE_DATA_REPORT = 0x68, // CL + HID_USAGE_PID_CUSTOMFORCE_DATA = 0x69, // DV + HID_USAGE_PID_CUSTOMFORCE_VENDOR_DEFINED_DATA = 0x6A, // DV + HID_USAGE_PID_SET_CUSTOMFORCE_REPORT = 0x6B, // CL/SV + HID_USAGE_PID_CUSTOMFORCE_DATA_OFFSET = 0x6C, // DV + HID_USAGE_PID_SAMPLE_COUNT = 0x6D, // DV + HID_USAGE_PID_SET_PERIODIC_REPORT = 0x6E, // CL/SV + HID_USAGE_PID_OFFSET = 0x6F, // DV + HID_USAGE_PID_MAGNITUDE = 0x70, // DV + HID_USAGE_PID_PHASE = 0x71, // DV + HID_USAGE_PID_PERIOD = 0x72, // DV + HID_USAGE_PID_SET_CONSTANTFORCE_REPORT = 0x73, // CL/SV + HID_USAGE_PID_SET_RAMPFORCE_REPORT = 0x74, // CL/SV + HID_USAGE_PID_RAMP_START = 0x75, // DV + HID_USAGE_PID_RAMP_END = 0x76, // DV + HID_USAGE_PID_EFFECT_OPERATION_REPORT = 0x77, // CL + HID_USAGE_PID_EFFECT_OPERATION = 0x78, // NAry + HID_USAGE_PID_OP_EFFECT_START = 0x79, // Sel + HID_USAGE_PID_OP_EFFECT_START_SOLO = 0x7A, // Sel + HID_USAGE_PID_OP_EFFECT_STOP = 0x7B, // Sel + HID_USAGE_PID_LOOP_COUNT = 0x7C, // DV + HID_USAGE_PID_DEVICE_GAIN_REPORT = 0x7D, // CL + HID_USAGE_PID_DEVICE_GAIN = 0x7E, // DV + HID_USAGE_PID_PARAMETER_BLOCK_POOLS_REPORT = 0x7F, // CL + HID_USAGE_PID_RAM_POOL_SIZE = 0x80, // DV + HID_USAGE_PID_ROM_POOL_SIZE = 0x81, // SV + HID_USAGE_PID_ROM_EFFECT_BLOCK_COUNT = 0x82, // SV + HID_USAGE_PID_SIMULTANEOUS_EFFECTS_MAX = 0x83, // SV + HID_USAGE_PID_POOL_ALIGNMENT = 0x84, // SV + HID_USAGE_PID_PARAMETER_BLOCK_MOVE_REPORT = 0x85, // CL + HID_USAGE_PID_MOVE_SOURCE = 0x86, // DV + HID_USAGE_PID_MOVE_DESTINATION = 0x87, // DV + HID_USAGE_PID_MOVE_LENGTH = 0x88, // DV + HID_USAGE_PID_EFFECT_PARAMETER_BLOCK_LOAD_REPORT = 0x89, // CL + // Reserved (0x8A) + HID_USAGE_PID_EFFECT_PARAMETER_BLOCK_LOAD_STATUS = 0x8B, // NAry + HID_USAGE_PID_BLOCK_LOAD_SUCCESS = 0x8C, // Sel + HID_USAGE_PID_BLOCK_LOAD_FULL = 0x8D, // Sel + HID_USAGE_PID_BLOCK_LOAD_ERROR = 0x8E, // Sel + HID_USAGE_PID_BLOCK_HANDLE = 0x8F, // DV + HID_USAGE_PID_EFFECT_PARAMETER_BLOCK_FREE_REPORT = 0x90, // CL + HID_USAGE_PID_TYPE_SPECIFIC_BLOCK_HANDLE = 0x91, // CL + HID_USAGE_PID_PID_STATE_REPORT = 0x92, // CL + // Reserved (0x93) + HID_USAGE_PID_EFFECT_PLAYING = 0x94, // DF + HID_USAGE_PID_PID_DEVICE_CONTROL_REPORT = 0x95, // CL + HID_USAGE_PID_PID_DEVICE_CONTROL = 0x96, // NAry + HID_USAGE_PID_DC_ENABLE_ACTUATORS = 0x97, // Sel + HID_USAGE_PID_DC_DISABLE_ACTUATORS = 0x98, // Sel + HID_USAGE_PID_DC_STOP_ALL_EFFECTS = 0x99, // Sel + HID_USAGE_PID_DC_RESET = 0x9A, // Sel + HID_USAGE_PID_DC_PAUSE = 0x9B, // Sel + HID_USAGE_PID_DC_CONTINUE = 0x9C, // Sel + // Reserved (0x9D - 0x9E) + HID_USAGE_PID_DEVICE_PAUSED = 0x9F, // DF + HID_USAGE_PID_ACTUATORS_ENABLED = 0xA0, // DF + // Reserved (0xA1 - 0xA3) + HID_USAGE_PID_SAFETY_SWITCH = 0xA4, // DF + HID_USAGE_PID_ACTUATOR_OVERRIDE_SWITCH = 0xA5, // DF + HID_USAGE_PID_ACTUATOR_POWER = 0xA6, // OOC + HID_USAGE_PID_START_DELAY = 0xA7, // DV + HID_USAGE_PID_PARAMETER_BLOCK_SIZE = 0xA8, // CL + HID_USAGE_PID_DEVICEMANAGED_POOL = 0xA9, // SF + HID_USAGE_PID_SHARED_PARAMETER_BLOCKS = 0xAA, // SF + HID_USAGE_PID_CREATE_NEW_EFFECT_PARAMETER_BLOCK_REPORT = 0xAB, // CL + HID_USAGE_PID_RAM_POOL_AVAILABLE = 0xAC, // DV + // Reserved (0xAD - 0xFFFF) +}; + +/// HID Usage Table: Unicode Page (0x10) +/// Intentionally skipped + +/// HID Usage Table: SoC Page (0x11) +enum { + HID_USAGE_SOC_SOC_CONTROL = 0x01, // CA + HID_USAGE_SOC_FIRMWARE_TRANSFER = 0x02, // CL + HID_USAGE_SOC_FIRMWARE_FILE_ID = 0x03, // DV + HID_USAGE_SOC_FILE_OFFSET_IN_BYTES = 0x04, // DV + HID_USAGE_SOC_FILE_TRANSFER_SIZE_MAX_IN_BYTES = 0x05, // DV + HID_USAGE_SOC_FILE_PAYLOAD = 0x06, // DV + HID_USAGE_SOC_FILE_PAYLOAD_SIZE_IN_BYTES = 0x07, // DV + HID_USAGE_SOC_FILE_PAYLOAD_CONTAINS_LAST_BYTES = 0x08, // DF + HID_USAGE_SOC_FILE_TRANSFER_STOP = 0x09, // DF + HID_USAGE_SOC_FILE_TRANSFER_TILL_END = 0x0A // DF + // Reserved (0x0B - 0xFFFF) +}; + +/// HID Usage Table: Eye and Head Trackers Page (0x12) +enum { + HID_USAGE_EYE_AND_HEAD_TRACKER_EYE_TRACKER = 0x0001, // CA + HID_USAGE_EYE_AND_HEAD_TRACKER_HEAD_TRACKER = 0x0002, // CA + // Reserved (0x0003 - 0x000F) + HID_USAGE_EYE_AND_HEAD_TRACKER_TRACKING_DATA = 0x0010, // CP + HID_USAGE_EYE_AND_HEAD_TRACKER_CAPABILITIES = 0x0011, // CL + HID_USAGE_EYE_AND_HEAD_TRACKER_CONFIGURATION = 0x0012, // CL + HID_USAGE_EYE_AND_HEAD_TRACKER_STATUS = 0x0013, // CL + HID_USAGE_EYE_AND_HEAD_TRACKER_CONTROL = 0x0014, // CL + // Reserved (0x0015 - 0x001F) + HID_USAGE_EYE_AND_HEAD_TRACKER_SENSOR_TIMESTAMP = 0x0020, // DV + HID_USAGE_EYE_AND_HEAD_TRACKER_POSITION_X = 0x0021, // DV + HID_USAGE_EYE_AND_HEAD_TRACKER_POSITION_Y = 0x0022, // DV + HID_USAGE_EYE_AND_HEAD_TRACKER_POSITION_Z = 0x0023, // DV + HID_USAGE_EYE_AND_HEAD_TRACKER_GAZE_POINT = 0x0024, // CP + HID_USAGE_EYE_AND_HEAD_TRACKER_LEFT_EYE_POSITION = 0x0025, // CP + HID_USAGE_EYE_AND_HEAD_TRACKER_RIGHT_EYE_POSITION = 0x0026, // CP + HID_USAGE_EYE_AND_HEAD_TRACKER_HEAD_POSITION = 0x0027, // CP + HID_USAGE_EYE_AND_HEAD_TRACKER_HEAD_DIRECTION_POINT = 0x0028, // CP + HID_USAGE_EYE_AND_HEAD_TRACKER_ROTATION_ABOUT_X_AXIS = 0x0029, // DV + HID_USAGE_EYE_AND_HEAD_TRACKER_ROTATION_ABOUT_Y_AXIS = 0x002A, // DV + HID_USAGE_EYE_AND_HEAD_TRACKER_ROTATION_ABOUT_Z_AXIS = 0x002B, // DV + // Reserved (0x002C - 0x00FF) + HID_USAGE_EYE_AND_HEAD_TRACKER_TRACKER_QUALITY = 0x0100, // SV + HID_USAGE_EYE_AND_HEAD_TRACKER_MINIMUM_TRACKING_DISTANCE = 0x0101, // SV + HID_USAGE_EYE_AND_HEAD_TRACKER_OPTIMUM_TRACKING_DISTANCE = 0x0102, // SV + HID_USAGE_EYE_AND_HEAD_TRACKER_MAXIMUM_TRACKING_DISTANCE = 0x0103, // SV + HID_USAGE_EYE_AND_HEAD_TRACKER_MAXIMUM_SCREEN_PLANE_WIDTH = 0x0104, // SV + HID_USAGE_EYE_AND_HEAD_TRACKER_MAXIMUM_SCREEN_PLANE_HEIGHT = 0x0105, // SV + // Reserved (0x0106 - 0x01FF) + HID_USAGE_EYE_AND_HEAD_TRACKER_DISPLAY_MANUFACTURER_ID = 0x0200, // SV + HID_USAGE_EYE_AND_HEAD_TRACKER_DISPLAY_PRODUCT_ID = 0x0201, // SV + HID_USAGE_EYE_AND_HEAD_TRACKER_DISPLAY_SERIAL_NUMBER = 0x0202, // SV + HID_USAGE_EYE_AND_HEAD_TRACKER_DISPLAY_MANUFACTURER_DATE = 0x0203, // SV + HID_USAGE_EYE_AND_HEAD_TRACKER_CALIBRATED_SCREEN_WIDTH = 0x0204, // SV + HID_USAGE_EYE_AND_HEAD_TRACKER_CALIBRATED_SCREEN_HEIGHT = 0x0205, // SV + // Reserved (0x0206 - 0x02FF) + HID_USAGE_EYE_AND_HEAD_TRACKER_SAMPLING_FREQUENCY = 0x0300, // DV + HID_USAGE_EYE_AND_HEAD_TRACKER_CONFIGURATION_STATUS = 0x0301, // DV + // Reserved (0x0302 - 0x03FF) + HID_USAGE_EYE_AND_HEAD_TRACKER_DEVICE_MODE_REQUEST = 0x0400, // DV + // Reserved (0x0401 - 0xFFFF) +}; + +/// HID Usage Table - Auxiliary Display Page (0x14) +enum { + HID_USAGE_AUX_DISPLAY_ALPHANUMERIC_DISPLAY = 0x01, // CA + HID_USAGE_AUX_DISPLAY_AUXILIARY_DISPLAY = 0x02, // CA + // Reserved (0x03 - 0x1F) + HID_USAGE_AUX_DISPLAY_DISPLAY_ATTRIBUTES_REPORT = 0x20, // CL + HID_USAGE_AUX_DISPLAY_ASCII_CHARACTER_SET = 0x21, // SF + HID_USAGE_AUX_DISPLAY_DATA_READ_BACK = 0x22, // SF + HID_USAGE_AUX_DISPLAY_FONT_READ_BACK = 0x23, // SF + HID_USAGE_AUX_DISPLAY_DISPLAY_CONTROL_REPORT = 0x24, // CL + HID_USAGE_AUX_DISPLAY_CLEAR_DISPLAY = 0x25, // DF + HID_USAGE_AUX_DISPLAY_DISPLAY_ENABLE = 0x26, // DF + HID_USAGE_AUX_DISPLAY_SCREEN_SAVER_DELAY = 0x27, // SV/DV + HID_USAGE_AUX_DISPLAY_SCREEN_SAVER_ENABLE = 0x28, // DF + HID_USAGE_AUX_DISPLAY_VERTICAL_SCROLL = 0x29, // SF/DF + HID_USAGE_AUX_DISPLAY_HORIZONTAL_SCROLL = 0x2A, // SF/DF + HID_USAGE_AUX_DISPLAY_CHARACTER_REPORT = 0x2B, // CL + HID_USAGE_AUX_DISPLAY_DISPLAY_DATA = 0x2C, // DV + HID_USAGE_AUX_DISPLAY_DISPLAY_STATUS = 0x2D, // CL + HID_USAGE_AUX_DISPLAY_STAT_NOT_READY = 0x2E, // Sel + HID_USAGE_AUX_DISPLAY_STAT_READY = 0x2F, // Sel + HID_USAGE_AUX_DISPLAY_ERR_NOT_A_LOADABLE_CHARACTER = 0x30, // Sel + HID_USAGE_AUX_DISPLAY_ERR_FONT_DATA_CANNOT_BE_READ = 0x31, // Sel + HID_USAGE_AUX_DISPLAY_CURSOR_POSITION_REPORT = 0x32, // Sel + HID_USAGE_AUX_DISPLAY_ROW = 0x33, // DV + HID_USAGE_AUX_DISPLAY_COLUMN = 0x34, // DV + HID_USAGE_AUX_DISPLAY_ROWS = 0x35, // SV + HID_USAGE_AUX_DISPLAY_COLUMNS = 0x36, // SV + HID_USAGE_AUX_DISPLAY_CURSOR_PIXEL_POSITIONING = 0x37, // SF + HID_USAGE_AUX_DISPLAY_CURSOR_MODE = 0x38, // DF + HID_USAGE_AUX_DISPLAY_CURSOR_ENABLE = 0x39, // DF + HID_USAGE_AUX_DISPLAY_CURSOR_BLINK = 0x3A, // DF + HID_USAGE_AUX_DISPLAY_FONT_REPORT = 0x3B, // CL + HID_USAGE_AUX_DISPLAY_FONT_DATA = 0x3C, // Buffered Bytes + HID_USAGE_AUX_DISPLAY_CHARACTER_WIDTH = 0x3D, // SV + HID_USAGE_AUX_DISPLAY_CHARACTER_HEIGHT = 0x3E, // SV + HID_USAGE_AUX_DISPLAY_CHARACTER_SPACING_HORIZONTAL = 0x3F, // SV + HID_USAGE_AUX_DISPLAY_CHARACTER_SPACING_VERTICAL = 0x40, // SV + HID_USAGE_AUX_DISPLAY_UNICODE_CHARACTER_SET = 0x41, // SF + HID_USAGE_AUX_DISPLAY_FONT_7_SEGMENT = 0x42, // SF + HID_USAGE_AUX_DISPLAY_7_SEGMENT_DIRECT_MAP = 0x43, // SF + HID_USAGE_AUX_DISPLAY_FONT_14_SEGMENT = 0x44, // SF + HID_USAGE_AUX_DISPLAY_14_SEGMENT_DIRECT_MAP = 0x45, // SF + HID_USAGE_AUX_DISPLAY_DISPLAY_BRIGHTNESS = 0x46, // DV + HID_USAGE_AUX_DISPLAY_DISPLAY_CONTRAST = 0x47, // DV + HID_USAGE_AUX_DISPLAY_CHARACTER_ATTRIBUTE = 0x48, // CL + HID_USAGE_AUX_DISPLAY_ATTRIBUTE_READBACK = 0x49, // SF + HID_USAGE_AUX_DISPLAY_ATTRIBUTE_DATA = 0x4A, // DV + HID_USAGE_AUX_DISPLAY_CHAR_ATTR_ENHANCE = 0x4B, // OOC + HID_USAGE_AUX_DISPLAY_CHAR_ATTR_UNDERLINE = 0x4C, // OOC + HID_USAGE_AUX_DISPLAY_CHAR_ATTR_BLINK = 0x4D, // OOC + // Reserved (0x4E - 0x7F) + HID_USAGE_AUX_DISPLAY_BITMAP_SIZE_X = 0x80, // SV + HID_USAGE_AUX_DISPLAY_BITMAP_SIZE_Y = 0x81, // SV + HID_USAGE_AUX_DISPLAY_MAX_BLIT_SIZE = 0x82, // SV + HID_USAGE_AUX_DISPLAY_BIT_DEPTH_FORMAT = 0x83, // SV + HID_USAGE_AUX_DISPLAY_DISPLAY_ORIENTATION = 0x84, // DV + HID_USAGE_AUX_DISPLAY_PALETTE_REPORT = 0x85, // CL + HID_USAGE_AUX_DISPLAY_PALETTE_DATA_SIZE = 0x86, // SV + HID_USAGE_AUX_DISPLAY_PALETTE_DATA_OFFSET = 0x87, // SV + HID_USAGE_AUX_DISPLAY_PALETTE_DATA = 0x88, // Buffered Bytes + // Reserved (0x89) + HID_USAGE_AUX_DISPLAY_BLIT_REPORT = 0x8A, // CL + HID_USAGE_AUX_DISPLAY_BLIT_RECTANGLE_X1 = 0x8B, // SV + HID_USAGE_AUX_DISPLAY_BLIT_RECTANGLE_Y1 = 0x8C, // SV + HID_USAGE_AUX_DISPLAY_BLIT_RECTANGLE_X2 = 0x8D, // SV + HID_USAGE_AUX_DISPLAY_BLIT_RECTANGLE_Y2 = 0x8E, // SV + HID_USAGE_AUX_DISPLAY_BLIT_DATA = 0x8F, // Buffered Bytes + HID_USAGE_AUX_DISPLAY_SOFT_BUTTON = 0x90, // CL + HID_USAGE_AUX_DISPLAY_SOFT_BUTTON_ID = 0x91, // SV + HID_USAGE_AUX_DISPLAY_SOFT_BUTTON_SIDE = 0x92, // SV + HID_USAGE_AUX_DISPLAY_SOFT_BUTTON_OFFSET_1 = 0x93, // SV + HID_USAGE_AUX_DISPLAY_SOFT_BUTTON_OFFSET_2 = 0x94, // SV + HID_USAGE_AUX_DISPLAY_SOFT_BUTTON_REPORT = 0x95, // SV + // Reserved (0x96 - 0xC1) + HID_USAGE_AUX_DISPLAY_SOFT_KEYS = 0xC2, // SV + // Reserved (0xC3 - 0xCB) + HID_USAGE_AUX_DISPLAY_DATA_EXTENSIONS = 0xCC, // SF + // Reserved (0xCD - 0xCE) + HID_USAGE_AUX_DISPLAY_CHARACTER_MAPPING = 0xCF, // SV + // Reserved (0xD0 - 0xDC) + HID_USAGE_AUX_DISPLAY_UNICODE_EQUIVALENT = 0xDD, // SV + // Reserved (0xDE) + HID_USAGE_AUX_DISPLAY_CHARACTER_PAGE_MAPPING = 0xDF, // SV + // Reserved (0xE0 - 0xFE) + HID_USAGE_AUX_DISPLAY_REQUEST_REPORT = 0xFF // DV + // Reserved (0x100 - 0xFFFF) +}; + +/// HID Usage Table - Medical Instrument Page (0x40) +enum { + HID_USAGE_MEDICAL_INSTRUMENT_MEDICAL_ULTRASOUND = 0x01, // CA + // Reserved (0x02 - 0x1F) + HID_USAGE_MEDICAL_INSTRUMENT_VCR_ACQUISITION = 0x20, // OOC + HID_USAGE_MEDICAL_INSTRUMENT_FREEZE_THAW = 0x21, // OOC + HID_USAGE_MEDICAL_INSTRUMENT_CLIP_STORE = 0x22, // OSC + HID_USAGE_MEDICAL_INSTRUMENT_UPDATE = 0x23, // OSC + HID_USAGE_MEDICAL_INSTRUMENT_NEXT = 0x24, // OSC + HID_USAGE_MEDICAL_INSTRUMENT_SAVE = 0x25, // OSC + HID_USAGE_MEDICAL_INSTRUMENT_PRINT = 0x26, // OSC + HID_USAGE_MEDICAL_INSTRUMENT_MICROPHONE_ENABLE = 0x27, // OSC + // Reserved (0x28 - 0x3F) + HID_USAGE_MEDICAL_INSTRUMENT_CINE = 0x40, // LC + HID_USAGE_MEDICAL_INSTRUMENT_TRANSMIT_POWER = 0x41, // LC + HID_USAGE_MEDICAL_INSTRUMENT_VOLUME = 0x42, // LC + HID_USAGE_MEDICAL_INSTRUMENT_FOCUS = 0x43, // LC + HID_USAGE_MEDICAL_INSTRUMENT_DEPTH = 0x44, // LC + // Reserved (0x45 - 0x5F) + HID_USAGE_MEDICAL_INSTRUMENT_SOFT_STEP_PRIMARY = 0x60, // LC + HID_USAGE_MEDICAL_INSTRUMENT_SOFT_STEP_SECONDARY = 0x61, // LC + // Reserved (0x62 - 0x6F) + HID_USAGE_MEDICAL_INSTRUMENT_DEPTH_GAIN_COMPENSATION = 0x70, // LC + // Reserved (0x71 - 0x7F) + HID_USAGE_MEDICAL_INSTRUMENT_ZOOM_SELECT = 0x80, // OSC + HID_USAGE_MEDICAL_INSTRUMENT_ZOOM_ADJUST = 0x81, // LC + HID_USAGE_MEDICAL_INSTRUMENT_SPECTRAL_DOPPLER_MODE_SELECT = 0x82, // OSC + HID_USAGE_MEDICAL_INSTRUMENT_SPECTRAL_DOPPLER_ADJUST = 0x83, // LC + HID_USAGE_MEDICAL_INSTRUMENT_COLOR_DOPPLER_MODE_SELECT = 0x84, // OSC + HID_USAGE_MEDICAL_INSTRUMENT_COLOR_DOPPLER_ADJUST = 0x85, // LC + HID_USAGE_MEDICAL_INSTRUMENT_MOTION_MODE_SELECT = 0x86, // OSC + HID_USAGE_MEDICAL_INSTRUMENT_MOTION_MODE_ADJUST = 0x87, // LC + HID_USAGE_MEDICAL_INSTRUMENT_2D_MODE_SELECT = 0x88, // OSC + HID_USAGE_MEDICAL_INSTRUMENT_2D_MODE_ADJUST = 0x89, // LC + // Reserved (0x8A - 0x9F) + HID_USAGE_MEDICAL_INSTRUMENT_SOFT_CONTROL_SELECT = 0xA0, // OSC + HID_USAGE_MEDICAL_INSTRUMENT_SOFT_CONTROL_ADJUST = 0xA1, // LC + // Reserved (0xA2 - 0xFFFF) }; /// HID Usage Table - Lighting And Illumination Page (0x59) enum { - HID_USAGE_LIGHTING_LAMP_ARRAY = 0x01, - HID_USAGE_LIGHTING_LAMP_ARRAY_ATTRIBUTES_REPORT = 0x02, - HID_USAGE_LIGHTING_LAMP_COUNT = 0x03, - HID_USAGE_LIGHTING_BOUNDING_BOX_WIDTH_IN_MICROMETERS = 0x04, - HID_USAGE_LIGHTING_BOUNDING_BOX_HEIGHT_IN_MICROMETERS = 0x05, - HID_USAGE_LIGHTING_BOUNDING_BOX_DEPTH_IN_MICROMETERS = 0x06, - HID_USAGE_LIGHTING_LAMP_ARRAY_KIND = 0x07, - HID_USAGE_LIGHTING_MIN_UPDATE_INTERVAL_IN_MICROSECONDS = 0x08, - HID_USAGE_LIGHTING_LAMP_ATTRIBUTES_REQUEST_REPORT = 0x20, - HID_USAGE_LIGHTING_LAMP_ID = 0x21, - HID_USAGE_LIGHTING_LAMP_ATTRIBUTES_RESPONSE_REPORT = 0x22, - HID_USAGE_LIGHTING_POSITION_X_IN_MICROMETERS = 0x23, - HID_USAGE_LIGHTING_POSITION_Y_IN_MICROMETERS = 0x24, - HID_USAGE_LIGHTING_POSITION_Z_IN_MICROMETERS = 0x25, - HID_USAGE_LIGHTING_LAMP_PURPOSES = 0x26, - HID_USAGE_LIGHTING_UPDATE_LATENCY_IN_MICROSECONDS = 0x27, - HID_USAGE_LIGHTING_RED_LEVEL_COUNT = 0x28, - HID_USAGE_LIGHTING_GREEN_LEVEL_COUNT = 0x29, - HID_USAGE_LIGHTING_BLUE_LEVEL_COUNT = 0x2A, - HID_USAGE_LIGHTING_INTENSITY_LEVEL_COUNT = 0x2B, - HID_USAGE_LIGHTING_IS_PROGRAMMABLE = 0x2C, - HID_USAGE_LIGHTING_INPUT_BINDING = 0x2D, - HID_USAGE_LIGHTING_LAMP_MULTI_UPDATE_REPORT = 0x50, - HID_USAGE_LIGHTING_RED_UPDATE_CHANNEL = 0x51, - HID_USAGE_LIGHTING_GREEN_UPDATE_CHANNEL = 0x52, - HID_USAGE_LIGHTING_BLUE_UPDATE_CHANNEL = 0x53, - HID_USAGE_LIGHTING_INTENSITY_UPDATE_CHANNEL = 0x54, - HID_USAGE_LIGHTING_LAMP_UPDATE_FLAGS = 0x55, - HID_USAGE_LIGHTING_LAMP_RANGE_UPDATE_REPORT = 0x60, - HID_USAGE_LIGHTING_LAMP_ID_START = 0x61, - HID_USAGE_LIGHTING_LAMP_ID_END = 0x62, - HID_USAGE_LIGHTING_LAMP_ARRAY_CONTROL_REPORT = 0x70, - HID_USAGE_LIGHTING_AUTONOMOUS_MODE = 0x71, + HID_USAGE_LIGHTING_LAMP_ARRAY = 0x01, // CA + HID_USAGE_LIGHTING_LAMP_ARRAY_ATTRIBUTES_REPORT = 0x02, // CL + HID_USAGE_LIGHTING_LAMP_COUNT = 0x03, // SV/DV + HID_USAGE_LIGHTING_BOUNDING_BOX_WIDTH_IN_MICROMETERS = 0x04, // SV + HID_USAGE_LIGHTING_BOUNDING_BOX_HEIGHT_IN_MICROMETERS = 0x05, // SV + HID_USAGE_LIGHTING_BOUNDING_BOX_DEPTH_IN_MICROMETERS = 0x06, // SV + HID_USAGE_LIGHTING_LAMP_ARRAY_KIND = 0x07, // SV + HID_USAGE_LIGHTING_MIN_UPDATE_INTERVAL_IN_MICROSECONDS = 0x08, // SV + // Reserved (0x09 - 0x1F) + HID_USAGE_LIGHTING_LAMP_ATTRIBUTES_REQUEST_REPORT = 0x20, // CL + HID_USAGE_LIGHTING_LAMP_ID = 0x21, // SV/DV + HID_USAGE_LIGHTING_LAMP_ATTRIBUTES_RESPONSE_REPORT = 0x22, // CL + HID_USAGE_LIGHTING_POSITION_X_IN_MICROMETERS = 0x23, // DV + HID_USAGE_LIGHTING_POSITION_Y_IN_MICROMETERS = 0x24, // DV + HID_USAGE_LIGHTING_POSITION_Z_IN_MICROMETERS = 0x25, // DV + HID_USAGE_LIGHTING_LAMP_PURPOSES = 0x26, // DV + HID_USAGE_LIGHTING_UPDATE_LATENCY_IN_MICROSECONDS = 0x27, // DV + HID_USAGE_LIGHTING_RED_LEVEL_COUNT = 0x28, // DV + HID_USAGE_LIGHTING_GREEN_LEVEL_COUNT = 0x29, // DV + HID_USAGE_LIGHTING_BLUE_LEVEL_COUNT = 0x2A, // DV + HID_USAGE_LIGHTING_INTENSITY_LEVEL_COUNT = 0x2B, // DV + HID_USAGE_LIGHTING_IS_PROGRAMMABLE = 0x2C, // DV + HID_USAGE_LIGHTING_INPUT_BINDING = 0x2D, // DV + // Reserved (0x2E - 0x4F) + HID_USAGE_LIGHTING_LAMP_MULTI_UPDATE_REPORT = 0x50, // CL + HID_USAGE_LIGHTING_RED_UPDATE_CHANNEL = 0x51, // DV + HID_USAGE_LIGHTING_GREEN_UPDATE_CHANNEL = 0x52, // DV + HID_USAGE_LIGHTING_BLUE_UPDATE_CHANNEL = 0x53, // DV + HID_USAGE_LIGHTING_INTENSITY_UPDATE_CHANNEL = 0x54, // DV + HID_USAGE_LIGHTING_LAMP_UPDATE_FLAGS = 0x55, // DV + // Reserved (0x56 - 0x5F) + HID_USAGE_LIGHTING_LAMP_RANGE_UPDATE_REPORT = 0x60, // CL + HID_USAGE_LIGHTING_LAMP_ID_START = 0x61, // DV + HID_USAGE_LIGHTING_LAMP_ID_END = 0x62, // DV + // Reserved (0x63 - 0x6F) + HID_USAGE_LIGHTING_LAMP_ARRAY_CONTROL_REPORT = 0x70, // CL + HID_USAGE_LIGHTING_AUTONOMOUS_MODE = 0x71, // DV + // Reserved (0x72 - 0xFFFF) +}; + +/// HID Usage Table: Monitor Page (0x80) +enum { + HID_USAGE_MONITOR_MONITOR_CONTROL = 0x01, // CA + HID_USAGE_MONITOR_EDID_INFORMATION = 0x02, // SV + HID_USAGE_MONITOR_VDIF_INFORMATION = 0x03, // SV + HID_USAGE_MONITOR_VESA_VERSION = 0x04 // SV + // Reserved (0x05 - 0xFFFF) +}; + +/// HID Usage Table: Monitor Enumerated Page (0x81) +/// Intentionally skipped + +/// HID Usage Table: VESA Virtual Controls Page (0x82) +enum { + HID_USAGE_VESA_VIRTUAL_CONTROLS_DEGAUSS = 0x01, // DV + // Reserved (0x02 - 0x0F) + HID_USAGE_VESA_VIRTUAL_CONTROLS_BRIGHTNESS = 0x10, // DV + // Reserved (0x11) + HID_USAGE_VESA_VIRTUAL_CONTROLS_CONTRAST = 0x12, // DV + // Reserved (0x13 - 0x15) + HID_USAGE_VESA_VIRTUAL_CONTROLS_RED_VIDEO_GAIN = 0x16, // DV + // Reserved (0x17) + HID_USAGE_VESA_VIRTUAL_CONTROLS_GREEN_VIDEO_GAIN = 0x18, // DV + // Reserved (0x19) + HID_USAGE_VESA_VIRTUAL_CONTROLS_BLUE_VIDEO_GAIN = 0x1A, // DV + // Reserved (0x1B) + HID_USAGE_VESA_VIRTUAL_CONTROLS_FOCUS = 0x1C, // DV + // Reserved (0x1D - 0x1F) + HID_USAGE_VESA_VIRTUAL_CONTROLS_HORIZONTAL_POSITION = 0x20, // DV + // Reserved (0x21) + HID_USAGE_VESA_VIRTUAL_CONTROLS_HORIZONTAL_SIZE = 0x22, // DV + // Reserved (0x23) + HID_USAGE_VESA_VIRTUAL_CONTROLS_HORIZONTAL_PINCUSHION = 0x24, // DV + // Reserved (0x25) + HID_USAGE_VESA_VIRTUAL_CONTROLS_HORIZONTAL_PINCUSHION_BALANCE = 0x26, // DV + // Reserved (0x27) + HID_USAGE_VESA_VIRTUAL_CONTROLS_HORIZONTAL_MISCONVERGENCE = 0x28, // DV + // Reserved (0x29) + HID_USAGE_VESA_VIRTUAL_CONTROLS_HORIZONTAL_LINEARITY = 0x2A, // DV + // Reserved (0x2B) + HID_USAGE_VESA_VIRTUAL_CONTROLS_HORIZONTAL_LINEARITY_BALANCE = 0x2C, // DV + // Reserved (0x2D - 0x2F) + HID_USAGE_VESA_VIRTUAL_CONTROLS_VERTICAL_POSITION = 0x30, // DV + // Reserved (0x31) + HID_USAGE_VESA_VIRTUAL_CONTROLS_VERTICAL_SIZE = 0x32, // DV + // Reserved (0x33) + HID_USAGE_VESA_VIRTUAL_CONTROLS_VERTICAL_PINCUSHION = 0x34, // DV + // Reserved (0x35) + HID_USAGE_VESA_VIRTUAL_CONTROLS_VERTICAL_PINCUSHION_BALANCE = 0x36, // DV + // Reserved (0x37) + HID_USAGE_VESA_VIRTUAL_CONTROLS_VERTICAL_MISCONVERGENCE = 0x38, // DV + // Reserved (0x39) + HID_USAGE_VESA_VIRTUAL_CONTROLS_VERTICAL_LINEARITY = 0x3A, // DV + // Reserved (0x3B) + HID_USAGE_VESA_VIRTUAL_CONTROLS_VERTICAL_LINEARITY_BALANCE = 0x3C, // DV + // Reserved (0x3D - 0x3F) + HID_USAGE_VESA_VIRTUAL_CONTROLS_PARALLELOGRAM_DISTORTION = 0x40, // DV + // Reserved (0x41) + HID_USAGE_VESA_VIRTUAL_CONTROLS_TRAPEZOIDAL_DISTORTION = 0x42, // DV + // Reserved (0x43) + HID_USAGE_VESA_VIRTUAL_CONTROLS_TILT = 0x44, // DV + // Reserved (0x45) + HID_USAGE_VESA_VIRTUAL_CONTROLS_TOP_CORNER_DISTORTION_CONTROL = 0x46, // DV + // Reserved (0x47) + HID_USAGE_VESA_VIRTUAL_CONTROLS_TOP_CORNER_DISTORTION_BALANCE = 0x48, // DV + // Reserved (0x49) + HID_USAGE_VESA_VIRTUAL_CONTROLS_BOTTOM_CORNER_DISTORTION_CONTROL = 0x4A, // DV + // Reserved (0x4B) + HID_USAGE_VESA_VIRTUAL_CONTROLS_BOTTOM_CORNER_DISTORTION_BALANCE = 0x4C, // DV + // Reserved (0x4D - 0x55) + HID_USAGE_VESA_VIRTUAL_CONTROLS_HORIZONTAL_MOIRE = 0x56, // DV + // Reserved (0x57) + HID_USAGE_VESA_VIRTUAL_CONTROLS_VERTICAL_MOIRE = 0x58, // DV + // Reserved (0x59 - 0x5D) + HID_USAGE_VESA_VIRTUAL_CONTROLS_INPUT_LEVEL_SELECT = 0x5E, // NAry + // Reserved (0x5F) + HID_USAGE_VESA_VIRTUAL_CONTROLS_INPUT_SOURCE_SELECT = 0x60, // NAry + // Reserved (0x61 - 0x6B) + HID_USAGE_VESA_VIRTUAL_CONTROLS_RED_VIDEO_BLACK_LEVEL = 0x6C, // DV + // Reserved (0x6D) + HID_USAGE_VESA_VIRTUAL_CONTROLS_GREEN_VIDEO_BLACK_LEVEL = 0x6E, // DV + // Reserved (0x6F) + HID_USAGE_VESA_VIRTUAL_CONTROLS_BLUE_VIDEO_BLACK_LEVEL = 0x70, // DV + // Reserved (0x71 - 0xA1) + HID_USAGE_VESA_VIRTUAL_CONTROLS_AUTO_SIZE_CENTER = 0xA2, // NAry + // Reserved (0xA3) + HID_USAGE_VESA_VIRTUAL_CONTROLS_POLARITY_HORIZONTAL_SYNC = 0xA4, // NAry + // Reserved (0xA5) + HID_USAGE_VESA_VIRTUAL_CONTROLS_POLARITY_VERTICAL_SYNC = 0xA6, // NAry + // Reserved (0xA7) + HID_USAGE_VESA_VIRTUAL_CONTROLS_SYNC_TYPE = 0xA8, // NAry + // Reserved (0xA9) + HID_USAGE_VESA_VIRTUAL_CONTROLS_SCREEN_ORIENTATION = 0xAA, // NAry + // Reserved (0xAB) + HID_USAGE_VESA_VIRTUAL_CONTROLS_HORIZONTAL_FREQUENCY = 0xAC, // DV + // Reserved (0xAD) + HID_USAGE_VESA_VIRTUAL_CONTROLS_VERTICAL_FREQUENCY = 0xAE, // DV + // Reserved (0xAF) + HID_USAGE_VESA_VIRTUAL_CONTROLS_SETTINGS = 0xB0, // NAry + // Reserved (0xB1 - 0xC9) + HID_USAGE_VESA_VIRTUAL_CONTROLS_ON_SCREEN_DISPLAY = 0xCA, // NAry + // Reserved (0xCB - 0xD3) + HID_USAGE_VESA_VIRTUAL_CONTROLS_STEREO_MODE = 0xD4, // NAry + // Reserved (0xD5 - 0xFFFF) }; /// HID Usage Table: Power Device Page (0x84) @@ -1665,6 +2579,7 @@ enum { HID_USAGE_POWER_I_MANUFACTURER = 0xFD, HID_USAGE_POWER_I_PRODUCT = 0xFE, HID_USAGE_POWER_I_SERIAL_NUMBER = 0xFF + // Reserved (0x100 - 0xFFFF) }; /// HID Usage Table: Battery System Page (0x85) @@ -1772,6 +2687,48 @@ enum { // F4-FF Reserved }; +/// HID Usage Table: Camera Control Page (0x90) +enum { + // Reserved (0x01 - 0x1F) + HID_USAGE_CAMERA_CONTROL_CAMERA_AUTO_FOCUS = 0x20, // OSC + HID_USAGE_CAMERA_CONTROL_CAMERA_SHUTTER = 0x21 // OSC + // Reserved (0x22 - 0xFFFF) +}; + +/// HID Usage Table: Arcade Page (0x91) +enum { + HID_USAGE_ARCADE_GENERAL_PURPOSE_IO_CARD = 0x01, // CA + HID_USAGE_ARCADE_COIN_DOOR = 0x02, // CA + HID_USAGE_ARCADE_WATCHDOG_TIMER = 0x03, // CA + // Reserved (0x04 - 0x2F) + HID_USAGE_ARCADE_GENERAL_PURPOSE_ANALOG_INPUT_STATE = 0x30, // DV + HID_USAGE_ARCADE_GENERAL_PURPOSE_DIGITAL_INPUT_STATE = 0x31, // DV + HID_USAGE_ARCADE_GENERAL_PURPOSE_OPTICAL_INPUT_STATE = 0x32, // DV + HID_USAGE_ARCADE_GENERAL_PURPOSE_DIGITAL_OUTPUT_STATE = 0x33, // DV + HID_USAGE_ARCADE_NUMBER_OF_COIN_DOORS = 0x34, // DV + HID_USAGE_ARCADE_COIN_DRAWER_DROP_COUNT = 0x35, // DV + HID_USAGE_ARCADE_COIN_DRAWER_START = 0x36, // OOC + HID_USAGE_ARCADE_COIN_DRAWER_SERVICE = 0x37, // OOC + HID_USAGE_ARCADE_COIN_DRAWER_TILT = 0x38, // OOC + HID_USAGE_ARCADE_COIN_DOOR_TEST = 0x39, // OOC + // Reserved (0x3A - 0x3F) + HID_USAGE_ARCADE_COIN_DOOR_LOCKOUT = 0x40, // OOC + HID_USAGE_ARCADE_WATCHDOG_TIMEOUT = 0x41, // DV + HID_USAGE_ARCADE_WATCHDOG_ACTION = 0x42, // NAry + HID_USAGE_ARCADE_WATCHDOG_REBOOT = 0x43, // Sel + HID_USAGE_ARCADE_WATCHDOG_RESTART = 0x44, // Sel + HID_USAGE_ARCADE_ALARM_INPUT = 0x45, // DV + HID_USAGE_ARCADE_COIN_DOOR_COUNTER = 0x46, // OOC + HID_USAGE_ARCADE_IO_DIRECTION_MAPPING = 0x47, // DV + HID_USAGE_ARCADE_SET_IO_DIRECTION_MAPPING = 0x48, // DV + HID_USAGE_ARCADE_EXTENDED_OPTICAL_INPUT_STATE = 0x49, // DV + HID_USAGE_ARCADE_PIN_PAD_INPUT_STATE = 0x4A, // DV + HID_USAGE_ARCADE_PIN_PAD_STATUS = 0x4B, // DV + HID_USAGE_ARCADE_PIN_PAD_OUTPUT = 0x4C, // OOC + HID_USAGE_ARCADE_PIN_PAD_COMMAND = 0x4D, // DV + // Reserved (0x4E - 0xFFFF) +}; + /// HID Usage Table: FIDO Alliance Page (0xF1D0) enum { HID_USAGE_FIDO_U2FHID = 0x01, // U2FHID usage for top-level collection diff --git a/src/class/midi/midi2_device.c b/src/class/midi/midi2_device.c index 1d40a2efa..b0a9e2503 100644 --- a/src/class/midi/midi2_device.c +++ b/src/class/midi/midi2_device.c @@ -36,6 +36,9 @@ TU_ATTR_WEAK const char* tud_midi2_fb_name_cb(uint8_t itf, uint8_t fb_idx) { TU_ATTR_WEAK tud_midi2_stream_result_t tud_midi2_stream_msg_cb(uint8_t itf, const uint32_t* ump_words) { (void) itf; (void) ump_words; return MIDI2_STREAM_PASS; } +TU_ATTR_WEAK bool tud_midi2_device_identity_cb(uint8_t itf, tud_midi2_device_identity_t* identity) { + (void) itf; (void) identity; return false; +} //--------------------------------------------------------------------+ // Byte order note @@ -59,6 +62,7 @@ enum { enum { STREAM_ENDPOINT_DISCOVERY = 0x000, STREAM_ENDPOINT_INFO = 0x001, + STREAM_DEVICE_IDENTITY = 0x002, STREAM_EP_NAME = 0x003, STREAM_PROD_INSTANCE_ID = 0x004, STREAM_CONFIG_REQUEST = 0x005, @@ -103,6 +107,16 @@ typedef struct { uint8_t protocol; bool negotiated; + // Discovery reply bits waiting for TX FIFO room, drained on TX complete + uint8_t nego_pending_ep_filter; + uint8_t nego_pending_fb_filter; + uint8_t nego_pending_fb_num; // block requested by the pending discovery, 0xFF = all + uint8_t nego_pending_fb_next; // next block index to reply for + bool nego_pending_fb_restart; // restart after the active FB name when requests merge + uint16_t nego_text_status; // text reply owning nego_text_offset, 0 = none + uint16_t nego_text_offset; // progress into the text reply being sent + uint8_t nego_text_index; // Function Block index for an active FB name + /*------------- From this point, data is not cleared by bus reset -------------*/ struct { midi2d_tx_t tx; @@ -327,16 +341,20 @@ static void _nego_send_endpoint_info(midi2d_interface_t* p_midi) { // index byte (the Function Block number for FB Name) and 13 chars fit per // packet; otherwise the text starts there and 14 chars fit (Endpoint Name, // Product Instance Id). -static void _nego_send_stream_text(midi2d_interface_t* p_midi, uint16_t status, - bool has_index, uint8_t index, const char* str) { - if (!str || str[0] == '\0') return; +// Sends a stream text from `offset` and returns how far it got. Resuming keeps +// the End packet, which dropping the tail would lose. +static uint16_t _nego_send_stream_text(midi2d_interface_t* p_midi, uint16_t status, + bool has_index, uint8_t index, const char* str, + uint16_t offset) { + if (!str || str[0] == '\0') return 0; - uint16_t total_len = (uint16_t) strlen(str); - uint16_t offset = 0; + const uint16_t total_len = (uint16_t) strlen(str); const uint8_t per_pkt = has_index ? 13 : 14; const uint8_t head_chars = has_index ? 1 : 2; // chars carried in word0 + if (offset >= total_len) return total_len; while (offset < total_len) { + if (tu_fifo_remaining(&p_midi->ep_stream.tx.ff) < 16) break; uint16_t remaining = total_len - offset; uint8_t n = (uint8_t)((remaining > per_pkt) ? per_pkt : remaining); bool is_first = (offset == 0); @@ -370,6 +388,7 @@ static void _nego_send_stream_text(midi2d_interface_t* p_midi, uint16_t status, _nego_send_ump(p_midi, msg, 4); offset += n; } + return offset; } static void _nego_send_config_notify(midi2d_interface_t* p_midi, uint8_t protocol) { @@ -380,6 +399,33 @@ static void _nego_send_config_notify(midi2d_interface_t* p_midi, uint8_t protoco _nego_send_ump(p_midi, msg, 4); } +static void _nego_send_device_identity(midi2d_interface_t* p_midi) { + tud_midi2_device_identity_t id; + tu_memclr(&id, sizeof(id)); + if (!tud_midi2_device_identity_cb(_itf_idx(p_midi), &id)) return; + + // Every field is a run of bytes, each carrying 7 bits, laid out in the same + // order as the MIDI 1.0 Device Inquiry reply this message mirrors. A 1-byte + // manufacturer ID occupies the first of the three bytes, the other two stay + // zero, so the caller passes it as 0x7D0000 and not 0x00007D. + uint32_t msg[4] = {0}; + msg[0] = ((uint32_t) MT_STREAM << 28) + | ((uint32_t) STREAM_DEVICE_IDENTITY << 16); + msg[1] = id.manufacturer & UINT32_C(0x7F7F7F); + // Family and model are 14-bit numbers sent least significant byte first, + // as in the Device Inquiry reply. Manufacturer above is a byte sequence + // rather than a number, so it keeps its own order. + msg[2] = ((uint32_t) (id.family & 0x7F) << 24) + | ((uint32_t) ((id.family >> 7) & 0x7F) << 16) + | ((uint32_t) (id.model & 0x7F) << 8) + | ((uint32_t) ((id.model >> 7) & 0x7F)); + msg[3] = ((uint32_t) ((id.sw_revision >> 24) & 0x7F) << 24) + | ((uint32_t) ((id.sw_revision >> 16) & 0x7F) << 16) + | ((uint32_t) ((id.sw_revision >> 8) & 0x7F) << 8) + | ((uint32_t) (id.sw_revision & 0x7F)); + _nego_send_ump(p_midi, msg, 4); +} + static void _nego_send_fb_info(midi2d_interface_t* p_midi, uint8_t fb_idx) { // Derive direction and group span for this block from the GTB descriptor. uint16_t gtb_len = 0; @@ -395,10 +441,122 @@ static void _nego_send_fb_info(midi2d_interface_t* p_midi, uint8_t fb_idx) { | ((uint32_t) fb_idx << 8) | _fb_dir_byte(type); // UI hint + bDirection from the GTB block type msg[1] = ((uint32_t) first_group << 24) - | ((uint32_t) num_groups << 16); + | ((uint32_t) num_groups << 16) + | ((uint32_t) (CFG_TUD_MIDI2_FB_CI_VERSION & 0xFF) << 8) + | ((uint32_t) (CFG_TUD_MIDI2_FB_SYSEX8_STREAMS & 0xFF)); _nego_send_ump(p_midi, msg, 4); } +static void _nego_clear_pending(midi2d_interface_t* p_midi) { + p_midi->nego_pending_ep_filter = 0; + p_midi->nego_pending_fb_filter = 0; + p_midi->nego_pending_fb_num = 0; + p_midi->nego_pending_fb_next = 0; + p_midi->nego_pending_fb_restart = false; + p_midi->nego_text_status = 0; + p_midi->nego_text_offset = 0; + p_midi->nego_text_index = 0; +} + +static const char* _nego_text_cb(midi2d_interface_t* p_midi, uint16_t status, uint8_t index) { + const uint8_t itf = _itf_idx(p_midi); + switch (status) { + case STREAM_EP_NAME: return tud_midi2_ep_name_cb(itf); + case STREAM_PROD_INSTANCE_ID: return tud_midi2_product_id_cb(itf); + case STREAM_FB_NAME: return tud_midi2_fb_name_cb(itf, index); + default: return NULL; + } +} + +// Send or resume one text reply. While it is incomplete, its status and index +// identify the sole owner of nego_text_offset so another discovery request +// cannot resume a different string from the same offset. +static bool _nego_send_text(midi2d_interface_t* p_midi, uint16_t status, uint8_t index) { + const char* text = _nego_text_cb(p_midi, status, index); + const uint16_t len = text ? (uint16_t) strlen(text) : 0; + + p_midi->nego_text_status = status; + p_midi->nego_text_index = index; + p_midi->nego_text_offset = _nego_send_stream_text(p_midi, status, status == STREAM_FB_NAME, + index, text, p_midi->nego_text_offset); + if (p_midi->nego_text_offset < len) return false; + + p_midi->nego_text_status = 0; + p_midi->nego_text_offset = 0; + p_midi->nego_text_index = 0; + return true; +} + +// Send pending discovery replies, one whole reply at a time and only when the +// TX FIFO can take it. A full-filter Endpoint Discovery asks for more bytes +// than the default FIFO holds; replies that do not fit stay pending and are +// retried from the TX complete path, paced by the transfer flow. +static void _nego_send_pending(midi2d_interface_t* p_midi) { + tu_fifo_t* tx_ff = &p_midi->ep_stream.tx.ff; + + // An incomplete text sequence must finish before any newly arrived request + // is serviced; otherwise its Continue/End packets could be attached to a + // different Endpoint or Function Block string. + if (p_midi->nego_text_status) { + const uint16_t status = p_midi->nego_text_status; + const uint8_t index = p_midi->nego_text_index; + if (!_nego_send_text(p_midi, status, index)) return; + + if (status == STREAM_FB_NAME) { + if (p_midi->nego_pending_fb_restart) { + p_midi->nego_pending_fb_next = 0; + p_midi->nego_pending_fb_restart = false; + } else { + p_midi->nego_pending_fb_next++; + } + } else { + const uint8_t bit = (status == STREAM_EP_NAME) ? 0x04 : 0x08; + p_midi->nego_pending_ep_filter &= (uint8_t) ~bit; + } + } + + while (p_midi->nego_pending_ep_filter) { + const uint8_t bit = (uint8_t)(p_midi->nego_pending_ep_filter & (uint8_t)(-p_midi->nego_pending_ep_filter)); + uint16_t status = 0; + switch (bit) { + case 0x04: status = STREAM_EP_NAME; break; + case 0x08: status = STREAM_PROD_INSTANCE_ID; break; + default: break; + } + + if (status != 0) { + if (!_nego_send_text(p_midi, status, 0)) return; + } else { + if (tu_fifo_remaining(tx_ff) < 16) return; + switch (bit) { + case 0x01: _nego_send_endpoint_info(p_midi); break; + case 0x02: _nego_send_device_identity(p_midi); break; + case 0x10: _nego_send_config_notify(p_midi, p_midi->protocol); break; + default: break; + } + } + p_midi->nego_pending_ep_filter &= (uint8_t) ~bit; + } + + const uint8_t fb_count = _gtb_block_count(p_midi); + while (p_midi->nego_pending_fb_filter && p_midi->nego_pending_fb_next < fb_count) { + const uint8_t f = p_midi->nego_pending_fb_next; + if (p_midi->nego_pending_fb_num != 0xFF && p_midi->nego_pending_fb_num != f) { + p_midi->nego_pending_fb_next++; + continue; + } + if ((p_midi->nego_pending_fb_filter & 0x01) && p_midi->nego_text_offset == 0) { + if (tu_fifo_remaining(tx_ff) < 16) return; + _nego_send_fb_info(p_midi, f); + } + if (p_midi->nego_pending_fb_filter & 0x02) { + if (!_nego_send_text(p_midi, STREAM_FB_NAME, f)) return; + } + p_midi->nego_pending_fb_next++; + } + if (p_midi->nego_pending_fb_next >= fb_count) p_midi->nego_pending_fb_filter = 0; +} + static void _nego_handle_stream_msg(midi2d_interface_t* p_midi, const uint32_t* words) { // Let the application override this message before the built-in responder. switch (tud_midi2_stream_msg_cb(_itf_idx(p_midi), words)) { @@ -421,9 +579,9 @@ static void _nego_handle_stream_msg(midi2d_interface_t* p_midi, const uint32_t* switch (status) { case STREAM_ENDPOINT_DISCOVERY: - _nego_send_endpoint_info(p_midi); - _nego_send_stream_text(p_midi, STREAM_EP_NAME, false, 0, tud_midi2_ep_name_cb(_itf_idx(p_midi))); - _nego_send_stream_text(p_midi, STREAM_PROD_INSTANCE_ID, false, 0, tud_midi2_product_id_cb(_itf_idx(p_midi))); + // Filter bitmap: each bit set asks for one individual reply. + p_midi->nego_pending_ep_filter |= (uint8_t)(words[1] & 0x1F); + _nego_send_pending(p_midi); break; case STREAM_CONFIG_REQUEST: { @@ -437,14 +595,23 @@ static void _nego_handle_stream_msg(midi2d_interface_t* p_midi, const uint32_t* } case STREAM_FB_DISCOVERY: { - uint8_t fb_idx = (words[0] >> 8) & 0xFF; - uint8_t filter = words[0] & 0xFF; // bit 0: FB Info, bit 1: FB Name - uint8_t fb_count = _gtb_block_count(p_midi); - for (uint8_t f = 0; f < fb_count; f++) { - if (fb_idx != 0xFF && fb_idx != f) continue; - if (filter & 0x01) _nego_send_fb_info(p_midi, f); - if (filter & 0x02) _nego_send_stream_text(p_midi, STREAM_FB_NAME, true, f, tud_midi2_fb_name_cb(_itf_idx(p_midi), f)); + const uint8_t req_num = (uint8_t)((words[0] >> 8) & 0xFF); + const uint8_t req_filter = (uint8_t)(words[0] & 0x03); + // Merge with a pending request: repeating a Function Block Info is allowed + // at any time, losing a requested one is not. + if (req_filter && p_midi->nego_pending_fb_filter) { + if (p_midi->nego_pending_fb_num != req_num) p_midi->nego_pending_fb_num = 0xFF; + if (p_midi->nego_text_status == STREAM_FB_NAME) { + p_midi->nego_pending_fb_restart = true; + } else { + p_midi->nego_pending_fb_next = 0; + } + } else if (!p_midi->nego_pending_fb_filter) { + p_midi->nego_pending_fb_num = req_num; + p_midi->nego_pending_fb_next = 0; } + p_midi->nego_pending_fb_filter |= req_filter; // bit 0: FB Info, bit 1: FB Name + _nego_send_pending(p_midi); break; } @@ -754,6 +921,7 @@ bool midi2d_control_xfer_cb(uint8_t rhport, uint8_t stage, const tusb_control_re tu_edpt_stream_clear(&p_midi->ep_stream.rx); tu_fifo_clear(&p_midi->ep_stream.tx.ff); + _nego_clear_pending(p_midi); if (alt == 1) { p_midi->negotiated = false; @@ -824,6 +992,10 @@ bool midi2d_xfer_cb(uint8_t rhport, uint8_t ep_addr, xfer_result_t result, uint3 } tu_edpt_stream_read_xfer(ep_rx); } else if (ep_addr == ep_tx->ep_addr && result == XFER_RESULT_SUCCESS) { + // Completed transfer freed FIFO room: flush discovery replies still pending. + if (p_midi->alt_setting == 1) { + _nego_send_pending(p_midi); + } uint16_t queued = _tx_start_xfer(p_midi); // Send ZLP if no more data is queued but the last transfer was exactly mps if (queued == 0 && tu_fifo_count(&ep_tx->ff) == 0 && xferred_bytes > 0 && diff --git a/src/class/midi/midi2_device.h b/src/class/midi/midi2_device.h index 171b404b7..e3eb084d9 100644 --- a/src/class/midi/midi2_device.h +++ b/src/class/midi/midi2_device.h @@ -58,6 +58,17 @@ extern "C" { #define CFG_TUD_MIDI2_PRODUCT_ID "TinyUSB-MIDI2" #endif +// Function Block capabilities reported in Function Block Info Notification. +// The GTB descriptor carries direction and group span, but not these: they +// depend on what the application implements, so they default to "none". +#ifndef CFG_TUD_MIDI2_FB_CI_VERSION + #define CFG_TUD_MIDI2_FB_CI_VERSION 0 // 0: none or unknown, 1 or higher: MIDI-CI version +#endif + +#ifndef CFG_TUD_MIDI2_FB_SYSEX8_STREAMS + #define CFG_TUD_MIDI2_FB_SYSEX8_STREAMS 0 // 0: unsupported, 1: single, 2-255: simultaneous streams +#endif + // String descriptor index for the Group Terminal Block (iBlockItem, Table 5-6). // 0 = no string descriptor (default, spec-allowed). #ifndef CFG_TUD_MIDI2_BLOCK_STRIDX @@ -118,6 +129,17 @@ typedef enum { MIDI2_STREAM_NEGOTIATED_MIDI2, } tud_midi2_stream_result_t; +// Device identity fields, as defined for the MIDI 1.0 Device Inquiry reply and +// reused by the Device Identity Notification. Every byte carries 7 bits. +// A 1-byte System Exclusive ID goes in the first of the three manufacturer +// bytes, so 0x7D is passed as 0x7D0000. +typedef struct { + uint32_t manufacturer; // 3 bytes, first byte is most significant + uint16_t family; // 2 bytes + uint16_t model; // 2 bytes + uint32_t sw_revision; // 4 bytes +} tud_midi2_device_identity_t; + //--------------------------------------------------------------------+ // Application Callback API (weak, optional) //--------------------------------------------------------------------+ @@ -138,6 +160,12 @@ const uint8_t* tud_midi2_gtb_desc_cb(uint8_t itf, uint16_t* len); // discovery. Return NULL or "" for no name. const char* tud_midi2_fb_name_cb(uint8_t itf, uint8_t fb_idx); +// Optional device identity, sent as a Device Identity Notification when the +// host sets the 'd' bit in the Endpoint Discovery filter. Same four fields as +// the MIDI 1.0 Device Inquiry reply. Return false to skip the notification, +// which is the default. All values are 7-bit per byte. +bool tud_midi2_device_identity_cb(uint8_t itf, tud_midi2_device_identity_t* identity); + // Optional: intercept an incoming UMP Stream message (MT 0xF). Return PASS to // let the built-in responder handle it, or HANDLED / NEGOTIATED_* if the app // answered it (e.g. via tud_midi2_n_ump_write). Lets an app override a single diff --git a/src/class/mtp/mtp_device.c b/src/class/mtp/mtp_device.c index 7657899ec..275c9f858 100644 --- a/src/class/mtp/mtp_device.c +++ b/src/class/mtp/mtp_device.c @@ -437,8 +437,11 @@ bool mtpd_xfer_cb(uint8_t rhport, uint8_t ep_addr, xfer_result_t event, uint32_t TU_LOG_DRV(" MTP Data %s CB: xferred_bytes=%lu, xferred_len/total_len=%lu/%lu, is_complete=%d\r\n", is_data_in ? "IN" : "OUT", xferred_bytes, p_mtp->xferred_len, p_mtp->total_len, is_complete ? 1 : 0); - // Send/queue ZLP if packet is full-sized but transfer is complete - if (is_complete && xferred_bytes > 0 && !(xferred_bytes & (threshold - 1))) { + // Send/queue ZLP if packet is full-sized but transfer is complete. + // OUT must deliver this final payload to the application before receiving + // its terminating ZLP below. + const bool need_zlp = is_complete && xferred_bytes > 0 && !(xferred_bytes & (threshold - 1)); + if (is_data_in && need_zlp) { TU_LOG_DRV(" queue ZLP\r\n"); TU_VERIFY(usbd_edpt_claim(p_mtp->rhport, ep_addr)); TU_ASSERT(usbd_edpt_xfer(p_mtp->rhport, ep_addr, NULL, 0, false)); @@ -466,9 +469,16 @@ bool mtpd_xfer_cb(uint8_t rhport, uint8_t ep_addr, xfer_result_t event, uint32_t cb_data.io_container = headerless_packet; cb_data.io_container.payload_bytes = xferred_bytes; } - tud_mtp_data_xfer_cb(&cb_data); + if (xferred_bytes > 0) { + tud_mtp_data_xfer_cb(&cb_data); + } - if (is_complete) { + if (need_zlp) { + TU_LOG_DRV(" queue ZLP\r\n"); + TU_VERIFY(usbd_edpt_claim(p_mtp->rhport, ep_addr)); + TU_ASSERT(usbd_edpt_xfer(p_mtp->rhport, ep_addr, NULL, 0, false)); + return true; + } else if (is_complete) { // back to header + payload for response cb_data.io_container = headered_packet; cb_data.io_container.header->len = sizeof(mtp_container_header_t); diff --git a/src/class/net/ncm_device.c b/src/class/net/ncm_device.c index 84a524f49..72b592787 100644 --- a/src/class/net/ncm_device.c +++ b/src/class/net/ncm_device.c @@ -800,31 +800,56 @@ static void tud_network_recv_renew_r(uint8_t rhport) { } // tud_network_recv_renew /** - * Set the link state and send notification to host + * usbd-task trampoline for tud_network_link_state(), packing rhport and is_up + * into a single pointer-sized argument. + * + * Runs entirely in the usbd task context, so it cannot race the notify + * xfer-completion callback over the notification state machine. Re-arming + * notification_xmit_state and kicking notification_xmit() (rather than + * sending NETWORK_CONNECTION directly) means a state change that collides + * with an in-flight notification is picked up by the existing completion + * callback instead of being silently dropped - which would otherwise leave + * the host stuck at NO-CARRIER after a link-state change. */ -void tud_network_link_state(uint8_t rhport, bool is_up) { - TU_LOG_DRV("tud_network_link_state(%d, %d)\n", rhport, is_up); +static void ncm_link_state_task(void *param) { + uintptr_t const arg = (uintptr_t) param; + uint8_t const rhport = (uint8_t) (arg >> 1); + bool const is_up = (arg & 1u) != 0; if (ncm_interface.link_is_up == is_up) { - // No change in link state - return; + return; // no change in link state } ncm_interface.link_is_up = is_up; - // Only send notification if we have an active data interface if (ncm_interface.itf_data_alt != 1) { - TU_LOG_DRV(" link state notification skipped (interface not active)\n"); - return; + TU_LOG_DRV(" link state notification deferred (interface not active)\n"); + return; // data interface not active yet; SET_INTERFACE(alt=1) will notify } - // Reset notification state to send speed change notification first, then link state notification + // A link toggle does not change the link speed, so strictly only the + // NETWORK_CONNECTION notification would need (re)sending. Re-running the + // speed-then-connection sequence keeps this on the same state machine the + // completion callback already drives, at the cost of a redundant speed + // notification on every toggle. ncm_interface.notification_xmit_state = NOTIFICATION_SPEED; - - // Trigger notification transmission notification_xmit(rhport, false); } +/** + * Set the link state and notify the host. + * + * Defers onto the usbd task so a caller running in a different task than + * tud_task() cannot race the notification state machine against the notify + * xfer-completion callback. + */ +void tud_network_link_state(uint8_t rhport, bool is_up) { + TU_LOG_DRV("tud_network_link_state(%d, %d)\n", rhport, is_up); + + uintptr_t const arg = ((uintptr_t) rhport << 1) | (is_up ? 1u : 0u); + usbd_defer_func(ncm_link_state_task, (void *) arg, false); +} + //----------------------------------------------------------------------------- // // all the netd_*() stuff (interface TinyUSB -> driver) diff --git a/src/class/usbtmc/usbtmc_device.c b/src/class/usbtmc/usbtmc_device.c index 07190d89f..0e9978a81 100644 --- a/src/class/usbtmc/usbtmc_device.c +++ b/src/class/usbtmc/usbtmc_device.c @@ -497,9 +497,24 @@ bool usbtmcd_xfer_cb(uint8_t rhport, uint8_t ep_addr, xfer_result_t result, uint #if (CFG_TUD_USBTMC_ENABLE_488) case USBTMC_MSGID_USB488_TRIGGER: - // Spec says we halt the EP if we didn't declare we support it. - TU_VERIFY(usbtmc_state.capabilities->bmIntfcCapabilities488.supportsTrigger); - TU_VERIFY(tud_usbtmc_msg_trigger_cb(msg)); + // Unlike the messages above, TRIGGER is complete on arrival and has no response, so nothing else + // will move us out of STATE_IDLE. Do it here, otherwise the tud_usbtmc_start_bus_read() below (and + // any call the application makes from its callback) is a no-op and the bulk-OUT endpoint is left + // un-armed, silently timing out every subsequent host transfer. + TU_VERIFY(atomicChangeState(STATE_IDLE, STATE_NAK)); + + // Spec says we halt the EP if we didn't declare we support it; do the same when the application + // rejects the trigger. The callback result must not be wrapped in TU_VERIFY() here: returning + // early would skip both the stall and the re-arm below. + if (!usbtmc_state.capabilities->bmIntfcCapabilities488.supportsTrigger || + !tud_usbtmc_msg_trigger_cb(msg)) { + usbd_edpt_stall(rhport, usbtmc_state.ep_bulk_out); + return false; + } + // Result deliberately ignored: false here means the endpoint is already armed - either the + // application re-armed it from its callback, or a transfer is still queued - not that arming + // failed. Stalling on it would halt a healthy endpoint. + tud_usbtmc_start_bus_read(); break; #endif diff --git a/src/class/usbtmc/usbtmc_device.h b/src/class/usbtmc/usbtmc_device.h index 3dc700876..efda84f16 100644 --- a/src/class/usbtmc/usbtmc_device.h +++ b/src/class/usbtmc/usbtmc_device.h @@ -25,7 +25,6 @@ // * tud_usbtmc_open_cb // * tud_usbtmc_msg_data_cb // * tud_usbtmc_msgBulkIn_complete_cb -// * tud_usbtmc_msg_trigger_cb // * (successful) tud_usbtmc_check_abort_bulk_out_cb // * (successful) tud_usbtmc_check_abort_bulk_in_cb // * (successful) tud_usmtmc_bulkOut_clearFeature_cb diff --git a/src/class/vendor/vendor_device.c b/src/class/vendor/vendor_device.c index b3537b665..c4a550ef0 100644 --- a/src/class/vendor/vendor_device.c +++ b/src/class/vendor/vendor_device.c @@ -21,6 +21,28 @@ typedef struct { uint8_t rhport; uint8_t itf_num; + #if CFG_TUD_VENDOR_EP_INT_OUT + uint8_t ep_int_out; + uint16_t int_rx_xfer_len; + #endif + #if CFG_TUD_VENDOR_EP_INT_IN + uint8_t ep_int_in; + #endif + #if CFG_TUD_VENDOR_EP_ISO_OUT + uint8_t ep_iso_out; + uint16_t iso_rx_xfer_len; + const tusb_desc_endpoint_t* iso_out_desc; // for deactivation on altsetting de-selection + #endif + #if CFG_TUD_VENDOR_EP_ISO_IN + uint8_t ep_iso_in; + const tusb_desc_endpoint_t* iso_in_desc; // for deactivation on altsetting de-selection + #endif + #if CFG_TUD_VENDOR_ALT_SETTINGS // implies non-buffered: fields cleared by bus reset + uint8_t cur_alt; + const uint8_t* p_itf_desc; // whole interface block incl. all altsettings (static app descriptor) + uint16_t itf_desc_len; + #endif + #if CFG_TUD_VENDOR_TXRX_BUFFERED /*------------- From this point, data is not cleared by bus reset -------------*/ tu_edpt_stream_t tx_stream; @@ -35,7 +57,10 @@ typedef struct { } vendord_interface_t; #if CFG_TUD_VENDOR_TXRX_BUFFERED - #define ITF_MEM_RESET_SIZE (offsetof(vendord_interface_t, itf_num) + TU_FIELD_SIZE(vendord_interface_t, itf_num)) + // The reset region is everything before the streams; tx_stream is the first preserved field + // (see the struct comment), so its offset is exactly that boundary regardless of which endpoint + // gates are enabled. + #define ITF_MEM_RESET_SIZE offsetof(vendord_interface_t, tx_stream) #else #define ITF_MEM_RESET_SIZE sizeof(vendord_interface_t) #endif @@ -52,6 +77,32 @@ typedef struct { CFG_TUD_MEM_SECTION static vendord_epbuf_t _vendord_epbuf[CFG_TUD_VENDOR]; #endif +#if CFG_TUD_VENDOR_EP_INT_OUT || CFG_TUD_VENDOR_EP_INT_IN +typedef struct { + #if CFG_TUD_VENDOR_EP_INT_OUT + TUD_EPBUF_DEF(int_out, CFG_TUD_VENDOR_EP_INT_OUT_BUFSIZE); + #endif + #if CFG_TUD_VENDOR_EP_INT_IN + TUD_EPBUF_DEF(int_in, CFG_TUD_VENDOR_EP_INT_IN_BUFSIZE); + #endif +} vendord_int_epbuf_t; + +CFG_TUD_MEM_SECTION static vendord_int_epbuf_t _vendord_int_epbuf[CFG_TUD_VENDOR]; +#endif + +#if CFG_TUD_VENDOR_EP_ISO_OUT || CFG_TUD_VENDOR_EP_ISO_IN +typedef struct { + #if CFG_TUD_VENDOR_EP_ISO_OUT + TUD_EPBUF_DEF(iso_out, CFG_TUD_VENDOR_EP_ISO_OUT_BUFSIZE); + #endif + #if CFG_TUD_VENDOR_EP_ISO_IN + TUD_EPBUF_DEF(iso_in, CFG_TUD_VENDOR_EP_ISO_IN_BUFSIZE); + #endif +} vendord_iso_epbuf_t; + +CFG_TUD_MEM_SECTION static vendord_iso_epbuf_t _vendord_iso_epbuf[CFG_TUD_VENDOR]; +#endif + //--------------------------------------------------------------------+ // Weak stubs: invoked if no strong implementation is available //--------------------------------------------------------------------+ @@ -66,15 +117,61 @@ TU_ATTR_WEAK void tud_vendor_tx_cb(uint8_t idx, uint32_t sent_bytes) { (void) sent_bytes; } +#if CFG_TUD_VENDOR_EP_INT_OUT +TU_ATTR_WEAK void tud_vendor_int_rx_cb(uint8_t idx, const uint8_t *buffer, uint32_t bufsize) { + (void)idx; + (void)buffer; + (void)bufsize; +} +#endif + +#if CFG_TUD_VENDOR_EP_INT_IN +TU_ATTR_WEAK void tud_vendor_int_tx_cb(uint8_t idx, uint32_t sent_bytes) { + (void)idx; + (void)sent_bytes; +} +#endif + +#if CFG_TUD_VENDOR_EP_ISO_OUT +TU_ATTR_WEAK void tud_vendor_iso_rx_cb(uint8_t idx, const uint8_t *buffer, uint32_t bufsize) { + (void)idx; + (void)buffer; + (void)bufsize; +} +#endif + +#if CFG_TUD_VENDOR_EP_ISO_IN +TU_ATTR_WEAK void tud_vendor_iso_tx_cb(uint8_t idx, uint32_t sent_bytes) { + (void)idx; + (void)sent_bytes; +} +#endif + bool tud_vendor_n_mounted(uint8_t idx) { TU_VERIFY(idx < CFG_TUD_VENDOR); vendord_interface_t *p_itf = &_vendord_itf[idx]; + // bulk may be absent (interrupt-only vendor interface): count the interrupt endpoints too #if CFG_TUD_VENDOR_TXRX_BUFFERED - return (p_itf->rx_stream.ep_addr != 0) || (p_itf->tx_stream.ep_addr != 0); + bool mounted = (p_itf->rx_stream.ep_addr != 0) || (p_itf->tx_stream.ep_addr != 0); #else - return (p_itf->ep_out != 0) || (p_itf->ep_in != 0); + bool mounted = (p_itf->ep_out != 0) || (p_itf->ep_in != 0); + #endif + #if CFG_TUD_VENDOR_EP_INT_OUT + mounted = mounted || (p_itf->ep_int_out != 0); #endif + #if CFG_TUD_VENDOR_EP_INT_IN + mounted = mounted || (p_itf->ep_int_in != 0); + #endif + // an altsetting may expose only isochronous endpoints; count them so apps that gate an iso + // pump on tud_vendor_mounted() still arm it + #if CFG_TUD_VENDOR_EP_ISO_OUT + mounted = mounted || (p_itf->ep_iso_out != 0); + #endif + #if CFG_TUD_VENDOR_EP_ISO_IN + mounted = mounted || (p_itf->ep_iso_in != 0); + #endif + return mounted; } //--------------------------------------------------------------------+ @@ -107,6 +204,30 @@ void tud_vendor_n_read_flush(uint8_t idx) { } #endif +// Shared non-buffered transfer helpers for the bulk / interrupt / isochronous endpoints, which are +// identical apart from the endpoint, its epbuf and its buffer size. TU_ATTR_UNUSED: in buffered +// mode with the int/iso gates off none is referenced, and clang/IAR error on an unused static. +TU_ATTR_UNUSED static inline uint32_t vendord_ep_write(vendord_interface_t *p_itf, uint8_t ep, uint8_t *epbuf, + uint32_t bufsize, const void *buffer, uint32_t len) { + TU_VERIFY(ep > 0, 0); // must be opened + TU_VERIFY(usbd_edpt_claim(p_itf->rhport, ep), 0); + const uint32_t xact_len = tu_min32(len, bufsize); + memcpy(epbuf, buffer, xact_len); + TU_ASSERT(usbd_edpt_xfer(p_itf->rhport, ep, epbuf, (uint16_t) xact_len, false), 0); + return xact_len; +} + +TU_ATTR_UNUSED static inline uint32_t vendord_ep_write_available(vendord_interface_t *p_itf, uint8_t ep, uint32_t bufsize) { + TU_VERIFY(ep > 0, 0); // must be opened + return usbd_edpt_busy(p_itf->rhport, ep) ? 0 : bufsize; +} + +TU_ATTR_UNUSED static inline bool vendord_ep_read_xfer(vendord_interface_t *p_itf, uint8_t ep, uint8_t *epbuf, uint16_t xfer_len) { + TU_VERIFY(ep > 0); // must be opened + TU_VERIFY(usbd_edpt_claim(p_itf->rhport, ep)); + return usbd_edpt_xfer(p_itf->rhport, ep, epbuf, xfer_len, false); +} + #if CFG_TUD_VENDOR_RX_MANUAL_XFER bool tud_vendor_n_read_xfer(uint8_t idx) { TU_VERIFY(idx < CFG_TUD_VENDOR); @@ -116,9 +237,8 @@ bool tud_vendor_n_read_xfer(uint8_t idx) { return tu_edpt_stream_read_xfer(&p_itf->rx_stream); #else - // Non-FIFO mode - TU_VERIFY(usbd_edpt_claim(p_itf->rhport, p_itf->ep_out)); - return usbd_edpt_xfer(p_itf->rhport, p_itf->ep_out, _vendord_epbuf[idx].epout, p_itf->rx_xfer_len, false); + // Non-FIFO mode (0 while an altsetting without a bulk OUT ep is active) + return vendord_ep_read_xfer(p_itf, p_itf->ep_out, _vendord_epbuf[idx].epout, p_itf->rx_xfer_len); #endif } #endif @@ -135,12 +255,8 @@ uint32_t tud_vendor_n_write(uint8_t idx, const void *buffer, uint32_t bufsize) { return tu_edpt_stream_write(&p_itf->tx_stream, buffer, (uint16_t)bufsize); #else - // non-fifo mode: direct transfer - TU_VERIFY(usbd_edpt_claim(p_itf->rhport, p_itf->ep_in), 0); - const uint32_t xact_len = tu_min32(bufsize, CFG_TUD_VENDOR_TX_EPSIZE); - memcpy(_vendord_epbuf[idx].epin, buffer, xact_len); - TU_ASSERT(usbd_edpt_xfer(p_itf->rhport, p_itf->ep_in, _vendord_epbuf[idx].epin, (uint16_t)xact_len, false), 0); - return xact_len; + // non-fifo mode: direct transfer (ep_in is 0 while an altsetting without a bulk IN ep is active) + return vendord_ep_write(p_itf, p_itf->ep_in, _vendord_epbuf[idx].epin, CFG_TUD_VENDOR_TX_EPSIZE, buffer, bufsize); #endif } @@ -152,9 +268,7 @@ uint32_t tud_vendor_n_write_available(uint8_t idx) { return tu_edpt_stream_write_available(&p_itf->tx_stream); #else - // Non-FIFO mode - TU_VERIFY(p_itf->ep_in > 0, 0); // must be opened - return usbd_edpt_busy(p_itf->rhport, p_itf->ep_in) ? 0 : CFG_TUD_VENDOR_TX_EPSIZE; + return vendord_ep_write_available(p_itf, p_itf->ep_in, CFG_TUD_VENDOR_TX_EPSIZE); #endif } @@ -174,6 +288,63 @@ bool tud_vendor_n_write_clear(uint8_t idx) { #endif //--------------------------------------------------------------------+ +// Interrupt endpoint API +//--------------------------------------------------------------------+ +#if CFG_TUD_VENDOR_EP_INT_OUT +bool tud_vendor_n_int_read_xfer(uint8_t idx) { + TU_VERIFY(idx < CFG_TUD_VENDOR); + vendord_interface_t *p_itf = &_vendord_itf[idx]; + return vendord_ep_read_xfer(p_itf, p_itf->ep_int_out, _vendord_int_epbuf[idx].int_out, p_itf->int_rx_xfer_len); +} +#endif + +#if CFG_TUD_VENDOR_EP_INT_IN +uint32_t tud_vendor_n_int_write(uint8_t idx, const void *buffer, uint32_t bufsize) { + TU_VERIFY(idx < CFG_TUD_VENDOR, 0); + vendord_interface_t *p_itf = &_vendord_itf[idx]; + return vendord_ep_write(p_itf, p_itf->ep_int_in, _vendord_int_epbuf[idx].int_in, CFG_TUD_VENDOR_EP_INT_IN_BUFSIZE, buffer, bufsize); +} + +uint32_t tud_vendor_n_int_write_available(uint8_t idx) { + TU_VERIFY(idx < CFG_TUD_VENDOR, 0); + vendord_interface_t *p_itf = &_vendord_itf[idx]; + return vendord_ep_write_available(p_itf, p_itf->ep_int_in, CFG_TUD_VENDOR_EP_INT_IN_BUFSIZE); +} +#endif + +//--------------------------------------------------------------------+ +// Isochronous endpoint API +//--------------------------------------------------------------------+ +#if CFG_TUD_VENDOR_EP_ISO_OUT +bool tud_vendor_n_iso_read_xfer(uint8_t idx) { + TU_VERIFY(idx < CFG_TUD_VENDOR); + vendord_interface_t *p_itf = &_vendord_itf[idx]; + return vendord_ep_read_xfer(p_itf, p_itf->ep_iso_out, _vendord_iso_epbuf[idx].iso_out, p_itf->iso_rx_xfer_len); +} +#endif + +#if CFG_TUD_VENDOR_EP_ISO_IN +uint32_t tud_vendor_n_iso_write(uint8_t idx, const void *buffer, uint32_t bufsize) { + TU_VERIFY(idx < CFG_TUD_VENDOR, 0); + vendord_interface_t *p_itf = &_vendord_itf[idx]; + return vendord_ep_write(p_itf, p_itf->ep_iso_in, _vendord_iso_epbuf[idx].iso_in, CFG_TUD_VENDOR_EP_ISO_IN_BUFSIZE, buffer, bufsize); +} + +uint32_t tud_vendor_n_iso_write_available(uint8_t idx) { + TU_VERIFY(idx < CFG_TUD_VENDOR, 0); + vendord_interface_t *p_itf = &_vendord_itf[idx]; + return vendord_ep_write_available(p_itf, p_itf->ep_iso_in, CFG_TUD_VENDOR_EP_ISO_IN_BUFSIZE); +} +#endif + +#if CFG_TUD_VENDOR_ALT_SETTINGS +uint8_t tud_vendor_n_alt(uint8_t idx) { + TU_VERIFY(idx < CFG_TUD_VENDOR, 0); + return _vendord_itf[idx].cur_alt; +} +#endif + +//--------------------------------------------------------------------+ // USBD Driver API //--------------------------------------------------------------------+ void vendord_init(void) { @@ -232,17 +403,61 @@ static uint8_t find_vendor_itf(uint8_t ep_addr) { for (uint8_t idx = 0; idx < CFG_TUD_VENDOR; idx++) { const vendord_interface_t *p_vendor = &_vendord_itf[idx]; if (ep_addr == 0) { - // find unused: require both ep == 0 - #if CFG_TUD_VENDOR_TXRX_BUFFERED - if (p_vendor->rx_stream.ep_addr == 0 && p_vendor->tx_stream.ep_addr == 0) { + // find unused interface slot + #if CFG_TUD_VENDOR_ALT_SETTINGS + // an opened interface parked in an altsetting without endpoints (the mandatory empty alt 0) + // has all ep fields 0, so the endpoint fields cannot distinguish free from open: use p_itf_desc + if (p_vendor->p_itf_desc == NULL) { + return idx; + } + #elif CFG_TUD_VENDOR_TXRX_BUFFERED + // A slot is free only if none of its endpoints are assigned; bulk may be absent + // (an interrupt-only vendor interface), so check the interrupt endpoints too. + if (p_vendor->rx_stream.ep_addr == 0 && p_vendor->tx_stream.ep_addr == 0 + #if CFG_TUD_VENDOR_EP_INT_OUT + && p_vendor->ep_int_out == 0 + #endif + #if CFG_TUD_VENDOR_EP_INT_IN + && p_vendor->ep_int_in == 0 + #endif + ) { return idx; } #else - if (p_vendor->ep_out == 0 && p_vendor->ep_in == 0) { + // A slot is free only if none of its endpoints are assigned. Bulk may be absent (an + // interrupt-only vendor interface), so the interrupt endpoints must be checked too. + if (p_vendor->ep_out == 0 && p_vendor->ep_in == 0 + #if CFG_TUD_VENDOR_EP_INT_OUT + && p_vendor->ep_int_out == 0 + #endif + #if CFG_TUD_VENDOR_EP_INT_IN + && p_vendor->ep_int_in == 0 + #endif + ) { return idx; } #endif } else { + #if CFG_TUD_VENDOR_EP_INT_OUT + if (ep_addr == p_vendor->ep_int_out) { + return idx; + } + #endif + #if CFG_TUD_VENDOR_EP_INT_IN + if (ep_addr == p_vendor->ep_int_in) { + return idx; + } + #endif + #if CFG_TUD_VENDOR_EP_ISO_OUT + if (ep_addr == p_vendor->ep_iso_out) { + return idx; + } + #endif + #if CFG_TUD_VENDOR_EP_ISO_IN + if (ep_addr == p_vendor->ep_iso_in) { + return idx; + } + #endif #if CFG_TUD_VENDOR_TXRX_BUFFERED if (ep_addr == p_vendor->rx_stream.ep_addr || ep_addr == p_vendor->tx_stream.ep_addr) { return idx; @@ -257,6 +472,287 @@ static uint8_t find_vendor_itf(uint8_t ep_addr) { return 0xff; } +#if CFG_TUD_VENDOR_ALT_SETTINGS + +// Reserve an isochronous endpoint at open time. Ports with a dedicated iso allocator +// (TUP_DCD_EDPT_ISO_ALLOC) reserve the FIFO here and (re)activate on altsetting selection; +// ports with dcd_edpt_close instead open it once here (open == allocate + activate). +static inline bool vendord_iso_ep_alloc(uint8_t rhport, const tusb_desc_endpoint_t* desc_ep) { + #ifdef TUP_DCD_EDPT_ISO_ALLOC + return usbd_edpt_iso_alloc(rhport, desc_ep->bEndpointAddress, tu_edpt_packet_size(desc_ep)); + #else + return usbd_edpt_open(rhport, desc_ep); + #endif +} + +// Deactivate a de-selected altsetting's isochronous endpoint: abort any in-flight +// transfer and release its usbd claim so a stale completion cannot fire into a +// no-longer-tracked endpoint (iso cannot be stalled like bulk/interrupt below). With the +// iso-alloc API, (re)activation with the endpoint's descriptor is the abort/scrub +// primitive; without it, close does (the next selection re-opens). +static inline void vendord_iso_ep_deactivate(uint8_t rhport, const tusb_desc_endpoint_t* desc_ep) { + if (desc_ep != NULL) { + #ifdef TUP_DCD_EDPT_ISO_ALLOC + usbd_edpt_iso_activate(rhport, desc_ep); + #else + usbd_edpt_close(rhport, desc_ep->bEndpointAddress); + #endif + } +} + +// (Re)activate an isochronous endpoint on altsetting selection. +static inline bool vendord_iso_ep_activate(uint8_t rhport, const tusb_desc_endpoint_t* desc_ep) { + #ifdef TUP_DCD_EDPT_ISO_ALLOC + return usbd_edpt_iso_activate(rhport, desc_ep); // resets ep_status, aborting any stale transfer + #else + // No iso alloc/activate API: close (which zeros ep_status and frees a stale claim left by a + // prior selection) then re-open, so a re-selected altsetting starts from a clean state. + usbd_edpt_close(rhport, desc_ep->bEndpointAddress); + return usbd_edpt_open(rhport, desc_ep); + #endif +} + +// Abort any in-flight transfer on a tracked endpoint and release its usbd claim; no-op for an +// unset (0) address. stall disables the endpoint in the dcd, clear-stall resets it to DATA0. +static inline void vendord_abort_ep(uint8_t rhport, uint8_t ep_addr) { + if (ep_addr) { + usbd_edpt_stall(rhport, ep_addr); + usbd_edpt_clear_stall(rhport, ep_addr); + } +} + +// Select an altsetting. Endpoints were hardware-opened once at vendord_open (dcds like +// dwc2 allocate FIFO linearly and cannot close/re-open endpoints dynamically): switching +// only re-targets the API to the selected altsetting's endpoints. Bulk/interrupt +// endpoints get a stall/clear-stall cycle, which portably aborts any in-flight transfer +// (stall disables the endpoint in the dcd) and resets the data toggle to DATA0 as +// SET_INTERFACE requires. Isochronous endpoints are (re)activated, which does the same. +// Single pass: the current altsetting's endpoints are dropped only once the target altsetting +// is confirmed present, so a SET_INTERFACE to an unknown alt leaves the interface intact. +static bool vendord_set_alt(uint8_t rhport, uint8_t idx, uint8_t alt) { + vendord_interface_t *p_vendor = &_vendord_itf[idx]; + const uint8_t* p_desc = p_vendor->p_itf_desc; + const uint8_t* desc_end = p_desc + p_vendor->itf_desc_len; + bool in_target_alt = false; + bool alt_found = false; + + while (tu_desc_in_bounds(p_desc, desc_end)) { + const uint8_t desc_type = tu_desc_type(p_desc); + if (desc_type == TUSB_DESC_INTERFACE) { + in_target_alt = (((const tusb_desc_interface_t*)p_desc)->bAlternateSetting == alt); + if (in_target_alt && !alt_found) { + alt_found = true; + // target altsetting confirmed present: abort then drop the previous altsetting's endpoints, + // so an endpoint absent from the target altsetting can't stay armed and keep its usbd + // claim in the dcd. (Endpoints the target altsetting reuses are reset again below; + // a double reset is harmless.) + vendord_abort_ep(rhport, p_vendor->ep_in); + vendord_abort_ep(rhport, p_vendor->ep_out); + #if CFG_TUD_VENDOR_EP_INT_OUT + vendord_abort_ep(rhport, p_vendor->ep_int_out); + #endif + #if CFG_TUD_VENDOR_EP_INT_IN + vendord_abort_ep(rhport, p_vendor->ep_int_in); + #endif + p_vendor->ep_in = 0; + p_vendor->ep_out = 0; + #if CFG_TUD_VENDOR_EP_INT_OUT + p_vendor->ep_int_out = 0; + #endif + #if CFG_TUD_VENDOR_EP_INT_IN + p_vendor->ep_int_in = 0; + #endif + #if CFG_TUD_VENDOR_EP_ISO_OUT + vendord_iso_ep_deactivate(rhport, p_vendor->iso_out_desc); + p_vendor->iso_out_desc = NULL; + p_vendor->ep_iso_out = 0; + #endif + #if CFG_TUD_VENDOR_EP_ISO_IN + vendord_iso_ep_deactivate(rhport, p_vendor->iso_in_desc); + p_vendor->iso_in_desc = NULL; + p_vendor->ep_iso_in = 0; + #endif + } + } else if (in_target_alt && desc_type == TUSB_DESC_ENDPOINT) { + const tusb_desc_endpoint_t* desc_ep = (const tusb_desc_endpoint_t*) p_desc; + const uint8_t ep_addr = desc_ep->bEndpointAddress; + const bool is_in = tu_edpt_dir(ep_addr) == TUSB_DIR_IN; + (void) is_in; + + switch (desc_ep->bmAttributes.xfer) { + case TUSB_XFER_BULK: + // abort in-flight transfer + reset data toggle + usbd_edpt_stall(rhport, ep_addr); + usbd_edpt_clear_stall(rhport, ep_addr); + if (is_in) { + p_vendor->ep_in = ep_addr; + } else { + p_vendor->ep_out = ep_addr; + p_vendor->rx_xfer_len = + CFG_TUD_VENDOR_RX_NEED_ZLP ? CFG_TUD_VENDOR_RX_EPSIZE : tu_edpt_packet_size(desc_ep); + #if CFG_TUD_VENDOR_RX_MANUAL_XFER == 0 + TU_ASSERT(usbd_edpt_xfer(rhport, p_vendor->ep_out, _vendord_epbuf[idx].epout, + p_vendor->rx_xfer_len, false)); + #endif + } + break; + + #if CFG_TUD_VENDOR_EP_INT_IN || CFG_TUD_VENDOR_EP_INT_OUT + case TUSB_XFER_INTERRUPT: + // stall/clear only for the enabled direction (an endpoint of a disabled + // direction was never opened, so must not be poked in the dcd) + #if CFG_TUD_VENDOR_EP_INT_IN + if (is_in) { + usbd_edpt_stall(rhport, ep_addr); + usbd_edpt_clear_stall(rhport, ep_addr); + p_vendor->ep_int_in = ep_addr; + } + #endif + #if CFG_TUD_VENDOR_EP_INT_OUT + if (!is_in) { + usbd_edpt_stall(rhport, ep_addr); + usbd_edpt_clear_stall(rhport, ep_addr); + p_vendor->ep_int_out = ep_addr; + p_vendor->int_rx_xfer_len = tu_edpt_packet_size(desc_ep); + } + #endif + break; + #endif + + #if CFG_TUD_VENDOR_EP_ISO_IN || CFG_TUD_VENDOR_EP_ISO_OUT + case TUSB_XFER_ISOCHRONOUS: + #if CFG_TUD_VENDOR_EP_ISO_IN + if (is_in) { + TU_ASSERT(vendord_iso_ep_activate(rhport, desc_ep)); + p_vendor->ep_iso_in = ep_addr; + p_vendor->iso_in_desc = desc_ep; // points into p_itf_desc (static app descriptor) + } + #endif + #if CFG_TUD_VENDOR_EP_ISO_OUT + if (!is_in) { + TU_ASSERT(vendord_iso_ep_activate(rhport, desc_ep)); + p_vendor->ep_iso_out = ep_addr; + p_vendor->iso_out_desc = desc_ep; // points into p_itf_desc (static app descriptor) + p_vendor->iso_rx_xfer_len = tu_edpt_packet_size(desc_ep); + } + #endif + break; + #endif + + default: + break; // unsupported endpoint type / direction gate disabled: ignore + } + } + p_desc = tu_desc_next(p_desc); + } + + TU_VERIFY(alt_found); // unknown alt: endpoints were never cleared, current altsetting intact + p_vendor->cur_alt = alt; + return true; +} + +uint16_t vendord_open(uint8_t rhport, const tusb_desc_interface_t *desc_itf, uint16_t max_len) { + TU_VERIFY(TUSB_CLASS_VENDOR_SPECIFIC == desc_itf->bInterfaceClass, 0); + const uint8_t* desc_end = (const uint8_t*)desc_itf + max_len; + + const uint8_t idx = find_vendor_itf(0); + TU_ASSERT(idx < CFG_TUD_VENDOR, 0); + vendord_interface_t *p_vendor = &_vendord_itf[idx]; + p_vendor->rhport = rhport; + p_vendor->itf_num = desc_itf->bInterfaceNumber; + // p_itf_desc is assigned only after the parse succeeds: find_vendor_itf() treats a non-NULL + // p_itf_desc as an occupied slot, so setting it before a mid-parse TU_ASSERT could fail would + // leak the slot (a retry would find no free interface until the next bus reset). + + // Consume every altsetting of this interface and hardware-open each endpoint exactly once + // (bulk/interrupt via usbd_edpt_open, isochronous FIFO-allocated; iso activation and the + // toggle reset happen on altsetting selection). An endpoint address that recurs in another + // altsetting must carry an identical configuration, since it is opened only on first sight; + // reconfiguring the same address per-alt is not supported and is rejected here. + uint8_t seen_type[CFG_TUD_ENDPPOINT_MAX][2]; + uint16_t seen_mps[CFG_TUD_ENDPPOINT_MAX][2]; + tu_memclr(seen_type, sizeof(seen_type)); // 0 == TUSB_XFER_CONTROL, never used as a data ep here + const uint8_t* p_desc = tu_desc_next(desc_itf); + while (tu_desc_in_bounds(p_desc, desc_end)) { + const uint8_t desc_type = tu_desc_type(p_desc); + if (desc_type == TUSB_DESC_INTERFACE_ASSOCIATION) { + break; + } + if (desc_type == TUSB_DESC_INTERFACE) { + if (((const tusb_desc_interface_t*)p_desc)->bInterfaceNumber != p_vendor->itf_num) { + break; // next interface + } + } else if (desc_type == TUSB_DESC_ENDPOINT) { + const tusb_desc_endpoint_t* desc_ep = (const tusb_desc_endpoint_t*) p_desc; + const uint8_t epnum = tu_edpt_number(desc_ep->bEndpointAddress); + const uint8_t dir = tu_edpt_dir(desc_ep->bEndpointAddress); + const uint8_t xfer = desc_ep->bmAttributes.xfer; + const uint16_t mps = tu_edpt_packet_size(desc_ep); + TU_ASSERT(epnum < CFG_TUD_ENDPPOINT_MAX, 0); + + if (seen_type[epnum][dir] != 0) { + // reused address in a later altsetting: must be an exact match (opened only once) + TU_ASSERT(seen_type[epnum][dir] == xfer && seen_mps[epnum][dir] == mps, 0); + } else { + switch (xfer) { + case TUSB_XFER_BULK: + TU_ASSERT(usbd_edpt_open(rhport, desc_ep), 0); + break; + + #if CFG_TUD_VENDOR_EP_INT_IN || CFG_TUD_VENDOR_EP_INT_OUT + case TUSB_XFER_INTERRUPT: + #if CFG_TUD_VENDOR_EP_INT_IN + if (dir == TUSB_DIR_IN) { + TU_ASSERT(mps <= CFG_TUD_VENDOR_EP_INT_IN_BUFSIZE, 0); + TU_ASSERT(usbd_edpt_open(rhport, desc_ep), 0); + } + #endif + #if CFG_TUD_VENDOR_EP_INT_OUT + if (dir == TUSB_DIR_OUT) { + TU_ASSERT(mps <= CFG_TUD_VENDOR_EP_INT_OUT_BUFSIZE, 0); + TU_ASSERT(usbd_edpt_open(rhport, desc_ep), 0); + } + #endif + break; + #endif + + #if CFG_TUD_VENDOR_EP_ISO_IN || CFG_TUD_VENDOR_EP_ISO_OUT + case TUSB_XFER_ISOCHRONOUS: + #if CFG_TUD_VENDOR_EP_ISO_IN + if (dir == TUSB_DIR_IN) { + TU_ASSERT(mps <= CFG_TUD_VENDOR_EP_ISO_IN_BUFSIZE, 0); + TU_ASSERT(vendord_iso_ep_alloc(rhport, desc_ep), 0); + } + #endif + #if CFG_TUD_VENDOR_EP_ISO_OUT + if (dir == TUSB_DIR_OUT) { + TU_ASSERT(mps <= CFG_TUD_VENDOR_EP_ISO_OUT_BUFSIZE, 0); + TU_ASSERT(vendord_iso_ep_alloc(rhport, desc_ep), 0); + } + #endif + break; + #endif + + default: + break; // unsupported endpoint type / direction gate disabled: ignore + } + seen_type[epnum][dir] = xfer; + seen_mps[epnum][dir] = mps; + } + } + p_desc = tu_desc_next(p_desc); + } + // parse succeeded: commit the descriptor pointer (marks the slot occupied) before selecting alt 0 + p_vendor->p_itf_desc = (const uint8_t*) desc_itf; + p_vendor->itf_desc_len = (uint16_t)((uintptr_t)p_desc - (uintptr_t)desc_itf); + + // default altsetting active until the host selects another + TU_ASSERT(vendord_set_alt(rhport, idx, 0), 0); + return p_vendor->itf_desc_len; +} + +#else // !CFG_TUD_VENDOR_ALT_SETTINGS + uint16_t vendord_open(uint8_t rhport, const tusb_desc_interface_t *desc_itf, uint16_t max_len) { TU_VERIFY(TUSB_CLASS_VENDOR_SPECIFIC == desc_itf->bInterfaceClass, 0); const uint8_t* desc_end = (const uint8_t*)desc_itf + max_len; @@ -275,6 +771,31 @@ uint16_t vendord_open(uint8_t rhport, const tusb_desc_interface_t *desc_itf, uin break; // end of this interface } else if (desc_type == TUSB_DESC_ENDPOINT) { const tusb_desc_endpoint_t* desc_ep = (const tusb_desc_endpoint_t*) p_desc; + + #if CFG_TUD_VENDOR_EP_INT_OUT || CFG_TUD_VENDOR_EP_INT_IN + if (desc_ep->bmAttributes.xfer == TUSB_XFER_INTERRUPT) { + const bool is_int_in = tu_edpt_dir(desc_ep->bEndpointAddress) == TUSB_DIR_IN; + (void) is_int_in; + #if CFG_TUD_VENDOR_EP_INT_IN + if (is_int_in) { + TU_ASSERT(tu_edpt_packet_size(desc_ep) <= CFG_TUD_VENDOR_EP_INT_IN_BUFSIZE, 0); + TU_ASSERT(usbd_edpt_open(rhport, desc_ep)); + p_vendor->ep_int_in = desc_ep->bEndpointAddress; + } + #endif + #if CFG_TUD_VENDOR_EP_INT_OUT + if (!is_int_in) { + TU_ASSERT(tu_edpt_packet_size(desc_ep) <= CFG_TUD_VENDOR_EP_INT_OUT_BUFSIZE, 0); + TU_ASSERT(usbd_edpt_open(rhport, desc_ep)); + p_vendor->ep_int_out = desc_ep->bEndpointAddress; + p_vendor->int_rx_xfer_len = tu_edpt_packet_size(desc_ep); + } + #endif + p_desc = tu_desc_next(p_desc); + continue; + } + #endif + TU_ASSERT(usbd_edpt_open(rhport, desc_ep)); uint16_t rx_xfer_len = CFG_TUD_VENDOR_RX_NEED_ZLP ? CFG_TUD_VENDOR_RX_EPSIZE : tu_edpt_packet_size(desc_ep); @@ -313,6 +834,40 @@ uint16_t vendord_open(uint8_t rhport, const tusb_desc_interface_t *desc_itf, uin return (uint16_t)((uintptr_t)p_desc - (uintptr_t)desc_itf); } +#endif // CFG_TUD_VENDOR_ALT_SETTINGS + +// Handle interface standard requests (GET/SET_INTERFACE when altsettings are enabled), +// delegate everything else to the application callback as before. +bool vendord_control_xfer_cb(uint8_t rhport, uint8_t stage, tusb_control_request_t const *request) { +#if CFG_TUD_VENDOR_ALT_SETTINGS + if (request->bmRequestType_bit.type == TUSB_REQ_TYPE_STANDARD && + request->bmRequestType_bit.recipient == TUSB_REQ_RCPT_INTERFACE) { + const uint8_t itf_num = tu_u16_low(request->wIndex); + uint8_t idx; + for (idx = 0; idx < CFG_TUD_VENDOR; idx++) { + if (_vendord_itf[idx].itf_num == itf_num && _vendord_itf[idx].p_itf_desc != NULL) { + break; + } + } + if (idx < CFG_TUD_VENDOR) { + if (request->bRequest == TUSB_REQ_SET_INTERFACE) { + if (stage == CONTROL_STAGE_SETUP) { + TU_VERIFY(vendord_set_alt(rhport, idx, tu_u16_low(request->wValue))); + return tud_control_status(rhport, request); + } + return true; + } else if (request->bRequest == TUSB_REQ_GET_INTERFACE) { + if (stage == CONTROL_STAGE_SETUP) { + return tud_control_xfer(rhport, request, &_vendord_itf[idx].cur_alt, 1); + } + return true; + } + } + } +#endif + return tud_vendor_control_xfer_cb(rhport, stage, request); +} + bool vendord_xfer_cb(uint8_t rhport, uint8_t ep_addr, xfer_result_t result, uint32_t xferred_bytes) { (void)rhport; (void)result; @@ -320,6 +875,33 @@ bool vendord_xfer_cb(uint8_t rhport, uint8_t ep_addr, xfer_result_t result, uint TU_VERIFY(idx < CFG_TUD_VENDOR); vendord_interface_t *p_vendor = &_vendord_itf[idx]; +#if CFG_TUD_VENDOR_EP_INT_OUT + if (ep_addr == p_vendor->ep_int_out) { + // not re-armed automatically: application calls tud_vendor_n_int_read_xfer() + tud_vendor_int_rx_cb(idx, _vendord_int_epbuf[idx].int_out, xferred_bytes); + return true; + } +#endif +#if CFG_TUD_VENDOR_EP_INT_IN + if (ep_addr == p_vendor->ep_int_in) { + tud_vendor_int_tx_cb(idx, xferred_bytes); + return true; + } +#endif +#if CFG_TUD_VENDOR_EP_ISO_OUT + if (ep_addr == p_vendor->ep_iso_out) { + // not re-armed automatically: application calls tud_vendor_n_iso_read_xfer() + tud_vendor_iso_rx_cb(idx, _vendord_iso_epbuf[idx].iso_out, xferred_bytes); + return true; + } +#endif +#if CFG_TUD_VENDOR_EP_ISO_IN + if (ep_addr == p_vendor->ep_iso_in) { + tud_vendor_iso_tx_cb(idx, xferred_bytes); + return true; + } +#endif + #if CFG_TUD_VENDOR_TXRX_BUFFERED if (ep_addr == p_vendor->rx_stream.ep_addr) { // Put received data to FIFO diff --git a/src/class/vendor/vendor_device.h b/src/class/vendor/vendor_device.h index 32216f25f..ce33e2f62 100644 --- a/src/class/vendor/vendor_device.h +++ b/src/class/vendor/vendor_device.h @@ -60,6 +60,68 @@ extern "C" { #define CFG_TUD_VENDOR_RX_NEED_ZLP 0 #endif +// Enable support for an optional interrupt OUT / interrupt IN endpoint in the vendor +// interface, each direction gated separately. Interrupt endpoints are non-buffered: +// OUT is armed manually one packet at a time with tud_vendor_n_int_read_xfer() (data +// delivered via tud_vendor_int_rx_cb), IN is a direct transfer via tud_vendor_n_int_write(). +#ifndef CFG_TUD_VENDOR_EP_INT_OUT + #define CFG_TUD_VENDOR_EP_INT_OUT 0 +#endif + +#ifndef CFG_TUD_VENDOR_EP_INT_IN + #define CFG_TUD_VENDOR_EP_INT_IN 0 +#endif + +// Buffer sizes for interrupt endpoint transfers, must be >= the endpoint max packet size +#ifndef CFG_TUD_VENDOR_EP_INT_OUT_BUFSIZE + #define CFG_TUD_VENDOR_EP_INT_OUT_BUFSIZE 64 +#endif + +#ifndef CFG_TUD_VENDOR_EP_INT_IN_BUFSIZE + #define CFG_TUD_VENDOR_EP_INT_IN_BUFSIZE 64 +#endif + +// Enable support for an optional isochronous OUT / IN endpoint, each direction gated +// separately, with the same non-buffered API shape as the interrupt pair. Isochronous +// endpoints must not claim bandwidth in the default altsetting (USB 2.0 5.6.3): place +// them in a non-zero altsetting and enable CFG_TUD_VENDOR_ALT_SETTINGS. +#ifndef CFG_TUD_VENDOR_EP_ISO_OUT + #define CFG_TUD_VENDOR_EP_ISO_OUT 0 +#endif + +#ifndef CFG_TUD_VENDOR_EP_ISO_IN + #define CFG_TUD_VENDOR_EP_ISO_IN 0 +#endif + +// Buffer sizes for isochronous endpoint transfers, must be >= the endpoint max packet size +#ifndef CFG_TUD_VENDOR_EP_ISO_OUT_BUFSIZE + #define CFG_TUD_VENDOR_EP_ISO_OUT_BUFSIZE 64 +#endif + +#ifndef CFG_TUD_VENDOR_EP_ISO_IN_BUFSIZE + #define CFG_TUD_VENDOR_EP_ISO_IN_BUFSIZE 64 +#endif + +// Enable alternate-setting support: the vendor interface may carry multiple altsettings, +// each with its own endpoint set. GET_INTERFACE is answered and SET_INTERFACE performed by +// closing the current altsetting's endpoints and opening the requested one's (isochronous +// endpoints are FIFO-allocated at open and activated on selection). The configuration +// descriptor must stay valid while mounted (static, the usual TinyUSB pattern). +// Non-buffered mode only. +#ifndef CFG_TUD_VENDOR_ALT_SETTINGS + #define CFG_TUD_VENDOR_ALT_SETTINGS 0 +#endif + +#if CFG_TUD_VENDOR_ALT_SETTINGS && CFG_TUD_VENDOR_TXRX_BUFFERED + #error CFG_TUD_VENDOR_ALT_SETTINGS requires non-buffered mode (CFG_TUD_VENDOR_RX/TX_BUFSIZE = 0) +#endif + +// An isochronous endpoint must not claim bandwidth in the default altsetting (USB 2.0 5.6.3), +// so it can only live in a non-zero altsetting, which requires alternate-setting support. +#if (CFG_TUD_VENDOR_EP_ISO_OUT || CFG_TUD_VENDOR_EP_ISO_IN) && !CFG_TUD_VENDOR_ALT_SETTINGS + #error CFG_TUD_VENDOR_EP_ISO_OUT/IN requires CFG_TUD_VENDOR_ALT_SETTINGS +#endif + //--------------------------------------------------------------------+ // Application API (Multiple Interfaces) i.e CFG_TUD_VENDOR > 1 //--------------------------------------------------------------------+ @@ -107,6 +169,43 @@ TU_ATTR_ALWAYS_INLINE static inline uint32_t tud_vendor_n_write_str(uint8_t idx, return tud_vendor_n_write(idx, str, strlen(str)); } +//------------- Interrupt endpoints -------------// +#if CFG_TUD_VENDOR_EP_INT_OUT +// Arm the interrupt OUT endpoint for one packet, return false if a transfer is still ongoing. +// Received data is delivered via tud_vendor_int_rx_cb(); re-arm from the callback or by polling. +bool tud_vendor_n_int_read_xfer(uint8_t idx); +#endif + +#if CFG_TUD_VENDOR_EP_INT_IN +// Send on the interrupt IN endpoint (direct transfer, up to CFG_TUD_VENDOR_EP_INT_IN_BUFSIZE +// bytes). Returns number of bytes queued, 0 if the endpoint is busy or not opened. +uint32_t tud_vendor_n_int_write(uint8_t idx, const void *buffer, uint32_t bufsize); + +// Return available bytes for interrupt IN write: 0 while busy, else the buffer size +uint32_t tud_vendor_n_int_write_available(uint8_t idx); +#endif + +//------------- Isochronous endpoints -------------// +#if CFG_TUD_VENDOR_EP_ISO_OUT +// Arm the isochronous OUT endpoint for one packet, return false if a transfer is still ongoing. +// Received data is delivered via tud_vendor_iso_rx_cb(); re-arm from the callback or by polling. +bool tud_vendor_n_iso_read_xfer(uint8_t idx); +#endif + +#if CFG_TUD_VENDOR_EP_ISO_IN +// Send on the isochronous IN endpoint (direct transfer, up to CFG_TUD_VENDOR_EP_ISO_IN_BUFSIZE +// bytes). Returns number of bytes queued, 0 if the endpoint is busy or not opened. +uint32_t tud_vendor_n_iso_write(uint8_t idx, const void *buffer, uint32_t bufsize); + +// Return available bytes for isochronous IN write: 0 while busy, else the buffer size +uint32_t tud_vendor_n_iso_write_available(uint8_t idx); +#endif + +#if CFG_TUD_VENDOR_ALT_SETTINGS +// Return the currently selected alternate setting +uint8_t tud_vendor_n_alt(uint8_t idx); +#endif + // backward compatible #define tud_vendor_n_flush(idx) tud_vendor_n_write_flush(idx) @@ -161,6 +260,44 @@ TU_ATTR_ALWAYS_INLINE static inline uint32_t tud_vendor_write_available(void) { return tud_vendor_n_write_available(0); } +#if CFG_TUD_VENDOR_EP_INT_OUT +TU_ATTR_ALWAYS_INLINE static inline bool tud_vendor_int_read_xfer(void) { + return tud_vendor_n_int_read_xfer(0); +} +#endif + +#if CFG_TUD_VENDOR_EP_INT_IN +TU_ATTR_ALWAYS_INLINE static inline uint32_t tud_vendor_int_write(const void *buffer, uint32_t bufsize) { + return tud_vendor_n_int_write(0, buffer, bufsize); +} + +TU_ATTR_ALWAYS_INLINE static inline uint32_t tud_vendor_int_write_available(void) { + return tud_vendor_n_int_write_available(0); +} +#endif + +#if CFG_TUD_VENDOR_EP_ISO_OUT +TU_ATTR_ALWAYS_INLINE static inline bool tud_vendor_iso_read_xfer(void) { + return tud_vendor_n_iso_read_xfer(0); +} +#endif + +#if CFG_TUD_VENDOR_EP_ISO_IN +TU_ATTR_ALWAYS_INLINE static inline uint32_t tud_vendor_iso_write(const void *buffer, uint32_t bufsize) { + return tud_vendor_n_iso_write(0, buffer, bufsize); +} + +TU_ATTR_ALWAYS_INLINE static inline uint32_t tud_vendor_iso_write_available(void) { + return tud_vendor_n_iso_write_available(0); +} +#endif + +#if CFG_TUD_VENDOR_ALT_SETTINGS +TU_ATTR_ALWAYS_INLINE static inline uint8_t tud_vendor_alt(void) { + return tud_vendor_n_alt(0); +} +#endif + // backward compatible #define tud_vendor_flush() tud_vendor_write_flush() @@ -176,6 +313,29 @@ void tud_vendor_rx_cb(uint8_t idx, const uint8_t *buffer, uint32_t bufsize); // Invoked when tx transfer is finished void tud_vendor_tx_cb(uint8_t idx, uint32_t sent_bytes); +#if CFG_TUD_VENDOR_EP_INT_OUT +// Invoked when data is received on the interrupt OUT endpoint. The endpoint is not +// re-armed automatically: call tud_vendor_n_int_read_xfer() to receive more. +void tud_vendor_int_rx_cb(uint8_t idx, const uint8_t *buffer, uint32_t bufsize); +#endif + +#if CFG_TUD_VENDOR_EP_INT_IN +// Invoked when an interrupt IN transfer is finished +void tud_vendor_int_tx_cb(uint8_t idx, uint32_t sent_bytes); +#endif + +#if CFG_TUD_VENDOR_EP_ISO_OUT +// Invoked when data is received on the isochronous OUT endpoint. The endpoint is not +// re-armed automatically: call tud_vendor_n_iso_read_xfer() to receive more. +void tud_vendor_iso_rx_cb(uint8_t idx, const uint8_t *buffer, uint32_t bufsize); +#endif + +#if CFG_TUD_VENDOR_EP_ISO_IN +// Invoked when an isochronous IN transfer is finished (result may be a missed frame: +// the data was not necessarily taken by the host, re-arm regardless) +void tud_vendor_iso_tx_cb(uint8_t idx, uint32_t sent_bytes); +#endif + //--------------------------------------------------------------------+ // Internal Class Driver API //--------------------------------------------------------------------+ @@ -183,6 +343,7 @@ void vendord_init(void); bool vendord_deinit(void); void vendord_reset(uint8_t rhport); uint16_t vendord_open(uint8_t rhport, const tusb_desc_interface_t *idx_desc, uint16_t max_len); +bool vendord_control_xfer_cb(uint8_t rhport, uint8_t stage, tusb_control_request_t const *request); bool vendord_xfer_cb(uint8_t rhport, uint8_t ep_addr, xfer_result_t event, uint32_t xferred_bytes); #ifdef __cplusplus diff --git a/src/class/video/video_device.c b/src/class/video/video_device.c index 3797e6b2b..770595178 100644 --- a/src/class/video/video_device.c +++ b/src/class/video/video_device.c @@ -1144,6 +1144,9 @@ static int handle_video_stm_cs_req(uint8_t rhport, uint8_t stage, video_probe_and_commit_control_t *param = &stm->probe_commit_payload; TU_VERIFY(_update_streaming_parameters(stm, param), VIDEO_ERROR_INVALID_VALUE_WITHIN_RANGE); /* Set the negotiated value */ + if (CFG_TUD_VIDEO_STREAMING_EP_BUFSIZE < param->dwMaxPayloadTransferSize) { + param->dwMaxPayloadTransferSize = CFG_TUD_VIDEO_STREAMING_EP_BUFSIZE; + } stm->max_payload_transfer_size = param->dwMaxPayloadTransferSize; int ret = tud_video_commit_cb(stm->index_vc, stm->index_vs, param); if (VIDEO_ERROR_NONE == ret) { diff --git a/src/common/tusb_mcu.h b/src/common/tusb_mcu.h index 1f2afb03a..af43dfb12 100644 --- a/src/common/tusb_mcu.h +++ b/src/common/tusb_mcu.h @@ -126,6 +126,14 @@ #define CFG_TUSB_MEM_DCACHE_LINE_SIZE_DEFAULT 32 #endif + // Errata ERR050101, listed for RT1015/RT1020/RT1024/RT1050 (no fix scheduled) and for + // RT1060/RT1064 rev A (fixed in rev B); not listed for RT1010 or the RT11xx family. + #if defined(MIMXRT1015_SERIES) || defined(MIMXRT1021_SERIES) || defined(MIMXRT1024_SERIES) || \ + defined(MIMXRT1051_SERIES) || defined(MIMXRT1052_SERIES) || defined(MIMXRT1061_SERIES) || \ + defined(MIMXRT1062_SERIES) || defined(MIMXRT1064_SERIES) + #define CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101 1 + #endif + #elif TU_CHECK_MCU(OPT_MCU_KINETIS_KL, OPT_MCU_KINETIS_K32L, OPT_MCU_KINETIS_K) #define TUP_USBIP_CHIPIDEA_FS #define TUP_USBIP_CHIPIDEA_FS_KINETIS @@ -141,7 +149,6 @@ #elif TU_CHECK_MCU(OPT_MCU_NRF5X) // 8 CBI + 1 ISO #define TUP_DCD_ENDPOINT_MAX 9 - #define TUP_DCD_EDPT_CLOSE_API #elif TU_CHECK_MCU(OPT_MCU_NRF54) #define TUP_USBIP_DWC2 @@ -673,7 +680,9 @@ #elif TU_CHECK_MCU(OPT_MCU_AT32F403A_407, OPT_MCU_AT32F413) #define TUP_USBIP_FSDEV #define TUP_USBIP_FSDEV_AT32 - #define CFG_TUSB_FSDEV_PMA_SIZE 512u + #define CFG_TUSB_FSDEV_PMA_SIZE 768u + #define CFG_TUSB_FIFO_HWFIFO_DATA_STRIDE 2 + #define CFG_TUSB_FIFO_HWFIFO_ADDR_STRIDE 4 #elif TU_CHECK_MCU(OPT_MCU_AT32F415) #define TUP_USBIP_DWC2 @@ -704,6 +713,25 @@ #define TU_ATTR_FAST_FUNC __attribute__((section(".fast"))) +//--------------------------------------------------------------------+ +// Puya +//--------------------------------------------------------------------+ +#elif TU_CHECK_MCU(OPT_MCU_PY32F0) + #define TUP_USBIP_MUSB + #define TUP_USBIP_MUSB_PY32 + #define TUP_DCD_ENDPOINT_MAX 6 + // PY32 shares the buffer between IN and OUT of the same endpoint number. + // Possible to share IN/OUT if only one direction is armed at any one time + #define CFG_TUD_ENDPOINT_ONE_DIRECTION_ONLY 1 + +//--------------------------------------------------------------------+ +// Geehy +//--------------------------------------------------------------------+ +#elif TU_CHECK_MCU(OPT_MCU_APM32F0XX) + #define TUP_USBIP_FSDEV + #define TUP_USBIP_FSDEV_APM32 + #define CFG_TUSB_FSDEV_PMA_SIZE 1024u + #endif // External USB controller @@ -741,15 +769,24 @@ #define TU_ATTR_FAST_FUNC #endif -#if defined(TUP_USBIP_IP3511) || defined(TUP_USBIP_RUSB2) - #define TUP_DCD_EDPT_CLOSE_API -#endif - -// USBIP implement dcd_edpt_close() and does not support ISO alloc & activate API +// TUP_DCD_EDPT_CLOSE_API is deprecated: these USBIPs implement dcd_edpt_close() and lack the +// ISO alloc & activate API. IP3511, RUSB2 and NRF5X have been migrated to ISO_ALLOC; the remaining +// CLOSE_API MCUs (mm32, pic, da1469x, f1c100s, ch32-usbhs) are pending per-board verification. #ifndef TUP_DCD_EDPT_CLOSE_API #define TUP_DCD_EDPT_ISO_ALLOC #endif +// Set by silicon whose isochronous IN endpoint can be unprimed by an IN token sent to that same +// endpoint number on ANOTHER device sharing the host, taking one of this device's OUT endpoints +// down with it - undetectable in software. Descriptors must then give an isochronous IN endpoint +// a number no other device on the bus uses; a number is only safe while it stays unique, so two +// affected boards on one hub must not pick the same one. Default 0 (no such conflict). Set it to +// 0 by hand on RT1060/RT1064 rev B, which carry the fix - the revision cannot be told apart at +// compile time, so the affected parts are assumed to be rev A. +#ifndef CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101 + #define CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101 0 +#endif + // Some USBIPs (SAMG, SAMX7X, PIC32, MAX3266x/MAX78002) cannot assign the same endpoint // number to both IN and OUT. Default to 0 (same endpoint number may be used for IN and OUT). #ifndef CFG_TUD_ENDPOINT_ONE_DIRECTION_ONLY diff --git a/src/common/tusb_types.h b/src/common/tusb_types.h index d49d277ed..fa4d67df1 100644 --- a/src/common/tusb_types.h +++ b/src/common/tusb_types.h @@ -244,7 +244,7 @@ enum { TUSB_DESC_CONFIG_ATT_SELF_POWERED = 1 << 6, }; -#define TUSB_DESC_CONFIG_POWER_MA(x) ((x)/2) +#define TUSB_DESC_CONFIG_POWER_MA(x) ((uint8_t)TU_MIN((x)/2, UINT8_MAX)) // USB 2.0 Spec Table 9-7: Test Mode Selectors typedef enum { diff --git a/src/device/dcd.h b/src/device/dcd.h index f005e9620..a4006ae0c 100644 --- a/src/device/dcd.h +++ b/src/device/dcd.h @@ -20,19 +20,27 @@ // MACRO CONSTANT TYPEDEF PROTYPES //--------------------------------------------------------------------+ +// Bus reset is reported as two edges. BUS_RESET_START is optional: a controller that +// cannot tell the edges apart emits only BUS_RESET_END, which stays self-sufficient (it +// performs the full teardown with or without a preceding START). Emit START when reset +// signaling is detected - the link is unusable and the speed is not negotiated yet - so +// the stack stops using endpoints immediately instead of at the end of the reset. typedef enum { - DCD_EVENT_INVALID = 0, // 0 - DCD_EVENT_BUS_RESET, // 1 - DCD_EVENT_UNPLUGGED, // 2 - DCD_EVENT_SOF, // 3 - DCD_EVENT_SUSPEND, // 4 TODO LPM Sleep L1 support - DCD_EVENT_RESUME, // 5 - DCD_EVENT_SETUP_RECEIVED, // 6 - DCD_EVENT_XFER_COMPLETE, // 7 - USBD_EVENT_FUNC_CALL, // 8 Not an DCD event, just a convenient way to defer ISR function + DCD_EVENT_INVALID = 0, // 0 + DCD_EVENT_BUS_RESET_START, // 1 + DCD_EVENT_BUS_RESET_END, // 2 with negotiated speed + DCD_EVENT_UNPLUGGED, // 3 + DCD_EVENT_SOF, // 4 + DCD_EVENT_SUSPEND, // 5 TODO LPM Sleep L1 support + DCD_EVENT_RESUME, // 6 + DCD_EVENT_SETUP_RECEIVED, // 7 + DCD_EVENT_XFER_COMPLETE, // 8 + USBD_EVENT_FUNC_CALL, // 9 Not an DCD event, just a convenient way to defer ISR function DCD_EVENT_COUNT } dcd_eventid_t; +#define DCD_EVENT_BUS_RESET DCD_EVENT_BUS_RESET_END // backward compatibility + typedef struct TU_ATTR_ALIGNED(4) { uint8_t rhport; uint8_t event_id; diff --git a/src/device/usbd.c b/src/device/usbd.c index 7a6e13f8d..e84d72fa4 100644 --- a/src/device/usbd.c +++ b/src/device/usbd.c @@ -256,7 +256,7 @@ static const usbd_class_driver_t _usbd_driver[] = { .deinit = vendord_deinit, .reset = vendord_reset, .open = vendord_open, - .control_xfer_cb = tud_vendor_control_xfer_cb, + .control_xfer_cb = vendord_control_xfer_cb, .xfer_cb = vendord_xfer_cb, .xfer_isr = NULL, .sof = NULL @@ -418,6 +418,7 @@ TU_ATTR_ALWAYS_INLINE static inline bool queue_event(dcd_event_t const * event, //--------------------------------------------------------------------+ static bool usbd_control_xfer_cb(uint8_t rhport, uint8_t ep_addr, xfer_result_t result, uint32_t xferred_bytes); static bool process_setup_received(uint8_t rhport, tusb_control_request_t const * p_request); +static bool process_get_status(uint8_t rhport, tusb_control_request_t const * request, uint16_t status); static bool process_set_config(uint8_t rhport, uint8_t cfg_num); static bool process_get_descriptor(uint8_t rhport, tusb_control_request_t const * p_request); @@ -455,7 +456,8 @@ TU_ATTR_WEAK bool dcd_configure(uint8_t rhport, uint32_t cfg_id, const void* cfg #if CFG_TUSB_DEBUG >= CFG_TUD_LOG_LEVEL static char const *const _usbd_event_str[DCD_EVENT_COUNT] = { "Invalid", - "Bus Reset", + "Bus Reset Start", + "Bus Reset End", "Unplugged", "SOF", "Suspend", @@ -641,6 +643,8 @@ static void configuration_reset(uint8_t rhport) { static void usbd_reset(uint8_t rhport) { configuration_reset(rhport); + // discard any pre-reset SETUP still counted: a stale count skips post-reset SETUPs + _usbd_queued_setup = 0; } bool tud_task_event_ready(void) { @@ -694,8 +698,15 @@ void tud_task_ext(uint32_t timeout_ms, bool in_isr) { #endif switch (event.event_id) { - case DCD_EVENT_BUS_RESET: + case DCD_EVENT_BUS_RESET_START: + TU_LOG_USBD("\r\n"); + usbd_reset(event.rhport); + break; + + case DCD_EVENT_BUS_RESET_END: TU_LOG_USBD(": %s Speed\r\n", tu_str_speed[event.bus_reset.speed]); + // TODO a DCD that reports both edges pays for two teardowns: track a per-rhport + // "start seen" flag and skip this reset, keeping it for the single-event DCDs. usbd_reset(event.rhport); _usbd_dev.speed = event.bus_reset.speed; break; @@ -746,7 +757,14 @@ void tud_task_ext(uint32_t timeout_ms, bool in_isr) { _usbd_dev.ep_status[epnum][ep_dir] &= (uint8_t) ~(TU_EDPT_STATE_BUSY | TU_EDPT_STATE_CLAIMED); if (0 == epnum) { - usbd_control_xfer_cb(event.rhport, ep_addr, (xfer_result_t) event.xfer_complete.result, event.xfer_complete.len); + // Not stalled on failure: a DCD refuses an EP0 prime when a newer setup is already + // latched, and EP0 stalls are cleared by hardware when that setup arrives - so a stall + // issued here lands after the auto-clear and would stall the transfer that superseded + // this one. The pending setup re-drives EP0 by itself. + if (!usbd_control_xfer_cb(event.rhport, ep_addr, (xfer_result_t) event.xfer_complete.result, + event.xfer_complete.len)) { + TU_LOG_USBD(" Control stage not continued\r\n"); + } } else { usbd_class_driver_t const* driver = get_driver(_usbd_dev.ep2drv[epnum][ep_dir]); TU_ASSERT(driver,); @@ -864,10 +882,10 @@ bool tud_control_xfer(uint8_t rhport, const tusb_control_request_t* request, voi if (ctrl_xfer->data_len > 0U) { TU_ASSERT(buffer); } - TU_ASSERT(data_stage_xact(rhport)); + TU_VERIFY(data_stage_xact(rhport)); } else { // wLength == 0: Status stage is always IN per USB 2.0 §9.3.1 - TU_ASSERT(status_stage_xact(rhport, TU_EP0_IN)); + TU_VERIFY(status_stage_xact(rhport, TU_EP0_IN)); } return true; @@ -918,7 +936,7 @@ static bool usbd_control_xfer_cb(uint8_t rhport, uint8_t ep_addr, xfer_result_t } if (is_ok) { - TU_ASSERT(status_stage_xact(rhport, ep_status)); + TU_VERIFY(status_stage_xact(rhport, ep_status)); } else { // Stall both IN and OUT control endpoint dcd_edpt_stall(rhport, TU_EP0_OUT); @@ -926,7 +944,7 @@ static bool usbd_control_xfer_cb(uint8_t rhport, uint8_t ep_addr, xfer_result_t } } else { // More data to transfer - TU_ASSERT(data_stage_xact(rhport)); + TU_VERIFY(data_stage_xact(rhport)); } return true; @@ -1041,9 +1059,7 @@ static bool process_std_device_request(uint8_t rhport, tusb_control_request_t co // Device status bit mask // - Bit 0: Self Powered TODO must invoke callback to get actual status // - Bit 1: Remote Wakeup enabled - uint16_t status = (uint16_t) _usbd_dev.dev_state_bm; - tud_control_xfer(rhport, p_request, &status, 2); - return true; + return process_get_status(rhport, p_request, (uint16_t) _usbd_dev.dev_state_bm); } default: @@ -1053,6 +1069,14 @@ static bool process_std_device_request(uint8_t rhport, tusb_control_request_t co } +// Reply to a standard GET_STATUS (device/interface/endpoint) with its 2-byte status word. +// GET_STATUS is Device-to-host only; reject a mis-directed (OUT) request rather than handing +// usbd the address of a stack local to write host data into after this frame has returned. +static bool process_get_status(uint8_t rhport, tusb_control_request_t const * request, uint16_t status) { + TU_VERIFY(request->bmRequestType_bit.direction == TUSB_DIR_IN); + return tud_control_xfer(rhport, request, &status, 2); +} + // This handles the actual request and its response. // Returns false if unable to complete the request, causing caller to stall control endpoints. static bool process_setup_received(uint8_t rhport, tusb_control_request_t const * p_request) { @@ -1150,9 +1174,18 @@ static bool process_setup_received(uint8_t rhport, tusb_control_request_t const } case TUSB_REQ_SET_INTERFACE: + // A class that implements altsettings handles SET_INTERFACE itself and returns true, + // so reaching here means the class does not — where only alt 0 is valid. Any non-zero + // alt (unimplemented, or rejected as invalid by the class) is a Request Error (stall). + TU_VERIFY(tu_u16_low(p_request->wValue) == 0); tud_control_status(rhport, p_request); break; + case TUSB_REQ_GET_STATUS: + // USB 2.0 9.4.5: interface GET_STATUS returns 2 reserved (zero) bytes + TU_VERIFY(process_get_status(rhport, p_request, 0x0000)); + break; + default: return false; } } @@ -1175,35 +1208,34 @@ static bool process_setup_received(uint8_t rhport, tusb_control_request_t const } else { // Handle STD request to endpoint switch (p_request->bRequest) { //-V2520 - case TUSB_REQ_GET_STATUS: { - uint16_t status = usbd_edpt_stalled(rhport, ep_addr) ? 0x0001u : 0x0000u; - tud_control_xfer(rhport, p_request, &status, 2); - } - break; + case TUSB_REQ_GET_STATUS: + // USB 2.0 9.4.5: endpoint GET_STATUS bit 0 = Halt + TU_VERIFY(process_get_status(rhport, p_request, usbd_edpt_stalled(rhport, ep_addr) ? 0x0001u : 0x0000u)); + break; case TUSB_REQ_CLEAR_FEATURE: case TUSB_REQ_SET_FEATURE: { - if ( TUSB_REQ_FEATURE_EDPT_HALT == p_request->wValue ) { - if ( TUSB_REQ_CLEAR_FEATURE == p_request->bRequest ) { - usbd_edpt_clear_stall(rhport, ep_addr); - }else { - usbd_edpt_stall(rhport, ep_addr); - } - } + // ENDPOINT_HALT is the only endpoint feature; it exists only on a non-control endpoint + // that an interface actually owns. Any other selector, the control endpoint (EP0 has no + // Halt feature, USB 2.0 9.4.9), or an endpoint no driver owns is a Request Error (stall). + TU_VERIFY(TUSB_REQ_FEATURE_EDPT_HALT == p_request->wValue); + TU_VERIFY(ep_num != 0); + TU_VERIFY(driver != NULL); - if (driver != NULL) { - // Some classes such as USBTMC needs to clear/re-init its buffer when receiving CLEAR_FEATURE request - // We will also forward std request targeted endpoint to class drivers as well + if ( TUSB_REQ_CLEAR_FEATURE == p_request->bRequest ) { + usbd_edpt_clear_stall(rhport, ep_addr); + } else { + usbd_edpt_stall(rhport, ep_addr); + } - // STD request must always be ACKed regardless of driver returned value - // Also clear complete callback if driver set since it can also stall the request. - (void) invoke_class_control(rhport, driver, p_request); - ctrl_xfer->complete_cb = NULL; + // Some classes such as USBTMC need to clear/re-init their buffer on CLEAR_FEATURE. + // Clear complete callback if driver set since it can also stall the request. + (void) invoke_class_control(rhport, driver, p_request); + ctrl_xfer->complete_cb = NULL; - // skip ZLP status if driver already did that - if (!(_usbd_dev.ep_status[0][TUSB_DIR_IN] & TU_EDPT_STATE_BUSY)) { - tud_control_status(rhport, p_request); - } + // STD request must always be ACKed; skip ZLP status if driver already did that. + if (!(_usbd_dev.ep_status[0][TUSB_DIR_IN] & TU_EDPT_STATE_BUSY)) { + tud_control_status(rhport, p_request); } } break; @@ -1458,8 +1490,18 @@ TU_ATTR_FAST_FUNC void dcd_event_handler(dcd_event_t const* event, bool in_isr) break; } - if (send) { - queue_event(event, in_isr); + if (send && !queue_event(event, in_isr)) { + // event dropped by a full queue: undo state that would otherwise wedge permanently + if (event->event_id == DCD_EVENT_SETUP_RECEIVED) { + // undo the increment, else every later SETUP is skipped as "other SETUP in queue" + // and EP0 is deaf until re-init + _usbd_queued_setup--; + } else if (event->event_id == DCD_EVENT_XFER_COMPLETE) { + // clear busy + claimed, else the endpoint can never be claimed or re-armed again + uint8_t const epnum = tu_edpt_number(event->xfer_complete.ep_addr); + uint8_t const ep_dir = tu_edpt_dir(event->xfer_complete.ep_addr); + _usbd_dev.ep_status[epnum][ep_dir] &= (uint8_t) ~(TU_EDPT_STATE_BUSY | TU_EDPT_STATE_CLAIMED); + } } } @@ -1573,10 +1615,12 @@ bool usbd_edpt_xfer(uint8_t rhport, uint8_t ep_addr, uint8_t* buffer, uint16_t t if (dcd_edpt_xfer(rhport, ep_addr, buffer, total_bytes, is_isr)) { return true; } else { - // DCD error, mark endpoint as ready to allow next transfer + // Driver refused the transfer, mark endpoint as ready to allow next transfer. This is a + // recoverable condition (e.g. a new setup superseding a control response), not a bug, so + // do not break into the debugger - TU_BREAKPOINT() halts the CPU whenever a probe is + // attached, which on a test rig is always. _usbd_dev.ep_status[epnum][dir] &= (uint8_t) ~(TU_EDPT_STATE_BUSY | TU_EDPT_STATE_CLAIMED); TU_LOG_USBD("FAILED\r\n"); - TU_BREAKPOINT(); return false; } } @@ -1648,10 +1692,21 @@ void usbd_edpt_clear_stall(uint8_t rhport, uint8_t ep_addr) { uint8_t const epnum = tu_edpt_number(ep_addr); uint8_t const dir = tu_edpt_dir(ep_addr); - // only clear if currently stalled TU_LOG_USBD(" Clear Stall EP %02X\r\n", ep_addr); + const bool was_stalled = (_usbd_dev.ep_status[epnum][dir] & TU_EDPT_STATE_STALLED) != 0; dcd_edpt_clear_stall(rhport, ep_addr); - _usbd_dev.ep_status[epnum][dir] &= (uint8_t) ~(TU_EDPT_STATE_STALLED | TU_EDPT_STATE_BUSY); + // Clear STALLED|BUSY unconditionally (long-standing behavior; some classes, e.g. audio's + // set-interface, call this on a non-stalled endpoint solely to drop a leftover BUSY bit). + // Only release the CLAIMED ownership bit when the endpoint was actually stalled: the stall + // aborts the in-flight transfer in the dcd with no completion event to release the claim, so + // clearing it here prevents starvation. On a non-stalled clear (e.g. a data-toggle reset) a + // transfer may still be legitimately claimed by another task, so keep CLAIMED to preserve the + // claim->xfer mutual exclusion. + uint8_t clear_mask = TU_EDPT_STATE_STALLED | TU_EDPT_STATE_BUSY; + if (was_stalled) { + clear_mask |= TU_EDPT_STATE_CLAIMED; + } + _usbd_dev.ep_status[epnum][dir] &= (uint8_t) ~clear_mask; } bool usbd_edpt_stalled(uint8_t rhport, uint8_t ep_addr) { diff --git a/src/device/usbd.h b/src/device/usbd.h index 9be12ed0a..2015d1869 100644 --- a/src/device/usbd.h +++ b/src/device/usbd.h @@ -229,7 +229,7 @@ bool tud_vendor_control_xfer_cb(uint8_t rhport, uint8_t stage, tusb_control_requ // Config number, interface count, string index, total length, attribute, power in mA #define TUD_CONFIG_DESCRIPTOR(config_num, _itfcount, _stridx, _total_len, _attribute, _power_ma) \ - 9, TUSB_DESC_CONFIGURATION, U16_TO_U8S_LE(_total_len), _itfcount, config_num, _stridx, TU_BIT(7) | _attribute, (_power_ma)/2 + 9, TUSB_DESC_CONFIGURATION, U16_TO_U8S_LE(_total_len), _itfcount, config_num, _stridx, TU_BIT(7) | _attribute, (uint8_t)TU_MIN((_power_ma)/2, UINT8_MAX) //--------------------------------------------------------------------+ // CDC Descriptor Templates diff --git a/src/portable/chipidea/ci_fs/ci_fs_type.h b/src/portable/chipidea/ci_fs/ci_fs_type.h index a525c96ca..857a75253 100644 --- a/src/portable/chipidea/ci_fs/ci_fs_type.h +++ b/src/portable/chipidea/ci_fs/ci_fs_type.h @@ -27,6 +27,60 @@ extern "C" { // align 4 is used to get rid of reserved fields #define _va32 volatile TU_ATTR_ALIGNED(4) +//--------------------------------------------------------------------+ +// Buffer Descriptor Table (BDT) - shared by the device (dcd) and host (hcd) drivers +// since both target the same ChipIdea-FS silicon. Keep the layout in one place so a +// fix cannot silently drift between the two drivers. +//--------------------------------------------------------------------+ + +// Token PID values reported in the BDT tok_pid field / written to the TOKEN register. +// The device driver only uses OUT/IN/SETUP; the rest are host-only. +enum { + TOK_PID_OUT = 0x1u, + TOK_PID_IN = 0x9u, + TOK_PID_SETUP = 0xDu, + TOK_PID_DATA0 = 0x3u, + TOK_PID_DATA1 = 0xbu, + TOK_PID_ACK = 0x2u, + TOK_PID_STALL = 0xeu, + TOK_PID_NAK = 0xau, + TOK_PID_BUSTO = 0x0u, + TOK_PID_ERR = 0xfu, +}; + +// Note: this header is included before the CMSIS device header, so use plain `volatile` +// rather than CMSIS `__IO`. +typedef struct TU_ATTR_PACKED +{ + union { + uint32_t head; + struct { + union { + struct { + uint16_t : 2; + volatile uint16_t tok_pid : 4; + uint16_t data : 1; + volatile uint16_t own : 1; + uint16_t : 8; + }; + struct { + uint16_t : 2; + uint16_t bdt_stall : 1; + uint16_t dts : 1; + uint16_t ninc : 1; + uint16_t keep : 1; + uint16_t : 10; + }; + }; + volatile uint16_t bc : 10; + uint16_t : 6; + }; + }; + uint8_t *addr; +}buffer_descriptor_t; + +TU_VERIFY_STATIC( sizeof(buffer_descriptor_t) == 8, "size is not correct" ); + typedef struct { _va32 uint8_t PER_ID; // [00] Peripheral ID register _va32 uint8_t ID_COMP; // [04] Peripheral ID complement register diff --git a/src/portable/chipidea/ci_fs/dcd_ci_fs.c b/src/portable/chipidea/ci_fs/dcd_ci_fs.c index 0f3675349..295f2e578 100644 --- a/src/portable/chipidea/ci_fs/dcd_ci_fs.c +++ b/src/portable/chipidea/ci_fs/dcd_ci_fs.c @@ -24,43 +24,7 @@ //--------------------------------------------------------------------+ // MACRO TYPEDEF CONSTANT ENUM DECLARATION //--------------------------------------------------------------------+ - -enum { - TOK_PID_OUT = 0x1u, - TOK_PID_IN = 0x9u, - TOK_PID_SETUP = 0xDu, -}; - -typedef struct TU_ATTR_PACKED -{ - union { - uint32_t head; - struct { - union { - struct { - uint16_t : 2; - __IO uint16_t tok_pid : 4; - uint16_t data : 1; - __IO uint16_t own : 1; - uint16_t : 8; - }; - struct { - uint16_t : 2; - uint16_t bdt_stall : 1; - uint16_t dts : 1; - uint16_t ninc : 1; - uint16_t keep : 1; - uint16_t : 10; - }; - }; - __IO uint16_t bc : 10; - uint16_t : 6; - }; - }; - uint8_t *addr; -}buffer_descriptor_t; - -TU_VERIFY_STATIC( sizeof(buffer_descriptor_t) == 8, "size is not correct" ); +// TOK_PID_* and buffer_descriptor_t are shared with the host driver in ci_fs_type.h typedef struct TU_ATTR_PACKED { @@ -175,6 +139,17 @@ static void process_tokdne(uint8_t rhport) return; } const unsigned length = ep->length; + + /* Transfer is complete. For OUT, a multi-packet transfer speculatively arms the + * sibling (even/odd) BDT to avoid NAK. When the transfer ends early - e.g. the host + * sends a short packet before filling both buffers - that sibling is left armed + * (own=1). A leftover armed BDT desyncs the even/odd ping-pong so the next OUT + * packet lands in the wrong buffer half (buffer + max_packet_size instead of + * buffer), making the stack read stale data. Disarm it here. */ + if (dir == TUSB_DIR_OUT) { + _dcd.bdt[epnum][dir][odd ^ 1].own = 0; + } + dcd_event_xfer_complete(rhport, tu_edpt_addr(epnum, dir), length - remaining, XFER_RESULT_SUCCESS, true); diff --git a/src/portable/nxp/khci/hcd_khci.c b/src/portable/chipidea/ci_fs/hcd_ci_fs.c index 209940656..a6f5405b5 100644 --- a/src/portable/nxp/khci/hcd_khci.c +++ b/src/portable/chipidea/ci_fs/hcd_ci_fs.c @@ -10,63 +10,28 @@ #if CFG_TUH_ENABLED && defined(TUP_USBIP_CHIPIDEA_FS) -#ifdef TUP_USBIP_CHIPIDEA_FS_KINETIS +#include "host/hcd.h" +#include "host/usbh.h" +#include "ci_fs_type.h" + +// Host is currently only available on NXP Kinetis. The ChipIdea-FS host controller +// interface is register-compatible via ci_fs_regs_t. Unlike the device driver, the host +// driver does not include the ci_fs_<family>.h header because those define the device +// dcd_int_enable()/dcd_int_disable() functions, which would collide in a dual-role build. +#if defined(TUP_USBIP_CHIPIDEA_FS_KINETIS) #include "fsl_device_registers.h" - #define KHCI USB0 + #define CI_FS_REG(_port) ((ci_fs_regs_t*) USB0_BASE) + #define CI_FS_IRQN USB0_IRQn #else #error "MCU is not supported" #endif -#include "host/hcd.h" -#include "host/usbh.h" +#define CI_REG CI_FS_REG(0) //--------------------------------------------------------------------+ // MACRO TYPEDEF CONSTANT ENUM DECLARATION //--------------------------------------------------------------------+ - -enum { - TOK_PID_OUT = 0x1u, - TOK_PID_IN = 0x9u, - TOK_PID_SETUP = 0xDu, - TOK_PID_DATA0 = 0x3u, - TOK_PID_DATA1 = 0xbu, - TOK_PID_ACK = 0x2u, - TOK_PID_STALL = 0xeu, - TOK_PID_NAK = 0xau, - TOK_PID_BUSTO = 0x0u, - TOK_PID_ERR = 0xfu, -}; - -typedef struct TU_ATTR_PACKED -{ - union { - uint32_t head; - struct { - union { - struct { - uint16_t : 2; - __IO uint16_t tok_pid : 4; - uint16_t data : 1; - __IO uint16_t own : 1; - uint16_t : 8; - }; - struct { - uint16_t : 2; - uint16_t bdt_stall : 1; - uint16_t dts : 1; - uint16_t ninc : 1; - uint16_t keep : 1; - uint16_t : 10; - }; - }; - __IO uint16_t bc : 10; - uint16_t : 6; - }; - }; - uint8_t *addr; -}buffer_descriptor_t; - -TU_VERIFY_STATIC( sizeof(buffer_descriptor_t) == 8, "size is not correct" ); +// TOK_PID_* and buffer_descriptor_t are shared with the device driver in ci_fs_type.h typedef struct TU_ATTR_PACKED { @@ -107,7 +72,10 @@ typedef struct union { /* [OUT,IN][EVEN,ODD] */ buffer_descriptor_t bdt[2][2]; - uint16_t bda[2*2]; + /* bda aliases bdt for STAT-register indexing: STAT gives the byte-offset/2 of the + * completed BD, so it indexes bda[] in uint16_t units. Each buffer_descriptor_t is + * 4 uint16_t, hence 2*2*4 elements to span the whole table (must equal sizeof bdt). */ + uint16_t bda[2*2*4]; }; endpoint_state_t endpoint[2]; pipe_state_t pipe[CFG_TUH_ENDPOINT_MAX * 2]; @@ -142,7 +110,9 @@ static int prepare_packets(int pipenum) endpoint_state_t *ep = &_hcd.endpoint[dir_tx]; unsigned const odd = ep->odd; buffer_descriptor_t *bd = _hcd.bdt[dir_tx]; - TU_ASSERT(0 == bd[odd].own, -1); + // The host shares a single BDT set across all pipes. If it is still owned by an + // in-flight transfer on another pipe, report busy so the caller can defer & retry. + if (bd[odd].own) return -1; // TU_LOG1(" %p dir %d odd %d data %d\r\n", &bd[odd], dir_tx, odd, pipe->data); @@ -200,11 +170,11 @@ static bool continue_transfer(int pipenum, buffer_descriptor_t *bd) bd->bc = next_rem > pipe->max_packet_size ? pipe->max_packet_size: next_rem; __DSB(); bd->own = 1; /* This bit must be set last */ - while (KHCI->CTL & USB_CTL_TXSUSPENDTOKENBUSY_MASK) ; - KHCI->TOKEN = KHCI->TOKEN; /* Queue the same token as the last */ + while (CI_REG->CTL & USB_CTL_TXSUSPENDTOKENBUSY_MASK) ; + CI_REG->TOKEN = CI_REG->TOKEN; /* Queue the same token as the last */ } else if (TUSB_DIR_IN == tu_edpt_dir(pipe->ep_addr)) { /* IN */ - while (KHCI->CTL & USB_CTL_TXSUSPENDTOKENBUSY_MASK) ; - KHCI->TOKEN = KHCI->TOKEN; + while (CI_REG->CTL & USB_CTL_TXSUSPENDTOKENBUSY_MASK) ; + CI_REG->TOKEN = CI_REG->TOKEN; } return true; } @@ -215,13 +185,21 @@ static bool continue_transfer(int pipenum, buffer_descriptor_t *bd) static bool resume_transfer(int pipenum) { int num_tokens = prepare_packets(pipenum); - TU_ASSERT(0 <= num_tokens); + if (num_tokens < 0) { + // Shared BDT still owned by an in-flight transfer on another pipe. Defer this + // pipe and retry on the next SOF once the BDT is free (avoids dropping the + // transfer, which stalls e.g. a 2nd device enumerating behind a hub while the + // app issues concurrent control transfers). + _hcd.pending |= TU_BIT(pipenum); + CI_REG->INT_EN |= USB_ISTAT_SOFTOK_MASK; + return true; + } - const unsigned ie = NVIC_GetEnableIRQ(USB0_IRQn); - NVIC_DisableIRQ(USB0_IRQn); + const unsigned ie = NVIC_GetEnableIRQ(CI_FS_IRQN); + NVIC_DisableIRQ(CI_FS_IRQN); pipe_state_t *pipe = &_hcd.pipe[pipenum]; - unsigned flags = KHCI->ENDPOINT[0].ENDPT & USB_ENDPT_HOSTWOHUB_MASK; + unsigned flags = CI_REG->EP[0].CTL & USB_ENDPT_HOSTWOHUB_MASK; flags |= USB_ENDPT_EPRXEN_MASK | USB_ENDPT_EPTXEN_MASK; switch (pipe->xfer) { case TUSB_XFER_CONTROL: @@ -236,16 +214,16 @@ static bool resume_transfer(int pipenum) } // TU_LOG1(" resume pipenum %d flags %x\r\n", pipenum, flags); - KHCI->ENDPOINT[0].ENDPT = flags; - KHCI->ADDR = (KHCI->ADDR & USB_ADDR_LSEN_MASK) | pipe->dev_addr; + CI_REG->EP[0].CTL = flags; + CI_REG->ADDR = (CI_REG->ADDR & USB_ADDR_LSEN_MASK) | pipe->dev_addr; unsigned const token = tu_edpt_number(pipe->ep_addr) | ((tu_edpt_dir(pipe->ep_addr) ? TOK_PID_IN: TOK_PID_OUT) << USB_TOKEN_TOKENPID_SHIFT); do { - while (KHCI->CTL & USB_CTL_TXSUSPENDTOKENBUSY_MASK) ; - KHCI->TOKEN = token; + while (CI_REG->CTL & USB_CTL_TXSUSPENDTOKENBUSY_MASK) ; + CI_REG->TOKEN = token; } while (--num_tokens); - if (ie) NVIC_EnableIRQ(USB0_IRQn); + if (ie) NVIC_EnableIRQ(CI_FS_IRQN); return true; } @@ -253,19 +231,38 @@ static void suspend_transfer(int pipenum, buffer_descriptor_t *bd) { pipe_state_t *pipe = &_hcd.pipe[pipenum]; pipe->buffer = bd->addr; - pipe->data = bd->data ^ 1; + // A NAK transfers no data, so the data toggle must be preserved for the retry. + // (Do NOT flip pipe->data here: flipping it makes the retried packet use the wrong + // DATA0/DATA1, which the device silently discards - breaking any bulk/interrupt + // transfer that is NAKed, e.g. the MSC CBW/CSW when the device is momentarily busy.) if ((TUSB_XFER_INTERRUPT == pipe->xfer) || (TUSB_XFER_BULK == pipe->xfer)) { _hcd.pending |= TU_BIT(pipenum); - KHCI->INTEN |= USB_ISTAT_SOFTOK_MASK; + CI_REG->INT_EN |= USB_ISTAT_SOFTOK_MASK; + } +} + +// Release the speculatively-armed sibling BDT of a multi-packet transfer. +// prepare_packets arms the sibling (odd^1) BDT (own=1) so a multi-packet transfer can +// ping-pong without NAKs. When the transfer ends - completes early on a short IN packet, +// stalls/errors, or (for IN) is NAKed before the sibling's token is issued - that sibling +// is left owned by the SIE. Because the host shares ONE BDT set across all pipes, a +// leftover armed sibling blocks every other pipe forever (e.g. a 2nd device stuck +// enumerating behind a hub). Release it - but ONLY for a multi-packet transfer: a +// single-packet transfer never armed a sibling, so that BDT slot may legitimately belong +// to another pipe's in-flight transfer. +static inline void release_sibling_bd(unsigned s, const pipe_state_t *pipe) +{ + if (pipe->length > pipe->max_packet_size) { + ((buffer_descriptor_t *)&_hcd.bda[s ^ USB_STAT_ODD_MASK])->own = 0; } } static void process_tokdne(uint8_t rhport) { (void)rhport; - const unsigned s = KHCI->STAT; - KHCI->ISTAT = USB_ISTAT_TOKDNE_MASK; /* fetch the next token if received */ + const unsigned s = CI_REG->STAT; + CI_REG->INT_STAT = USB_ISTAT_TOKDNE_MASK; /* fetch the next token if received */ uint8_t const dir_in = (s & USB_STAT_TX_MASK) ? TUSB_DIR_OUT: TUSB_DIR_IN; unsigned const odd = (s & USB_STAT_ODD_MASK) ? 1 : 0; @@ -295,6 +292,12 @@ static void process_tokdne(uint8_t rhport) result = XFER_RESULT_SUCCESS; break; case TOK_PID_NAK: + // Release the speculatively-armed sibling so the deferred retry (and any other pipe + // sharing the single BDT) can claim it; otherwise it stays own=1 forever and every + // same-direction transfer wedges. IN only: an IN issues just one token so the sibling + // was never put on the wire, whereas an OUT issues both tokens and its sibling may + // still be in flight - touching it there would race the SIE write-back. + if (TUSB_DIR_IN == dir_in) release_sibling_bd(s, &_hcd.pipe[pipenum]); suspend_transfer(pipenum, bd); next_pipenum = select_next_pipenum(pipenum); if (0 <= next_pipenum) @@ -310,8 +313,9 @@ static void process_tokdne(uint8_t rhport) } _hcd.in_progress &= ~TU_BIT(pipenum); pipe_state_t *pipe = &_hcd.pipe[ep->pipenum]; + release_sibling_bd(s, pipe); hcd_event_xfer_complete(pipe->dev_addr, - tu_edpt_addr(KHCI->TOKEN & USB_TOKEN_TOKENENDPT_MASK, dir_in), + tu_edpt_addr(CI_REG->TOKEN & USB_TOKEN_TOKENENDPT_MASK, dir_in), pipe->length - pipe->remaining, result, true); next_pipenum = select_next_pipenum(pipenum); @@ -321,29 +325,32 @@ static void process_tokdne(uint8_t rhport) static void process_attach(uint8_t rhport) { - unsigned ctl = KHCI->CTL; + unsigned ctl = CI_REG->CTL; if (!(ctl & USB_CTL_JSTATE_MASK)) { /* The attached device is a low speed device. */ - KHCI->ADDR = USB_ADDR_LSEN_MASK; - KHCI->ENDPOINT[0].ENDPT = USB_ENDPT_HOSTWOHUB_MASK; + CI_REG->ADDR = USB_ADDR_LSEN_MASK; + CI_REG->EP[0].CTL = USB_ENDPT_HOSTWOHUB_MASK; } hcd_event_device_attach(rhport, true); } static void process_bus_reset(uint8_t rhport) { - KHCI->ISTAT = USB_ISTAT_TOKDNE_MASK; - KHCI->USBCTRL &= ~USB_USBCTRL_SUSP_MASK; - KHCI->CTL &= ~USB_CTL_USBENSOFEN_MASK; - KHCI->ADDR = 0; - KHCI->ENDPOINT[0].ENDPT = 0; + CI_REG->INT_STAT = USB_ISTAT_TOKDNE_MASK; + CI_REG->USBCTRL &= ~USB_USBCTRL_SUSP_MASK; + CI_REG->CTL &= ~USB_CTL_USBENSOFEN_MASK; + CI_REG->ADDR = 0; + CI_REG->EP[0].CTL = 0; hcd_event_device_remove(rhport, true); _hcd.in_progress = 0; _hcd.pending = 0; + // Clear the ENTIRE shared BDT (both directions, both even/odd). Clearing only the IN + // pair left a stale OUT/SETUP descriptor (own=1) after a disconnect mid-OUT, which then + // blocks the first control transfer on re-enumeration. buffer_descriptor_t *bd = &_hcd.bdt[0][0]; - for (unsigned i = 0; i < 2; ++i, ++bd) { + for (unsigned i = 0; i < 2 * 2; ++i, ++bd) { bd->head = 0; } } @@ -354,31 +361,31 @@ static void process_bus_reset(uint8_t rhport) bool hcd_init(uint8_t rhport, const tusb_rhport_init_t* rh_init) { (void) rhport; (void) rh_init; - KHCI->USBTRC0 |= USB_USBTRC0_USBRESET_MASK; - while (KHCI->USBTRC0 & USB_USBTRC0_USBRESET_MASK); + CI_REG->USBTRC0 |= USB_USBTRC0_USBRESET_MASK; + while (CI_REG->USBTRC0 & USB_USBTRC0_USBRESET_MASK); tu_memclr(&_hcd, sizeof(_hcd)); - KHCI->USBTRC0 |= TU_BIT(6); /* software must set this bit to 1 */ - KHCI->BDTPAGE1 = (uint8_t)((uintptr_t)_hcd.bdt >> 8); - KHCI->BDTPAGE2 = (uint8_t)((uintptr_t)_hcd.bdt >> 16); - KHCI->BDTPAGE3 = (uint8_t)((uintptr_t)_hcd.bdt >> 24); + CI_REG->USBTRC0 |= TU_BIT(6); /* software must set this bit to 1 */ + CI_REG->BDT_PAGE1 = (uint8_t)((uintptr_t)_hcd.bdt >> 8); + CI_REG->BDT_PAGE2 = (uint8_t)((uintptr_t)_hcd.bdt >> 16); + CI_REG->BDT_PAGE3 = (uint8_t)((uintptr_t)_hcd.bdt >> 24); - KHCI->USBCTRL &= ~USB_USBCTRL_SUSP_MASK; - KHCI->CTL |= USB_CTL_ODDRST_MASK; + CI_REG->USBCTRL &= ~USB_USBCTRL_SUSP_MASK; + CI_REG->CTL |= USB_CTL_ODDRST_MASK; for (unsigned i = 0; i < 16; ++i) { - KHCI->ENDPOINT[i].ENDPT = 0; + CI_REG->EP[i].CTL = 0; } - KHCI->CTL &= ~USB_CTL_ODDRST_MASK; + CI_REG->CTL &= ~USB_CTL_ODDRST_MASK; - KHCI->SOFTHLD = 74; /* for 64-byte packets */ - // KHCI->SOFTHLD = 144; /* for low speed 8-byte packets */ - KHCI->CTL = USB_CTL_HOSTMODEEN_MASK | USB_CTL_SE0_MASK; - KHCI->USBCTRL = USB_USBCTRL_PDE_MASK; + CI_REG->SOF_THLD = 74; /* for 64-byte packets */ + // CI_REG->SOF_THLD = 144; /* for low speed 8-byte packets */ + CI_REG->CTL = USB_CTL_HOSTMODEEN_MASK | USB_CTL_SE0_MASK; + CI_REG->USBCTRL = USB_USBCTRL_PDE_MASK; - NVIC_ClearPendingIRQ(USB0_IRQn); - KHCI->INTEN = USB_INTEN_ATTACHEN_MASK | USB_INTEN_TOKDNEEN_MASK | + NVIC_ClearPendingIRQ(CI_FS_IRQN); + CI_REG->INT_EN = USB_INTEN_ATTACHEN_MASK | USB_INTEN_TOKDNEEN_MASK | USB_INTEN_USBRSTEN_MASK | USB_INTEN_ERROREN_MASK | USB_INTEN_STALLEN_MASK; - KHCI->ERREN = 0xff; + CI_REG->ERR_ENB = 0xff; return true; } @@ -386,13 +393,13 @@ bool hcd_init(uint8_t rhport, const tusb_rhport_init_t* rh_init) { void hcd_int_enable(uint8_t rhport) { (void)rhport; - NVIC_EnableIRQ(USB0_IRQn); + NVIC_EnableIRQ(CI_FS_IRQN); } void hcd_int_disable(uint8_t rhport) { (void)rhport; - NVIC_DisableIRQ(USB0_IRQn); + NVIC_DisableIRQ(CI_FS_IRQN); } uint32_t hcd_frame_number(uint8_t rhport) @@ -401,8 +408,8 @@ uint32_t hcd_frame_number(uint8_t rhport) /* The device must be reset at least once after connection * in order to start the frame counter. */ if (_hcd.need_reset) hcd_port_reset(rhport); - uint32_t frmnum = KHCI->FRMNUML; - frmnum |= KHCI->FRMNUMH << 8u; + uint32_t frmnum = CI_REG->FRM_NUML; + frmnum |= CI_REG->FRM_NUMH << 8u; return frmnum; } @@ -412,7 +419,7 @@ uint32_t hcd_frame_number(uint8_t rhport) bool hcd_port_connect_status(uint8_t rhport) { (void)rhport; - if (KHCI->ISTAT & USB_ISTAT_ATTACH_MASK) + if (CI_REG->INT_STAT & USB_ISTAT_ATTACH_MASK) return true; return false; } @@ -420,12 +427,12 @@ bool hcd_port_connect_status(uint8_t rhport) void hcd_port_reset(uint8_t rhport) { (void)rhport; - KHCI->CTL &= ~USB_CTL_USBENSOFEN_MASK; - KHCI->CTL |= USB_CTL_RESET_MASK; + CI_REG->CTL &= ~USB_CTL_USBENSOFEN_MASK; + CI_REG->CTL |= USB_CTL_RESET_MASK; unsigned cnt = SystemCoreClock / 100; while (cnt--) __NOP(); - KHCI->CTL &= ~USB_CTL_RESET_MASK; - KHCI->CTL |= USB_CTL_USBENSOFEN_MASK; + CI_REG->CTL &= ~USB_CTL_RESET_MASK; + CI_REG->CTL |= USB_CTL_USBENSOFEN_MASK; _hcd.need_reset = false; } @@ -437,26 +444,26 @@ tusb_speed_t hcd_port_speed_get(uint8_t rhport) { (void)rhport; tusb_speed_t speed = TUSB_SPEED_FULL; - const unsigned ie = NVIC_GetEnableIRQ(USB0_IRQn); - NVIC_DisableIRQ(USB0_IRQn); - if (KHCI->ADDR & USB_ADDR_LSEN_MASK) + const unsigned ie = NVIC_GetEnableIRQ(CI_FS_IRQN); + NVIC_DisableIRQ(CI_FS_IRQN); + if (CI_REG->ADDR & USB_ADDR_LSEN_MASK) speed = TUSB_SPEED_LOW; - if (ie) NVIC_EnableIRQ(USB0_IRQn); + if (ie) NVIC_EnableIRQ(CI_FS_IRQN); return speed; } void hcd_device_close(uint8_t rhport, uint8_t dev_addr) { (void)rhport; - const unsigned ie = NVIC_GetEnableIRQ(USB0_IRQn); - NVIC_DisableIRQ(USB0_IRQn); + const unsigned ie = NVIC_GetEnableIRQ(CI_FS_IRQN); + NVIC_DisableIRQ(CI_FS_IRQN); pipe_state_t *p = &_hcd.pipe[0]; pipe_state_t *end = &_hcd.pipe[CFG_TUH_ENDPOINT_MAX * 2]; for (;p != end; ++p) { if (p->dev_addr == dev_addr) tu_memclr(p, sizeof(*p)); } - if (ie) NVIC_EnableIRQ(USB0_IRQn); + if (ie) NVIC_EnableIRQ(CI_FS_IRQN); } //--------------------------------------------------------------------+ @@ -483,12 +490,12 @@ bool hcd_setup_send(uint8_t rhport, uint8_t dev_addr, uint8_t const setup_packet _hcd.in_progress |= TU_BIT(pipenum); - unsigned hostwohub = KHCI->ENDPOINT[0].ENDPT & USB_ENDPT_HOSTWOHUB_MASK; - KHCI->ENDPOINT[0].ENDPT = hostwohub | + unsigned hostwohub = CI_REG->EP[0].CTL & USB_ENDPT_HOSTWOHUB_MASK; + CI_REG->EP[0].CTL = hostwohub | USB_ENDPT_EPHSHK_MASK | USB_ENDPT_EPRXEN_MASK | USB_ENDPT_EPTXEN_MASK; - KHCI->ADDR = (KHCI->ADDR & USB_ADDR_LSEN_MASK) | dev_addr; - while (KHCI->CTL & USB_CTL_TXSUSPENDTOKENBUSY_MASK) ; - KHCI->TOKEN = (TOK_PID_SETUP << USB_TOKEN_TOKENPID_SHIFT); + CI_REG->ADDR = (CI_REG->ADDR & USB_ADDR_LSEN_MASK) | dev_addr; + while (CI_REG->CTL & USB_CTL_TXSUSPENDTOKENBUSY_MASK) ; + CI_REG->TOKEN = (TOK_PID_SETUP << USB_TOKEN_TOKENPID_SHIFT); return true; } @@ -540,16 +547,16 @@ bool hcd_edpt_xfer(uint8_t rhport, uint8_t dev_addr, uint8_t ep_addr, uint8_t * TU_ASSERT(0 <= pipenum); TU_ASSERT(0 == (_hcd.in_progress & TU_BIT(pipenum))); - unsigned const ie = NVIC_GetEnableIRQ(USB0_IRQn); - NVIC_DisableIRQ(USB0_IRQn); + unsigned const ie = NVIC_GetEnableIRQ(CI_FS_IRQN); + NVIC_DisableIRQ(CI_FS_IRQN); pipe_state_t *pipe = &_hcd.pipe[pipenum]; pipe->buffer = buffer; pipe->length = buflen; pipe->remaining = buflen; _hcd.in_progress |= TU_BIT(pipenum); _hcd.pending |= TU_BIT(pipenum); /* Send at the next Frame */ - KHCI->INTEN |= USB_ISTAT_SOFTOK_MASK; - if (ie) NVIC_EnableIRQ(USB0_IRQn); + CI_REG->INT_EN |= USB_ISTAT_SOFTOK_MASK; + if (ie) NVIC_EnableIRQ(CI_FS_IRQN); return true; } @@ -577,42 +584,42 @@ bool hcd_edpt_clear_stall(uint8_t rhport, uint8_t dev_addr, uint8_t ep_addr) { void hcd_int_handler(uint8_t rhport, bool in_isr) { (void) in_isr; - uint32_t is = KHCI->ISTAT; - uint32_t msk = KHCI->INTEN; + uint32_t is = CI_REG->INT_STAT; + uint32_t msk = CI_REG->INT_EN; // TU_LOG1("S %lx\r\n", is); /* clear disabled interrupts */ - KHCI->ISTAT = (is & ~msk & ~USB_ISTAT_TOKDNE_MASK) | USB_ISTAT_SOFTOK_MASK; + CI_REG->INT_STAT = (is & ~msk & ~USB_ISTAT_TOKDNE_MASK) | USB_ISTAT_SOFTOK_MASK; is &= msk; if (is & USB_ISTAT_ERROR_MASK) { - unsigned err = KHCI->ERRSTAT; + unsigned err = CI_REG->ERR_STAT; if (err) { TU_LOG1(" ERR %x\r\n", err); - KHCI->ERRSTAT = err; + CI_REG->ERR_STAT = err; } else { - KHCI->INTEN &= ~USB_ISTAT_ERROR_MASK; + CI_REG->INT_EN &= ~USB_ISTAT_ERROR_MASK; } } if (is & USB_ISTAT_USBRST_MASK) { - KHCI->INTEN = (msk & ~USB_INTEN_USBRSTEN_MASK) | USB_INTEN_ATTACHEN_MASK; + CI_REG->INT_EN = (msk & ~USB_INTEN_USBRSTEN_MASK) | USB_INTEN_ATTACHEN_MASK; process_bus_reset(rhport); return; } if (is & USB_ISTAT_ATTACH_MASK) { - KHCI->INTEN = (msk & ~USB_INTEN_ATTACHEN_MASK) | USB_INTEN_USBRSTEN_MASK; + CI_REG->INT_EN = (msk & ~USB_INTEN_ATTACHEN_MASK) | USB_INTEN_USBRSTEN_MASK; _hcd.need_reset = true; process_attach(rhport); return; } if (is & USB_ISTAT_STALL_MASK) { - KHCI->ISTAT = USB_ISTAT_STALL_MASK; + CI_REG->INT_STAT = USB_ISTAT_STALL_MASK; } if (is & USB_ISTAT_SOFTOK_MASK) { msk &= ~USB_ISTAT_SOFTOK_MASK; - KHCI->INTEN = msk; + CI_REG->INT_EN = msk; if (_hcd.pending) { int pipenum = __builtin_ctz(_hcd.pending); _hcd.pending = 0; diff --git a/src/portable/chipidea/ci_hs/ci_hs_imxrt.h b/src/portable/chipidea/ci_hs/ci_hs_imxrt.h index f0f918fe2..8f0d6083e 100644 --- a/src/portable/chipidea/ci_hs/ci_hs_imxrt.h +++ b/src/portable/chipidea/ci_hs/ci_hs_imxrt.h @@ -36,6 +36,9 @@ static const ci_hs_controller_t _ci_controller[] = #define CI_HS_REG(_port) ((ci_hs_regs_t*) _ci_controller[_port].reg_base) +// NXP recommends AHBBRST = INCR16 (remainder as unspecified-length bursts) +#define CI_HS_SET_AHB_BURST(_p) (CI_HS_REG(_p)->SBUSCFG = SBUSCFG_AHBBRST_INCR16_UNSPEC) + //------------- DCD -------------// #define CI_DCD_INT_ENABLE(_p) NVIC_EnableIRQ ((IRQn_Type)_ci_controller[_p].irqnum) #define CI_DCD_INT_DISABLE(_p) NVIC_DisableIRQ((IRQn_Type)_ci_controller[_p].irqnum) diff --git a/src/portable/chipidea/ci_hs/ci_hs_lpc18_43.h b/src/portable/chipidea/ci_hs/ci_hs_lpc18_43.h index f2061bd7a..c7dc7e69f 100644 --- a/src/portable/chipidea/ci_hs/ci_hs_lpc18_43.h +++ b/src/portable/chipidea/ci_hs/ci_hs_lpc18_43.h @@ -34,4 +34,9 @@ static const ci_hs_controller_t _ci_controller[] = #define CI_HCD_INT_ENABLE(_p) NVIC_EnableIRQ ((IRQn_Type)_ci_controller[_p].irqnum) #define CI_HCD_INT_DISABLE(_p) NVIC_DisableIRQ((IRQn_Type)_ci_controller[_p].irqnum) +// USB0 (high-speed) only: NXP recommends AHBBRST = INCR16 (remainder as +// unspecified-length bursts) +#define CI_HS_SET_AHB_BURST(_p) \ + do { if ((_p) == 0) { CI_HS_REG(_p)->SBUSCFG = SBUSCFG_AHBBRST_INCR16_UNSPEC; } } while (0) + #endif diff --git a/src/portable/chipidea/ci_hs/ci_hs_type.h b/src/portable/chipidea/ci_hs/ci_hs_type.h index 70817a6e3..5baa14821 100644 --- a/src/portable/chipidea/ci_hs/ci_hs_type.h +++ b/src/portable/chipidea/ci_hs/ci_hs_type.h @@ -36,10 +36,18 @@ enum { PORTSC1_CURRENT_CONNECT_STATUS = TU_BIT(0), PORTSC1_FORCE_PORT_RESUME = TU_BIT(6), PORTSC1_SUSPEND = TU_BIT(7), + PORTSC1_PORT_RESET = TU_BIT(8), // read-only in device mode: a reset is being driven PORTSC1_FORCE_FULL_SPEED = TU_BIT(24), PORTSC1_PORT_SPEED = TU_BIT(26) | TU_BIT(27) }; +// PORTSC1 PSPD field values, once shifted down by PORTSC1_PORT_SPEED_POS. 3 is undefined. +enum { + PORTSC1_PORT_SPEED_FULL = 0, + PORTSC1_PORT_SPEED_LOW = 1, + PORTSC1_PORT_SPEED_HIGH = 2, +}; + // OTGSC enum { OTGSC_VBUS_DISCHARGE = TU_BIT(0), @@ -71,11 +79,18 @@ enum { USBMODE_VBUS_POWER_SELECT = TU_BIT(5), // Need to be enabled for LPC18XX/43XX in host mode }; +// SBUSCFG +enum { + SBUSCFG_AHBBRST_INCR16_UNSPEC = 7, // INCR16 burst, remainder as unspecified-length bursts +}; + // Device Registers typedef struct { //------------- ID + HW Parameter Registers-------------// - volatile uint32_t TU_RESERVED[64]; ///< For iMX RT10xx, but not used by LPC18XX/LPC43XX + volatile uint32_t TU_RESERVED[36]; ///< ID/HW parameter registers, not used by this driver + volatile uint32_t SBUSCFG; ///< System Bus Interface Configuration (not present on every MCU) + volatile uint32_t TU_RESERVED[27]; //------------- Capability Registers-------------// volatile uint8_t CAPLENGTH; ///< Capability Registers Length diff --git a/src/portable/chipidea/ci_hs/dcd_ci_hs.c b/src/portable/chipidea/ci_hs/dcd_ci_hs.c index 55906e678..6ab28e0be 100644 --- a/src/portable/chipidea/ci_hs/dcd_ci_hs.c +++ b/src/portable/chipidea/ci_hs/dcd_ci_hs.c @@ -154,6 +154,14 @@ TU_VERIFY_STATIC(sizeof(dcd_qhd_t) == 64, "size is not correct"); #define QTD_NEXT_INVALID 0x01 +// Bounded spin for register waits. The longest legitimate wait is a flush held off by a packet +// already in progress: ~50 us for a full-speed 64-byte packet, a low thousands of dependent +// register reads, so healthy hardware never approaches this bound. Exceeding it means the +// controller has stopped responding, and the spin then only serves to keep an ISR (or an +// IRQ-masked caller) from hanging outright - the 3 ms reset-cleanup window of IMXRT1060RM 42.5.6.2.1 (p.2394) +// is already unreachable in that state, and the manual's remedy there is a controller reset. +#define CI_HS_BUSY_SPIN 10000u + typedef struct { // Must be at 2K alignment // Each endpoint with direction (IN/OUT) occupies a queue head @@ -164,6 +172,17 @@ typedef struct { CFG_TUD_MEM_SECTION TU_ATTR_ALIGNED(2048) static dcd_data_t _dcd_data; +// What the next Port Change Detect will be. Each one is preceded by the interrupt that causes it: +// a reset interrupt for the end of a bus reset - where the speed first becomes final - or a +// suspend interrupt for the resume that ends the suspend. A suspend itself raises no port change, +// which is why there is no such value here. Indexed by rhport, which is 0 or 1 on every ci_hs +// variant (NOT the controller count: mcx/rw61x map rhport 1 to controller 0). +enum { + PORT_CHANGE_REASON_RESET = 0, + PORT_CHANGE_REASON_RESUME = 1, +}; +static volatile uint8_t _port_change_reason[2]; + //--------------------------------------------------------------------+ // Prototypes and Helper Functions //--------------------------------------------------------------------+ @@ -172,12 +191,37 @@ TU_ATTR_ALWAYS_INLINE static inline uint8_t ci_ep_count(const ci_hs_regs_t *dcd_ return dcd_reg->DCCPARAMS & DCCPARAMS_DEN_MASK; } +static bool controller_reset(uint8_t rhport); + //--------------------------------------------------------------------+ // Controller API //--------------------------------------------------------------------+ -/// follows LPC43xx User Manual 23.10.3 -static void bus_reset(uint8_t rhport) { +// Flush endpoint buffers, following IMXRT1060RM 42.5.6.6.5 Flushing/De-priming an Endpoint +// (p.2413): write ENDPTFLUSH, wait for the controller +// to acknowledge, then confirm ENDPTSTAT went to zero. The controller refuses the flush when a +// packet is in progress, and the manual requires the procedure be repeated until it takes. +// Callers proceed regardless of the result; the bound only prevents an ISR-context hang on dead +// hardware. +static bool flush_endpoints(ci_hs_regs_t *dcd_reg, uint32_t mask) { + uint32_t guard = CI_HS_BUSY_SPIN; + do { + dcd_reg->ENDPTFLUSH = mask; + while (dcd_reg->ENDPTFLUSH & mask) { + if (!guard--) { + return false; + } + } + } while ((dcd_reg->ENDPTSTAT & mask) && guard--); + + return !(dcd_reg->ENDPTSTAT & mask); +} + +/// Everything the manual asks of the DCD when a reset is detected, in its order: clear the setup +/// and completion semaphores, cancel every prime, check the reset is still being driven, and free +/// the dTDs. All of it belongs inside the reset window (IMXRT1060RM 42.5.6.2.1, p.2394); nothing +/// is left for the port change that ends the reset, which only reports the negotiated speed. +static void bus_reset_begin(uint8_t rhport) { ci_hs_regs_t *dcd_reg = CI_HS_REG(rhport); // The reset value for all endpoint types is the control endpoint. If one endpoint @@ -193,17 +237,24 @@ static void bus_reset(uint8_t rhport) { //------------- Clear All Registers -------------// dcd_reg->ENDPTNAK = dcd_reg->ENDPTNAK; dcd_reg->ENDPTNAKEN = 0; - dcd_reg->USBSTS = dcd_reg->USBSTS; dcd_reg->ENDPTSETUPSTAT = dcd_reg->ENDPTSETUPSTAT; dcd_reg->ENDPTCOMPLETE = dcd_reg->ENDPTCOMPLETE; - while (dcd_reg->ENDPTPRIME) {} - dcd_reg->ENDPTFLUSH = 0xFFFFFFFF; - while (dcd_reg->ENDPTFLUSH) {} + uint32_t guard = CI_HS_BUSY_SPIN; + while (dcd_reg->ENDPTPRIME && guard--) {} + dcd_reg->ENDPTFLUSH = 0xFFFFFFFFUL; - // read reset bit in portsc + // All of the above must land while the reset is still being driven - it lasts at least 3 ms. + // Arriving late leaves the controller in an undefined state, and the manual's remedy is to + // hardware-reset it. That clears Run/Stop, so the device detaches and the host will drive a + // fresh reset and enumeration - which is why nothing below this point is worth doing here. + if (!(dcd_reg->PORTSC1 & PORTSC1_PORT_RESET)) { + TU_LOG1("ci_hs: reset cleanup ran past the end of the reset, resetting controller\r\n"); + controller_reset(rhport); + return; // the controller detached; the host's next reset redoes everything below + } - //------------- Queue Head & Queue TD -------------// + //------------- Free all allocated dTDs: the controller will not execute them again -------------// tu_memclr(&_dcd_data, sizeof(dcd_data_t)); //------------- Set up Control Endpoints (0 OUT, 1 IN) -------------// @@ -216,27 +267,29 @@ static void bus_reset(uint8_t rhport) { dcd_dcache_clean_invalidate(&_dcd_data, sizeof(dcd_data_t)); } -bool dcd_init(uint8_t rhport, const tusb_rhport_init_t *rh_init) { - (void)rh_init; - tu_memclr(&_dcd_data, sizeof(dcd_data_t)); - +/// Reset the controller and bring it back up in device mode. Also the manual's remedy when the +/// reset cleanup misses its window: the controller reset clears Run/Stop and detaches the device, +/// so it must be re-initialised completely afterwards (IMXRT1060RM 42.5.6.2.1, p.2394). +static bool controller_reset(uint8_t rhport) { ci_hs_regs_t *dcd_reg = CI_HS_REG(rhport); - TU_ASSERT(ci_ep_count(dcd_reg) <= TUP_DCD_ENDPOINT_MAX); - - #if TU_CHECK_MCU(OPT_MCU_HPM) - usb_phy_init((USB_Type *)dcd_reg, false); - #endif + tu_memclr(&_dcd_data, sizeof(dcd_data_t)); // Reset controller dcd_reg->USBCMD |= USBCMD_RESET; - while (dcd_reg->USBCMD & USBCMD_RESET) {} + uint32_t guard = CI_HS_BUSY_SPIN; + while ((dcd_reg->USBCMD & USBCMD_RESET) && guard--) {} + TU_VERIFY(!(dcd_reg->USBCMD & USBCMD_RESET)); // reached from the ISR too, so never halt here // Set mode to device, must be set immediately after reset uint32_t usbmode = dcd_reg->USBMODE & ~USBMOD_CM_MASK; usbmode |= USBMODE_CM_DEVICE; dcd_reg->USBMODE = usbmode; + #ifdef CI_HS_SET_AHB_BURST + CI_HS_SET_AHB_BURST(rhport); + #endif + #ifdef CFG_TUD_CI_HS_VBUS_CHARGE dcd_reg->OTGSC = OTGSC_VBUS_CHARGE | OTGSC_OTG_TERMINATION; #else @@ -253,9 +306,11 @@ bool dcd_init(uint8_t rhport, const tusb_rhport_init_t *rh_init) { dcd_dcache_clean_invalidate(&_dcd_data, sizeof(dcd_data_t)); + _port_change_reason[rhport] = PORT_CHANGE_REASON_RESET; + dcd_reg->ENDPTLISTADDR = (uint32_t)_dcd_data.qhd; // Endpoint List Address has to be 2K alignment dcd_reg->USBSTS = dcd_reg->USBSTS; - dcd_reg->USBINTR = INTR_USB | INTR_ERROR | INTR_PORT_CHANGE | INTR_SUSPEND; + dcd_reg->USBINTR = INTR_USB | INTR_ERROR | INTR_PORT_CHANGE | INTR_RESET | INTR_SUSPEND; uint32_t usbcmd = dcd_reg->USBCMD; usbcmd &= ~USBCMD_INTR_THRESHOLD_MASK; // Interrupt Threshold Interval = 0 @@ -266,8 +321,22 @@ bool dcd_init(uint8_t rhport, const tusb_rhport_init_t *rh_init) { return true; } +bool dcd_init(uint8_t rhport, const tusb_rhport_init_t *rh_init) { + (void)rh_init; + ci_hs_regs_t *dcd_reg = CI_HS_REG(rhport); + + TU_ASSERT(ci_ep_count(dcd_reg) <= TUP_DCD_ENDPOINT_MAX); + + #if TU_CHECK_MCU(OPT_MCU_HPM) + usb_phy_init((USB_Type *)dcd_reg, false); + #endif + + return controller_reset(rhport); +} + bool dcd_deinit(uint8_t rhport) { ci_hs_regs_t* dcd_reg = CI_HS_REG(rhport); + _port_change_reason[rhport] = PORT_CHANGE_REASON_RESET; // disable all interrupt dcd_reg->USBINTR = 0; @@ -276,9 +345,9 @@ bool dcd_deinit(uint8_t rhport) { dcd_reg->USBCMD &= ~USBCMD_RUN_STOP; // flush all endpoints - while (dcd_reg->ENDPTPRIME) {} - dcd_reg->ENDPTFLUSH = 0xFFFFFFFF; - while (dcd_reg->ENDPTFLUSH) {} + uint32_t guard = CI_HS_BUSY_SPIN; + while (dcd_reg->ENDPTPRIME && guard--) {} + flush_endpoints(dcd_reg, 0xFFFFFFFF); return true; } @@ -292,11 +361,13 @@ void dcd_int_disable(uint8_t rhport) { } void dcd_set_address(uint8_t rhport, uint8_t dev_addr) { - // Response with status first before changing device address - dcd_edpt_xfer(rhport, tu_edpt_addr(0, TUSB_DIR_IN), NULL, 0, false); - - ci_hs_regs_t *dcd_reg = CI_HS_REG(rhport); - dcd_reg->DEVICEADDR = (dev_addr << 25) | TU_BIT(24); + // Response with status first before changing device address. A refused prime means a new + // setup superseded this transfer; staging an address whose ACK will never arrive would + // leave the device answering on it, so only arm the address when the status went out. + if (dcd_edpt_xfer(rhport, tu_edpt_addr(0, TUSB_DIR_IN), NULL, 0, false)) { + ci_hs_regs_t *dcd_reg = CI_HS_REG(rhport); + dcd_reg->DEVICEADDR = (dev_addr << 25) | TU_BIT(24); + } } void dcd_remote_wakeup(uint8_t rhport) { @@ -393,7 +464,8 @@ void dcd_edpt_stall(uint8_t rhport, uint8_t ep_addr) { ci_hs_regs_t *dcd_reg = CI_HS_REG(rhport); dcd_reg->ENDPTCTRL[epnum] |= ENDPTCTRL_STALL << (dir ? 16 : 0); - // flush to abort any primed buffer + // flush to abort any primed buffer; the aborted transfer's dQH overlay can be left + // ACTIVE with mid-transfer state - qhd_start_xfer clears it before the next prime dcd_reg->ENDPTFLUSH = TU_BIT(epnum + (dir ? 16 : 0)); } @@ -463,9 +535,7 @@ bool dcd_edpt_iso_activate(uint8_t rhport, const tusb_desc_endpoint_t *desc_ep) // dcd_dcache_clean_invalidate(&_dcd_data, sizeof(dcd_data_t)); // Flush EP - const uint32_t flush_mask = TU_BIT(epnum + (dir ? 16 : 0)); - dcd_reg->ENDPTFLUSH = flush_mask; - while (dcd_reg->ENDPTFLUSH & flush_mask) {} + flush_endpoints(dcd_reg, TU_BIT(epnum + (dir ? 16 : 0))); // disable to change max packet size ep_ctrl_clear(endptctrl, dir, ENDPTCTRL_ENABLE); @@ -491,25 +561,35 @@ void dcd_edpt_close_all(uint8_t rhport) { } } -static void qhd_start_xfer(uint8_t rhport, uint8_t epnum, uint8_t dir) { +static bool qhd_start_xfer(uint8_t rhport, uint8_t epnum, uint8_t dir) { ci_hs_regs_t *dcd_reg = CI_HS_REG(rhport); dcd_qhd_t *p_qhd = &_dcd_data.qhd[epnum][dir]; dcd_qtd_t *p_qtd = &_dcd_data.qtd[epnum][dir]; p_qhd->qtd_overlay.halted = false; // clear any previous error + p_qhd->qtd_overlay.active = false; // a flushed prime leaves stale ACTIVE state; clear it so the fresh qtd loads p_qhd->qtd_overlay.next = (uint32_t)p_qtd; // link qtd to qhd // flush cache dcd_dcache_clean_invalidate(&_dcd_data, sizeof(dcd_data_t)); if (epnum == 0) { - // follows UM 24.10.8.1.1 Setup packet handling using setup lockout mechanism - // wait until ENDPTSETUPSTAT before priming data/status in response TODO add time out - while (dcd_reg->ENDPTSETUPSTAT & TU_BIT(0)) {} + // Setup lockout (IMXRT1060RM 42.5.6.4.2.1 Setup Phase, p.2403): never prime EP0 while a new + // SETUP is pending. The ISR + // normally consumes ENDPTSETUPSTAT quickly; if the guard trips, fail the transfer so usbd + // releases the endpoint (a pending SETUP supersedes this response anyway; without one, usbd + // stalls EP0 and the host recovers with a fresh control transfer). + uint32_t guard = CI_HS_BUSY_SPIN; + while (dcd_reg->ENDPTSETUPSTAT & TU_BIT(0)) { + if (!guard--) { + return false; + } + } } // start transfer dcd_reg->ENDPTPRIME = TU_BIT(epnum + (dir ? 16 : 0)); + return true; } bool dcd_edpt_xfer(uint8_t rhport, uint8_t ep_addr, uint8_t *buffer, uint16_t total_bytes, bool is_isr) { @@ -525,9 +605,7 @@ bool dcd_edpt_xfer(uint8_t rhport, uint8_t ep_addr, uint8_t *buffer, uint16_t to // Start qhd transfer p_qhd->ff = NULL; - qhd_start_xfer(rhport, epnum, dir); - - return true; + return qhd_start_xfer(rhport, epnum, dir); } #if !CFG_TUD_MEM_DCACHE_ENABLE @@ -578,9 +656,7 @@ bool dcd_edpt_xfer_fifo(uint8_t rhport, uint8_t ep_addr, tu_fifo_t *ff, uint16_t // Start qhd transfer p_qhd->ff = ff; - qhd_start_xfer(rhport, epnum, dir); - - return true; + return qhd_start_xfer(rhport, epnum, dir); } #endif @@ -628,43 +704,43 @@ void dcd_int_handler(uint8_t rhport) { return; } - // Set if the port controller enters the full or high-speed operational state. - // either from Bus Reset or Suspended state - if (int_status & INTR_PORT_CHANGE) { - // TU_LOG2("PortChange %08lx\r\n", dcd_reg->PORTSC1); + const uint8_t pci_reason = _port_change_reason[rhport]; // save current pci_reason - // Reset interrupt is not enabled, we manually check if Port Change is due - // to connection / disconnection - if (dcd_reg->USBSTS & INTR_RESET) { - dcd_reg->USBSTS = INTR_RESET; - - if (dcd_reg->PORTSC1 & PORTSC1_CURRENT_CONNECT_STATUS) { - const uint32_t speed = (dcd_reg->PORTSC1 & PORTSC1_PORT_SPEED) >> PORTSC1_PORT_SPEED_POS; - bus_reset(rhport); - dcd_event_bus_reset(rhport, (tusb_speed_t)speed, true); - } else { - dcd_event_bus_signal(rhport, DCD_EVENT_UNPLUGGED, true); - } - } else { - // Triggered by resuming from suspended state - if (!(dcd_reg->PORTSC1 & PORTSC1_SUSPEND)) { - dcd_event_bus_signal(rhport, DCD_EVENT_RESUME, true); - } - } + if (int_status & INTR_SUSPEND) { + _port_change_reason[rhport] = PORT_CHANGE_REASON_RESUME; // next PCI is resume + dcd_event_bus_signal(rhport, DCD_EVENT_SUSPEND, true); } - if (int_status & INTR_SUSPEND) { - // TU_LOG2("Suspend %08lx\r\n", dcd_reg->PORTSC1); + // USB Reset Received: register cleanup runs here within the reset window (IMXRT1060RM 42.5.6.2.1, p.2394) + // and BUS_RESET_START fires now; BUS_RESET_END, with the final speed, is triggered later by PCI. + if (int_status & INTR_RESET) { + _port_change_reason[rhport] = PORT_CHANGE_REASON_RESET; + bus_reset_begin(rhport); + dcd_event_bus_signal(rhport, DCD_EVENT_BUS_RESET_START, true); + } - if (dcd_reg->PORTSC1 & PORTSC1_SUSPEND) { - // Note: Host may delay more than 3 ms before and/or after bus reset before doing enumeration. - // Skip suspend event if we are not addressed - if ((dcd_reg->DEVICEADDR >> 25) & 0x0f) { - dcd_event_bus_signal(rhport, DCD_EVENT_SUSPEND, true); - } + // Port entered the full/high-speed operational state: the end of a bus reset, or a resume. + if (int_status & INTR_PORT_CHANGE) { + if (pci_reason == PORT_CHANGE_REASON_RESUME) { + dcd_event_bus_signal(rhport, DCD_EVENT_RESUME, true); + } else { + // the undefined encoding falls back to full speed + const uint32_t pspd = (dcd_reg->PORTSC1 & PORTSC1_PORT_SPEED) >> PORTSC1_PORT_SPEED_POS; + const tusb_speed_t speed = (pspd == PORTSC1_PORT_SPEED_LOW) ? TUSB_SPEED_LOW : + (pspd == PORTSC1_PORT_SPEED_HIGH) ? TUSB_SPEED_HIGH : TUSB_SPEED_FULL; + dcd_event_bus_reset(rhport, speed, true); + // This reset is over, so the next port change is a resume. Leaving it at RESET instead would + // dispatch every later resume as another end-of-reset, clearing the queue heads mid-session. + _port_change_reason[rhport] = PORT_CHANGE_REASON_RESUME; } } + // No unplug detection yet, by the manual rather than by omission: IMXRT1060RM 42.7.31 (p.2470) says a zero + // Current Connect Status means the device "did not attach successfully or was forcibly + // disconnected by the software writing a zero to the Run bit ... It does not state the device + // being disconnected or suspended", so a cable pull raises no port change at all. VBUS via + // OTGSC BSV is the manual's disconnect indicator, and it is board dependent. + if (int_status & INTR_USB) { // Make sure we read the latest version of _dcd_data. dcd_dcache_clean_invalidate(&_dcd_data, sizeof(dcd_data_t)); @@ -672,7 +748,7 @@ void dcd_int_handler(uint8_t rhport) { const uint32_t edpt_complete = dcd_reg->ENDPTCOMPLETE; dcd_reg->ENDPTCOMPLETE = edpt_complete; // acknowledge - // 23.10.12.3 Failed QTD also get ENDPTCOMPLETE set + // 42.5.6.6.4 Transfer Completion (p.2413): a failed dTD also sets ENDPTCOMPLETE // nothing to do, we will submit xfer as error to usbd // if (int_status & INTR_ERROR) { } @@ -688,12 +764,39 @@ void dcd_int_handler(uint8_t rhport) { } // Set up Received - // 23.10.10.2 Operational model for setup transfers + // 42.5.6.4.2 Control Endpoint Operation Model (p.2403) // Must be after normal transfer complete since it is possible to have both previous control status + new setup // in the same frame and we should handle previous status first. if (dcd_reg->ENDPTSETUPSTAT) { + // 42.5.6.4.2.1 Setup Phase (p.2403) steps 1-2: duplicate the setup payload BEFORE clearing + // ENDPTSETUPSTAT - + // the clear releases the setup lockout and a back-to-back SETUP (usbtest case 10) can + // overwrite the queue-head buffer immediately after. The copy is read through the volatile + // qualifier rather than memcpy'd because C orders volatile accesses only against each + // other: a plain copy may legally be sunk past the lockout-releasing store below. + union { + tusb_control_request_t request; + uint8_t byte[8]; + } setup; + const volatile uint8_t *setup_src = (const volatile uint8_t *)&_dcd_data.qhd[0][0].setup_request; + for (uint8_t i = 0; i < sizeof(setup.request); i++) { + setup.byte[i] = setup_src[i]; + } dcd_reg->ENDPTSETUPSTAT = dcd_reg->ENDPTSETUPSTAT; - dcd_event_setup_received(rhport, (uint8_t *)(uintptr_t)&_dcd_data.qhd[0][0].setup_request, true); + + // Retire a status/handshake phase left primed by the previous control sequence + // (IMXRT1060RM 42.5.6.4.2.1, p.2403), which would otherwise retire the response the task is about to + // prime for this setup. Skipped when EP0 has nothing primed or priming, since the manual + // does not want the flush wait in an interrupt handler when it has nothing to do. + // One volatile read per statement: C leaves their order unspecified within a single + // expression, which IAR rejects outright (Pa082). + const uint32_t ep0_mask = TU_BIT(0) | TU_BIT(16); + const uint32_t ep0_stat = dcd_reg->ENDPTSTAT; + const uint32_t ep0_prime = dcd_reg->ENDPTPRIME; + if ((ep0_stat | ep0_prime) & ep0_mask) { + flush_endpoints(dcd_reg, ep0_mask); + } + dcd_event_setup_received(rhport, setup.byte, true); } } diff --git a/src/portable/chipidea/ci_hs/hcd_ci_hs.c b/src/portable/chipidea/ci_hs/hcd_ci_hs.c index 3cb69acfa..0f24f5bb6 100644 --- a/src/portable/chipidea/ci_hs/hcd_ci_hs.c +++ b/src/portable/chipidea/ci_hs/hcd_ci_hs.c @@ -82,6 +82,10 @@ bool hcd_init(uint8_t rhport, const tusb_rhport_init_t *rh_init) { hcd_reg->USBMODE = USBMODE_CM_HOST; #endif + #ifdef CI_HS_SET_AHB_BURST + CI_HS_SET_AHB_BURST(rhport); + #endif + #if !TUH_OPT_HIGH_SPEED hcd_reg->PORTSC1 |= PORTSC1_FORCE_FULL_SPEED; #endif diff --git a/src/portable/mentor/musb/dcd_musb.c b/src/portable/mentor/musb/dcd_musb.c index 1ebd1fe02..92c9fe92e 100644 --- a/src/portable/mentor/musb/dcd_musb.c +++ b/src/portable/mentor/musb/dcd_musb.c @@ -24,6 +24,8 @@ #include "musb_ti.h" #elif defined(TUP_USBIP_MUSB_ADI) #include "musb_max32.h" +#elif defined(TUP_USBIP_MUSB_PY32) + #include "musb_py32.h" #else #error "Unsupported MCU" #endif @@ -45,6 +47,7 @@ typedef struct { }; uint16_t length; /* the number of bytes in the buffer */ uint16_t remaining; /* the number of bytes remaining in the buffer */ + uint16_t mps; /* maximum packet size */ bool armed; /* true while a transfer is posted */ bool use_fifo; /* true: buf is tu_fifo_t*; false: buf is plain byte pointer. */ } pipe_state_t; @@ -159,6 +162,14 @@ TU_ATTR_ALWAYS_INLINE static inline pipe_state_t* pipe_get(uint8_t epnum, tusb_d return &_dcd.pipe[idx]; } +TU_ATTR_ALWAYS_INLINE static inline uint16_t musb_mps_to_maxp(uint16_t mps) { +#if defined(TUP_USBIP_MUSB_PY32) + return (uint8_t) ((mps + 7u) / 8u); +#else + return mps; +#endif +} + //-------------------------------------------------------------------- // HW FIFO Helper // Note: Index register is already set by caller @@ -223,7 +234,12 @@ TU_ATTR_ALWAYS_INLINE static inline bool hwfifo_config(musb_regs_t* musb, unsign bool double_packet) { (void) mps; - #if defined(TUP_USBIP_MUSB_ADI) + #if defined(TUP_USBIP_MUSB_PY32) + (void) musb; (void) is_rx; (void) double_packet; + // Puya FIFO sizes: EP0 = 64 B, EP1 = 512 B, EP2..4 = 128 B, EP5 = 64 B, shared between IN and OUT. + static const uint16_t py32_fifo_size[] = { 64, 512, 128, 128, 128, 64 }; + return epnum < TU_ARRAY_SIZE(py32_fifo_size) && mps <= py32_fifo_size[epnum]; + #elif defined(TUP_USBIP_MUSB_ADI) // AnalogDevice FIFO sizes: EP1..7 = 512 B, EP8..9 = 2048 B, EP10..11 = 4096 B. // DPB requires FIFO >= 2 * MPS. For HS bulk (MPS=512) only EP >= 8 qualifies. // Force single-buffered on EP < 8 even if the caller requested DPB. @@ -266,7 +282,7 @@ TU_ATTR_ALWAYS_INLINE static inline void hwfifo_flush(musb_regs_t* musb, unsigne // write to txfifo using pipe_state_t info static void pipe_write(musb_regs_t* musb_regs, pipe_state_t* pipe, uint8_t epnum) { musb_ep_csr_t* ep_csr = &musb_regs->indexed_csr; - const uint16_t mps = ep_csr->tx_maxp & MUSB_TXMAXP_PACKET_SIZE_M; + const uint16_t mps = pipe->mps; const uint16_t xact_len = tu_min16(mps, pipe->remaining); volatile void *hwfifo = &musb_regs->fifo[epnum]; if (xact_len) { @@ -292,6 +308,12 @@ static void process_epin_isr(uint8_t rhport, musb_regs_t *musb_regs, uint8_t epn } pipe_state_t* pipe = pipe_get(epnum, TUSB_DIR_IN); + // No active transfer: a halt/abort disarmed the pipe (armed=false) but may leave remaining>0. + // Do not keep loading the aborted transfer — that would re-fill the just-flushed FIFO and the + // next (re-armed) transfer's data would stack on top (host sees an oversized packet -> babble). + if (!pipe->armed) { + return; + } if (pipe->remaining > 0) { pipe_write(musb_regs, pipe, epnum); } else { @@ -314,7 +336,7 @@ static void process_epin_isr(uint8_t rhport, musb_regs_t *musb_regs, uint8_t epn // release the FIFO slot by clearing RXRDY. return true if short packet static bool pipe_read(musb_regs_t* musb_regs, pipe_state_t* pipe, uint8_t epnum) { musb_ep_csr_t* ep_csr = &musb_regs->indexed_csr; // index already set in process_epout_isr() - const uint16_t mps = ep_csr->rx_maxp & MUSB_RXMAXP_PACKET_SIZE_M; + const uint16_t mps = pipe->mps; const uint16_t rx_count = ep_csr->rx_count; const uint16_t xact_len = tu_min16(tu_min16(pipe->remaining, mps), rx_count); volatile void *hwfifo = &musb_regs->fifo[epnum]; @@ -626,7 +648,11 @@ static void process_bus_reset_isr(uint8_t rhport) { hwfifo_reset(musb, i, 0); hwfifo_reset(musb, i, 1); } +#if defined(TUP_USBIP_MUSB_PY32) + dcd_event_bus_reset(rhport, TUSB_SPEED_FULL, true); +#else dcd_event_bus_reset(rhport, (musb->power & MUSB_POWER_HSMODE) ? TUSB_SPEED_HIGH : TUSB_SPEED_FULL, true); +#endif } /*------------------------------------------------------------------ @@ -635,6 +661,11 @@ static void process_bus_reset_isr(uint8_t rhport) { #if CFG_TUSB_DEBUG >= MUSB_DEBUG static void print_musb_info(musb_regs_t* musb_regs) { +#if defined(TUP_USBIP_MUSB_PY32) + (void) musb_regs; + // musb discovery fields not present + TU_LOG1("musb py32 fixed full-speed configuration\r\n"); +#else // print version, epinfo, raminfo, config_data0, fifo_size TU_LOG1("musb version = %u.%u\r\n", musb_regs->hwvers_bit.major, musb_regs->hwvers_bit.minor); TU_LOG1("Number of endpoints: %u TX, %u RX\r\n", musb_regs->epinfo_bit.tx_ep_num, musb_regs->epinfo_bit.rx_ep_num); @@ -651,6 +682,7 @@ static void print_musb_info(musb_regs_t* musb_regs) { TU_LOG1("FIFO %u Size: TX %u RX %u\r\n", i, musb_regs->indexed_csr.fifo_size_bit.tx, musb_regs->indexed_csr.fifo_size_bit.rx); } #endif +#endif } #endif @@ -706,6 +738,18 @@ void dcd_remote_wakeup(uint8_t rhport) { musb_regs->power &= ~MUSB_POWER_RESUME; } +#if defined(TUP_USBIP_MUSB_PY32) +void dcd_connect(uint8_t rhport) +{ + (void) rhport; +} + +void dcd_disconnect(uint8_t rhport) +{ + (void) rhport; +} +#else + // Connect by enabling internal pull-up resistor on D+/D- void dcd_connect(uint8_t rhport) { @@ -721,6 +765,8 @@ void dcd_disconnect(uint8_t rhport) musb_regs->power &= ~MUSB_POWER_SOFTCONN; } +#endif + void dcd_sof_enable(uint8_t rhport, bool en) { (void) rhport; @@ -744,6 +790,7 @@ bool dcd_edpt_open(uint8_t rhport, tusb_desc_endpoint_t const * ep_desc) { pipe->buf = NULL; pipe->length = 0; pipe->remaining = 0; + pipe->mps = (uint16_t) mps; pipe->armed = false; musb_regs_t* musb = MUSB_REGS(rhport); @@ -751,7 +798,7 @@ bool dcd_edpt_open(uint8_t rhport, tusb_desc_endpoint_t const * ep_desc) { const uint8_t is_rx = (1 - epdir); musb_ep_maxp_csr_t* maxp_csr = &ep_csr->maxp_csr[is_rx]; - maxp_csr->maxp = mps; + maxp_csr->maxp = musb_mps_to_maxp((uint16_t) mps); maxp_csr->csrh = 0; #if MUSB_CFG_SHARED_FIFO if (epdir) { @@ -762,7 +809,7 @@ bool dcd_edpt_open(uint8_t rhport, tusb_desc_endpoint_t const * ep_desc) { hwfifo_flush(musb, epn, is_rx, true); TU_ASSERT(hwfifo_config(musb, epn, is_rx, mps, ep_desc->bmAttributes.xfer == TUSB_XFER_BULK)); - musb->intren_ep[is_rx] |= TU_BIT(epn); + musb->intren_ep[is_rx ^ MUSB_INTR_EP_TX_RX_SWAP] |= TU_BIT(epn); return true; } @@ -773,8 +820,11 @@ bool dcd_edpt_iso_alloc(uint8_t rhport, uint8_t ep_addr, uint16_t largest_packet musb_regs_t* musb = MUSB_REGS(rhport); musb_ep_csr_t* ep_csr = get_ep_csr(musb, epn); const uint8_t is_rx = 1 - dir_in; + pipe_state_t *pipe = pipe_get(epn, dir_in); + pipe->mps = largest_packet_size; ep_csr->maxp_csr[is_rx].csrh = 0; - return hwfifo_config(musb, epn, is_rx, largest_packet_size, true); + TU_ASSERT(hwfifo_config(musb, epn, is_rx, largest_packet_size, true)); + return true; } bool dcd_edpt_iso_activate(uint8_t rhport, tusb_desc_endpoint_t const *ep_desc ) { @@ -790,6 +840,7 @@ bool dcd_edpt_iso_activate(uint8_t rhport, tusb_desc_endpoint_t const *ep_desc ) pipe->buf = NULL; pipe->length = 0; pipe->remaining = 0; + pipe->mps = (uint16_t) mps; pipe->armed = false; musb_regs_t* musb = MUSB_REGS(rhport); @@ -797,7 +848,7 @@ bool dcd_edpt_iso_activate(uint8_t rhport, tusb_desc_endpoint_t const *ep_desc ) const uint8_t is_rx = 1 - dir_in; musb_ep_maxp_csr_t* maxp_csr = &ep_csr->maxp_csr[is_rx]; - maxp_csr->maxp = mps; + maxp_csr->maxp = musb_mps_to_maxp((uint16_t) mps); maxp_csr->csrh |= MUSB_CSRH_ISO; #if MUSB_CFG_SHARED_FIFO if (dir_in) { @@ -812,7 +863,7 @@ bool dcd_edpt_iso_activate(uint8_t rhport, tusb_desc_endpoint_t const *ep_desc ) musb->fifo_size[is_rx] = hwfifo_byte2size(mps) | MUSB_FIFOSZ_DOUBLE_PACKET; #endif - musb->intren_ep[is_rx] |= TU_BIT(epn); + musb->intren_ep[is_rx ^ MUSB_INTR_EP_TX_RX_SWAP] |= TU_BIT(epn); if (ie) musb_dcd_int_enable(rhport); @@ -833,7 +884,7 @@ void dcd_edpt_close_all(uint8_t rhport) musb_ep_maxp_csr_t* maxp_csr = &ep_csr->maxp_csr[d]; hwfifo_flush(musb, i, d, true); hwfifo_reset(musb, i, d); - maxp_csr->maxp = 0; + maxp_csr->maxp = musb_mps_to_maxp(0); maxp_csr->csrh = 0; } } @@ -910,6 +961,10 @@ void dcd_edpt_stall(uint8_t rhport, uint8_t ep_addr) { } else { const tusb_dir_t ep_dir = tu_edpt_dir(ep_addr); const uint8_t is_rx = (ep_dir == TUSB_DIR_OUT ? 1u : 0u); + // A halt aborts the transfer: flush staged FIFO packet(s) before stalling, else leftover TX data + // concatenates with the next transfer after un-halt -> host sees an oversized packet (babble). + // FLUSH must precede SEND_STALL, which clears the TXRDY that hwfifo_flush() gates on. + hwfifo_flush(musb_regs, epn, is_rx, false); ep_csr->maxp_csr[is_rx].csrl = MUSB_CSRL_SEND_STALL(is_rx); pipe_state_t* pipe = pipe_get(epn, ep_dir); pipe->armed = false; diff --git a/src/portable/mentor/musb/musb_max32.h b/src/portable/mentor/musb/musb_max32.h index 628454216..9cf99126e 100644 --- a/src/portable/mentor/musb/musb_max32.h +++ b/src/portable/mentor/musb/musb_max32.h @@ -28,6 +28,7 @@ extern "C" { #define MUSB_CFG_SHARED_FIFO 1 // shared FIFO for TX and RX endpoints #define MUSB_CFG_DYNAMIC_FIFO 0 // dynamic EP FIFO sizing +#define MUSB_INTR_EP_TX_RX_SWAP 0 static const uintptr_t MUSB_BASES[] = { MXC_BASE_USBHS }; diff --git a/src/portable/mentor/musb/musb_py32.h b/src/portable/mentor/musb/musb_py32.h new file mode 100644 index 000000000..68d58ccb8 --- /dev/null +++ b/src/portable/mentor/musb/musb_py32.h @@ -0,0 +1,79 @@ +/* + * The MIT License (MIT) + * + * Copyright (c) 2026, Ha Thach (tinyusb.org) + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + * This file is part of the TinyUSB stack. + */ + +#ifndef TUSB_MUSB_PY32_H_ +#define TUSB_MUSB_PY32_H_ + +#include "py32f0xx.h" + +#ifdef __cplusplus +extern "C" { +#endif + +// PY32 does not expose generic MUSB shared FIFO allocation registers; this +// selects the existing TX_MODE path required for IN endpoints. +#define MUSB_CFG_SHARED_FIFO 1 +#define MUSB_CFG_DYNAMIC_FIFO 0 +#define MUSB_INTR_EP_TX_RX_SWAP 1 + +static const uintptr_t MUSB_BASES[] = { USBD_BASE }; +static const IRQn_Type musb_irqs[] = { USB_IRQn }; + +TU_ATTR_ALWAYS_INLINE static inline void musb_dcd_phy_init(uint8_t rhport) { + musb_regs_t* musb_regs = MUSB_REGS(rhport); + + musb_regs->index = 0; + musb_regs->faddr = 0; + musb_regs->intr_usben = MUSB_IE_RESET | MUSB_IE_RESUME | MUSB_IE_SUSPND; + musb_regs->intr_txen = TU_BIT(0); + musb_regs->intr_rxen = 0; +} + +TU_ATTR_ALWAYS_INLINE static inline void musb_dcd_int_enable(uint8_t rhport) { + NVIC_EnableIRQ(musb_irqs[rhport]); +} + +TU_ATTR_ALWAYS_INLINE static inline void musb_dcd_int_disable(uint8_t rhport) { + NVIC_DisableIRQ(musb_irqs[rhport]); +} + +TU_ATTR_ALWAYS_INLINE static inline void musb_dcd_int_clear(uint8_t rhport) { + NVIC_ClearPendingIRQ(musb_irqs[rhport]); +} + +TU_ATTR_ALWAYS_INLINE static inline unsigned musb_dcd_get_int_enable(uint8_t rhport) { + return NVIC_GetEnableIRQ(musb_irqs[rhport]); +} + +TU_ATTR_ALWAYS_INLINE static inline void musb_dcd_int_handler_enter(uint8_t rhport) { + (void) rhport; +} + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/src/portable/mentor/musb/musb_ti.h b/src/portable/mentor/musb/musb_ti.h index db2e237d2..1cf26b1cb 100644 --- a/src/portable/mentor/musb/musb_ti.h +++ b/src/portable/mentor/musb/musb_ti.h @@ -30,6 +30,7 @@ #define MUSB_CFG_SHARED_FIFO 0 #define MUSB_CFG_DYNAMIC_FIFO 1 #define MUSB_CFG_DYNAMIC_FIFO_SIZE 4096 +#define MUSB_INTR_EP_TX_RX_SWAP 0 static const uintptr_t MUSB_BASES[] = { USB0_BASE }; diff --git a/src/portable/mentor/musb/musb_type.h b/src/portable/mentor/musb/musb_type.h index 3c079cb25..1e8a761c0 100644 --- a/src/portable/mentor/musb/musb_type.h +++ b/src/portable/mentor/musb/musb_type.h @@ -64,6 +64,75 @@ #define __R volatile const #endif +#if defined(TUP_USBIP_MUSB_PY32) + +typedef struct TU_ATTR_PACKED { + __IO uint8_t csrh; // 0x04, 0x08: CSRH + __IO uint8_t csrl; // 0x05, 0x09: CSRL + __IO uint8_t maxp; // 0x06, 0x0A: MAXP + __I uint8_t reserved; +} musb_ep_maxp_csr_t; + +TU_VERIFY_STATIC(sizeof(musb_ep_maxp_csr_t) == 4, "size is not correct"); + +typedef struct TU_ATTR_PACKED { + __IO uint8_t csr0l; // 0x00: CSR0 + __IO uint8_t count0; // 0x01: COUNT0 + __I uint8_t reserved_0x02[2]; + + union { + struct { + __IO uint8_t tx_csrh; // 0x04: TX CSRH + __IO uint8_t tx_csrl; // 0x05: TX CSRL + __IO uint8_t tx_maxp; // 0x06: TX MAXP + __I uint8_t reserved_0x07; + + __IO uint8_t rx_csrh; // 0x08: RX CSRH + __IO uint8_t rx_csrl; // 0x09: RX CSRL + __IO uint8_t rx_maxp; // 0x0A: RX MAXP + __I uint8_t reserved_0x0b; + }; + + musb_ep_maxp_csr_t maxp_csr[2]; + }; + + __IO uint16_t rx_count; // 0x0C: RX COUNT + __I uint8_t reserved_0x0e[2]; +} musb_ep_csr_t; + +TU_VERIFY_STATIC(sizeof(musb_ep_csr_t) == 16, "size is not correct"); + +typedef struct TU_ATTR_PACKED { + __IO uint8_t faddr; // 0x00: FADDR + __IO uint8_t power; // 0x01: POWER + __I uint8_t reserved_0x02[2]; + + __IO uint8_t intr_usb; // 0x04: INTRUSB + __IO uint8_t intr_rx; // 0x05: INTRRX + __IO uint8_t intr_tx; // 0x06: INTRTX + __I uint8_t reserved_0x07; + + __IO uint8_t intr_usben; // 0x08: INTRUSBEN + union { + struct { + __IO uint8_t intr_rxen; // 0x09: INTRRXEN + __IO uint8_t intr_txen; // 0x0A: INTRTXEN + }; + + __IO uint8_t intren_ep[2]; // 0x09-0x0A: RX, TX + }; + __I uint8_t reserved_0x0b; + + __IO uint16_t frame; // 0x0C: FRAME + __IO uint8_t index; // 0x0E: INDEX + __I uint8_t reserved_0x0f; + + musb_ep_csr_t indexed_csr; // 0x10-0x1F: Indexed CSR + __IO uint32_t fifo[16]; // 0x20-0x5F: FIFO 0-15 +} musb_regs_t; + +#else + typedef struct TU_ATTR_PACKED { __IO uint16_t maxp; // 0x00, 0x04: MAXP __IO uint8_t csrl; // 0x02, 0x06: CSRL @@ -277,6 +346,8 @@ typedef struct { TU_VERIFY_STATIC(sizeof(musb_regs_t) == 0x350, "size is not correct"); +#endif + //--------------------------------------------------------------------+ // Helper //--------------------------------------------------------------------+ diff --git a/src/portable/microchip/samd/dcd_samd.c b/src/portable/microchip/samd/dcd_samd.c index 32ddd3422..54ef34c8e 100644 --- a/src/portable/microchip/samd/dcd_samd.c +++ b/src/portable/microchip/samd/dcd_samd.c @@ -229,15 +229,49 @@ bool dcd_edpt_open (uint8_t rhport, tusb_desc_endpoint_t const * desc_edpt) bool dcd_edpt_iso_alloc(uint8_t rhport, uint8_t ep_addr, uint16_t largest_packet_size) { (void) rhport; - (void) ep_addr; - (void)largest_packet_size; - return false; + uint8_t const epnum = tu_edpt_number(ep_addr); + uint8_t const dir = tu_edpt_dir(ep_addr); + + // Reserve the endpoint bank with the largest packet size (persists across altsettings). The + // buffer address/count are filled per-transfer in dcd_edpt_xfer; only the SIZE bucket is fixed. + UsbDeviceDescBank* bank = &sram_registers[epnum][dir]; + uint32_t size_value = 0; + while (size_value < 7) { + if (1 << (size_value + 3) >= largest_packet_size) { + break; + } + size_value++; + } + if ( size_value == 7 && largest_packet_size > 1023 ) return false; + + bank->PCKSIZE.bit.SIZE = size_value; + return true; } bool dcd_edpt_iso_activate(uint8_t rhport, const tusb_desc_endpoint_t *desc_ep) { (void)rhport; - (void)desc_ep; - return false; + uint8_t const epnum = tu_edpt_number(desc_ep->bEndpointAddress); + uint8_t const dir = tu_edpt_dir(desc_ep->bEndpointAddress); + + // Configure and enable the ISO endpoint on altsetting selection (bank SIZE already reserved by + // dcd_edpt_iso_alloc). Mirrors the per-direction setup in dcd_edpt_open(), plus a bank scrub: + // under ISO_ALLOC the EP is never disabled on alt0 (usbd_edpt_close is a no-op), so the bank-ready + // state from the previous streaming session survives into re-activation. Leave the EP un-armed so + // a stale bank can't move a packet before dcd_edpt_xfer re-arms it (a leftover BK1RDY with a stale + // BYTE_COUNT would otherwise babble on the first IN token after re-selecting alt1). + UsbDeviceEndpoint* ep = &USB->DEVICE.DeviceEndpoint[epnum]; + if ( dir == TUSB_DIR_OUT ) { + ep->EPCFG.bit.EPTYPE0 = desc_ep->bmAttributes.xfer + 1; + ep->EPSTATUSCLR.reg = USB_DEVICE_EPSTATUSCLR_STALLRQ0 | USB_DEVICE_EPSTATUSCLR_DTGLOUT; + ep->EPSTATUSSET.reg = USB_DEVICE_EPSTATUSSET_BK0RDY; // OUT: not ready to receive until armed + ep->EPINTENSET.bit.TRCPT0 = true; + } else { + ep->EPCFG.bit.EPTYPE1 = desc_ep->bmAttributes.xfer + 1; + ep->EPSTATUSCLR.reg = USB_DEVICE_EPSTATUSCLR_STALLRQ1 | USB_DEVICE_EPSTATUSCLR_DTGLIN | + USB_DEVICE_EPSTATUSCLR_BK1RDY; // IN: clear stale "loaded" bank + ep->EPINTENSET.bit.TRCPT1 = true; + } + return true; } void dcd_edpt_close_all (uint8_t rhport) diff --git a/src/portable/nordic/nrf5x/dcd_nrf5x.c b/src/portable/nordic/nrf5x/dcd_nrf5x.c index dc85ff78e..5170e0645 100644 --- a/src/portable/nordic/nrf5x/dcd_nrf5x.c +++ b/src/portable/nordic/nrf5x/dcd_nrf5x.c @@ -122,8 +122,22 @@ TU_ATTR_ALWAYS_INLINE static inline bool is_in_isr(void) { return (SCB->ICSR & SCB_ICSR_VECTACTIVE_Msk) ? true : false; } +// Errata 199 "USBD cannot receive tasks during DMA": while an EasyDMA transfer is in progress the +// controller may drop an incoming SETUP/IN/OUT token (lost event -> stuck EP0, esp. under rapid +// back-to-back control transfers). The workaround latches an undocumented "DMA in progress" test +// register (0x40027C1C) so tokens are held instead. Gated on the anomaly being present (all +// nRF52840 revisions; absent on other nRF52 parts). Mirrors nrfx usbd_dma_pending_set/clear(). +#define NRF_USBD_ERRATA_199_REG (*((volatile uint32_t*) 0x40027C1CUL)) + // helper to start DMA static void start_dma(volatile uint32_t* reg_startep) { + // EP0STATUS / EP0RCVOUT take the EasyDMA slot but do not transfer data, so no ERRATA-199 latch. + const bool no_dma = (reg_startep == &NRF_USBD->TASKS_EP0STATUS) || (reg_startep == &NRF_USBD->TASKS_EP0RCVOUT); + + if (!no_dma && nrf52_errata_199()) { + NRF_USBD_ERRATA_199_REG = 0x00000082UL; + } + (*reg_startep) = 1; __ISB(); __DSB(); @@ -131,7 +145,7 @@ static void start_dma(volatile uint32_t* reg_startep) { // TASKS_EP0STATUS, TASKS_EP0RCVOUT seem to need EasyDMA to be available // However these don't trigger any DMA transfer and got ENDED event subsequently // Therefore dma_pending is corrected right away - if ((reg_startep == &NRF_USBD->TASKS_EP0STATUS) || (reg_startep == &NRF_USBD->TASKS_EP0RCVOUT)) { + if (no_dma) { atomic_flag_clear(&_dcd.dma_running); } } @@ -146,6 +160,10 @@ static void edpt_dma_start(volatile uint32_t* reg_startep) { // DMA is complete static void edpt_dma_end(void) { + // Clear the ERRATA-199 "DMA in progress" latch set in start_dma(). + if (nrf52_errata_199()) { + NRF_USBD_ERRATA_199_REG = 0x00000000UL; + } atomic_flag_clear(&_dcd.dma_running); } @@ -377,58 +395,52 @@ void dcd_edpt_close_all(uint8_t rhport) { dcd_int_enable(rhport); } -void dcd_edpt_close(uint8_t rhport, uint8_t ep_addr) { - (void) rhport; +bool dcd_edpt_iso_alloc(uint8_t rhport, uint8_t ep_addr, uint16_t largest_packet_size) { + (void)rhport; + (void)largest_packet_size; + // nRF ISO endpoints are hardware-fixed to EP8 and use EasyDMA, so there is no packet buffer to + // pre-allocate here; the endpoint is enabled on dcd_edpt_iso_activate(). + TU_ASSERT(tu_edpt_number(ep_addr) == EP_ISO_NUM); + return true; +} +bool dcd_edpt_iso_activate(uint8_t rhport, const tusb_desc_endpoint_t *desc_ep) { + (void)rhport; + uint8_t const ep_addr = desc_ep->bEndpointAddress; uint8_t const epnum = tu_edpt_number(ep_addr); uint8_t const dir = tu_edpt_dir(ep_addr); + TU_ASSERT(epnum == EP_ISO_NUM); - if (epnum != EP_ISO_NUM) { - // CBI - if (dir == TUSB_DIR_OUT) { - NRF_USBD->INTENCLR = TU_BIT(USBD_INTEN_ENDEPOUT0_Pos + epnum); - NRF_USBD->EPOUTEN &= ~TU_BIT(epnum); - } else { - NRF_USBD->INTENCLR = TU_BIT(USBD_INTEN_ENDEPIN0_Pos + epnum); - NRF_USBD->EPINEN &= ~TU_BIT(epnum); - } + // A transfer armed before SET_INTERFACE survives to here (this port has no dcd close); usbd has + // just reset the endpoint's claim/busy state, so drop the stale descriptor too — otherwise the + // class's next arm trips TU_ASSERT(!xfer->started) in dcd_edpt_xfer(). + _dcd.xfer[epnum][dir].started = false; + _dcd.xfer[epnum][dir].data_received = false; + _dcd.xfer[epnum][dir].iso_in_transfer_ready = false; + + _dcd.xfer[epnum][dir].mps = tu_edpt_packet_size(desc_ep); + + if (dir == TUSB_DIR_OUT) { + // SPLIT ISO buffer when the ISO IN endpoint is already active. + if (_dcd.xfer[EP_ISO_NUM][TUSB_DIR_IN].mps) NRF_USBD->ISOSPLIT = USBD_ISOSPLIT_SPLIT_HalfIN; + NRF_USBD->EVENTS_ENDISOOUT = 0; + if ((NRF_USBD->INTEN & USBD_INTEN_SOF_Msk) == 0) NRF_USBD->EVENTS_SOF = 0; + NRF_USBD->INTENSET = USBD_INTENSET_ENDISOOUT_Msk | USBD_INTENSET_SOF_Msk; + NRF_USBD->EPOUTEN |= USBD_EPOUTEN_ISOOUT_Msk; } else { - _dcd.xfer[EP_ISO_NUM][dir].mps = 0; - // ISO - if (dir == TUSB_DIR_OUT) { - NRF_USBD->INTENCLR = USBD_INTENCLR_ENDISOOUT_Msk; - NRF_USBD->EPOUTEN &= ~USBD_EPOUTEN_ISOOUT_Msk; - NRF_USBD->EVENTS_ENDISOOUT = 0; - } else { - NRF_USBD->INTENCLR = USBD_INTENCLR_ENDISOIN_Msk; - NRF_USBD->EPINEN &= ~USBD_EPINEN_ISOIN_Msk; - } - // One of the ISO endpoints closed, no need to split buffers any more. - NRF_USBD->ISOSPLIT = USBD_ISOSPLIT_SPLIT_OneDir; - // When both ISO endpoint are close there is no need for SOF any more. - if (_dcd.xfer[EP_ISO_NUM][TUSB_DIR_IN].mps + _dcd.xfer[EP_ISO_NUM][TUSB_DIR_OUT].mps == 0) - NRF_USBD->INTENCLR = USBD_INTENCLR_SOF_Msk; + NRF_USBD->EVENTS_ENDISOIN = 0; + // SPLIT ISO buffer when the ISO OUT endpoint is already active. + if (_dcd.xfer[EP_ISO_NUM][TUSB_DIR_OUT].mps) NRF_USBD->ISOSPLIT = USBD_ISOSPLIT_SPLIT_HalfIN; + if ((NRF_USBD->INTEN & USBD_INTEN_SOF_Msk) == 0) NRF_USBD->EVENTS_SOF = 0; + NRF_USBD->INTENSET = USBD_INTENSET_ENDISOIN_Msk | USBD_INTENSET_SOF_Msk; + NRF_USBD->EPINEN |= USBD_EPINEN_ISOIN_Msk; } - _dcd.xfer[epnum][dir].started = false; + __ISB(); __DSB(); + return true; } -#if 0 -bool dcd_edpt_iso_alloc(uint8_t rhport, uint8_t ep_addr, uint16_t largest_packet_size) { - (void)rhport; - (void)ep_addr; - (void)largest_packet_size; - return false; -} - -bool dcd_edpt_iso_activate(uint8_t rhport, const tusb_desc_endpoint_t *desc_ep) { - (void)rhport; - (void)desc_ep; - return false; -} -#endif - bool dcd_edpt_xfer(uint8_t rhport, uint8_t ep_addr, uint8_t* buffer, uint16_t total_bytes, bool is_isr) { (void) rhport; (void) is_isr; diff --git a/src/portable/nxp/khci/dcd_khci.c b/src/portable/nxp/khci/dcd_khci.c deleted file mode 100644 index 9f61bd236..000000000 --- a/src/portable/nxp/khci/dcd_khci.c +++ /dev/null @@ -1,560 +0,0 @@ -/* - * SPDX-FileCopyrightText: Copyright (c) 2020 Koji Kitayama - * SPDX-FileCopyrightText: Copyright (c) 2020 Ha Thach (tinyusb.org) - * SPDX-License-Identifier: MIT - * - * This file is part of the TinyUSB stack. - */ - -#include "tusb_option.h" - -#if CFG_TUD_ENABLED && defined(TUP_USBIP_CHIPIDEA_FS) - -#ifdef TUP_USBIP_CHIPIDEA_FS_KINETIS - #include "fsl_device_registers.h" - #define KHCI USB0 -#else - #error "MCU is not supported" -#endif - -#include "device/dcd.h" - -//--------------------------------------------------------------------+ -// MACRO TYPEDEF CONSTANT ENUM DECLARATION -//--------------------------------------------------------------------+ - -enum { - TOK_PID_OUT = 0x1u, - TOK_PID_IN = 0x9u, - TOK_PID_SETUP = 0xDu, -}; - -typedef struct TU_ATTR_PACKED -{ - union { - uint32_t head; - struct { - union { - struct { - uint16_t : 2; - __IO uint16_t tok_pid : 4; - uint16_t data : 1; - __IO uint16_t own : 1; - uint16_t : 8; - }; - struct { - uint16_t : 2; - uint16_t bdt_stall : 1; - uint16_t dts : 1; - uint16_t ninc : 1; - uint16_t keep : 1; - uint16_t : 10; - }; - }; - __IO uint16_t bc : 10; - uint16_t : 6; - }; - }; - uint8_t *addr; -}buffer_descriptor_t; - -TU_VERIFY_STATIC( sizeof(buffer_descriptor_t) == 8, "size is not correct" ); - -typedef struct TU_ATTR_PACKED -{ - union { - uint32_t state; - struct { - uint32_t max_packet_size :11; - uint32_t : 5; - uint32_t odd : 1; - uint32_t :15; - }; - }; - uint16_t length; - uint16_t remaining; -}endpoint_state_t; - -TU_VERIFY_STATIC( sizeof(endpoint_state_t) == 8, "size is not correct" ); - -typedef struct -{ - union { - /* [#EP][OUT,IN][EVEN,ODD] */ - buffer_descriptor_t bdt[16][2][2]; - uint16_t bda[512]; - }; - TU_ATTR_ALIGNED(4) union { - endpoint_state_t endpoint[16][2]; - endpoint_state_t endpoint_unified[16 * 2]; - }; - uint8_t setup_packet[8]; - uint8_t addr; -}dcd_data_t; - -//--------------------------------------------------------------------+ -// INTERNAL OBJECT & FUNCTION DECLARATION -//--------------------------------------------------------------------+ -// BDT(Buffer Descriptor Table) must be 256-byte aligned -CFG_TUD_MEM_SECTION TU_ATTR_ALIGNED(512) static dcd_data_t _dcd; - -TU_VERIFY_STATIC( sizeof(_dcd.bdt) == 512, "size is not correct" ); - -static void prepare_next_setup_packet(uint8_t rhport) -{ - const unsigned out_odd = _dcd.endpoint[0][0].odd; - const unsigned in_odd = _dcd.endpoint[0][1].odd; - TU_ASSERT(0 == _dcd.bdt[0][0][out_odd].own, ); - - _dcd.bdt[0][0][out_odd].data = 0; - _dcd.bdt[0][0][out_odd ^ 1].data = 1; - _dcd.bdt[0][1][in_odd].data = 1; - _dcd.bdt[0][1][in_odd ^ 1].data = 0; - dcd_edpt_xfer(rhport, tu_edpt_addr(0, TUSB_DIR_OUT), - _dcd.setup_packet, sizeof(_dcd.setup_packet), false); -} - -static void process_stall(uint8_t rhport) -{ - for (int i = 0; i < 16; ++i) { - unsigned const endpt = KHCI->ENDPOINT[i].ENDPT; - - if (endpt & USB_ENDPT_EPSTALL_MASK) { - // prepare next setup if endpoint0 - if ( i == 0 ) prepare_next_setup_packet(rhport); - - // clear stall bit - KHCI->ENDPOINT[i].ENDPT = endpt & ~USB_ENDPT_EPSTALL_MASK; - } - } -} - -static void process_tokdne(uint8_t rhport) -{ - const unsigned s = KHCI->STAT; - KHCI->ISTAT = USB_ISTAT_TOKDNE_MASK; /* fetch the next token if received */ - - uint8_t const epnum = (s >> USB_STAT_ENDP_SHIFT); - uint8_t const dir = (s & USB_STAT_TX_MASK) >> USB_STAT_TX_SHIFT; - unsigned const odd = (s & USB_STAT_ODD_MASK) ? 1 : 0; - - buffer_descriptor_t *bd = (buffer_descriptor_t *)&_dcd.bda[s]; - endpoint_state_t *ep = &_dcd.endpoint_unified[s >> 3]; - - /* fetch pid before discarded by the next steps */ - const unsigned pid = bd->tok_pid; - - /* reset values for a next transfer */ - bd->bdt_stall = 0; - bd->dts = 1; - bd->ninc = 0; - bd->keep = 0; - /* update the odd variable to prepare for the next transfer */ - ep->odd = odd ^ 1; - if (pid == TOK_PID_SETUP) { - dcd_event_setup_received(rhport, bd->addr, true); - KHCI->CTL &= ~USB_CTL_TXSUSPENDTOKENBUSY_MASK; - return; - } - - const unsigned bc = bd->bc; - const unsigned remaining = ep->remaining - bc; - if (remaining && bc == ep->max_packet_size) { - /* continue the transferring consecutive data */ - ep->remaining = remaining; - const int next_remaining = remaining - ep->max_packet_size; - if (next_remaining > 0) { - /* prepare to the after next transfer */ - bd->addr += ep->max_packet_size * 2; - bd->bc = next_remaining > ep->max_packet_size ? ep->max_packet_size: next_remaining; - __DSB(); - bd->own = 1; /* the own bit must set after addr */ - } - return; - } - const unsigned length = ep->length; - dcd_event_xfer_complete(rhport, - tu_edpt_addr(epnum, dir), - length - remaining, XFER_RESULT_SUCCESS, true); - if (0 == epnum && 0 == length) { - /* After completion a ZLP of control transfer, - * it prepares for the next steup transfer. */ - if (_dcd.addr) { - /* When the transfer was the SetAddress, - * the device address should be updated here. */ - KHCI->ADDR = _dcd.addr; - _dcd.addr = 0; - } - prepare_next_setup_packet(rhport); - } -} - -static void process_bus_reset(uint8_t rhport) -{ - KHCI->USBCTRL &= ~USB_USBCTRL_SUSP_MASK; - KHCI->CTL |= USB_CTL_ODDRST_MASK; - KHCI->ADDR = 0; - KHCI->INTEN = USB_INTEN_USBRSTEN_MASK | USB_INTEN_TOKDNEEN_MASK | USB_INTEN_SLEEPEN_MASK | - USB_INTEN_ERROREN_MASK | USB_INTEN_STALLEN_MASK; - - KHCI->ENDPOINT[0].ENDPT = USB_ENDPT_EPHSHK_MASK | USB_ENDPT_EPRXEN_MASK | USB_ENDPT_EPTXEN_MASK; - for (unsigned i = 1; i < 16; ++i) { - KHCI->ENDPOINT[i].ENDPT = 0; - } - buffer_descriptor_t *bd = _dcd.bdt[0][0]; - for (unsigned i = 0; i < sizeof(_dcd.bdt)/sizeof(*bd); ++i, ++bd) { - bd->head = 0; - } - const endpoint_state_t ep0 = { - .max_packet_size = CFG_TUD_ENDPOINT0_SIZE, - .odd = 0, - .length = 0, - .remaining = 0, - }; - _dcd.endpoint[0][0] = ep0; - _dcd.endpoint[0][1] = ep0; - tu_memclr(_dcd.endpoint[1], sizeof(_dcd.endpoint) - sizeof(_dcd.endpoint[0])); - _dcd.addr = 0; - prepare_next_setup_packet(rhport); - KHCI->CTL &= ~USB_CTL_ODDRST_MASK; - dcd_event_bus_reset(rhport, TUSB_SPEED_FULL, true); -} - -static void process_bus_sleep(uint8_t rhport) -{ - // Enable resume & disable suspend interrupt - const unsigned inten = KHCI->INTEN; - - KHCI->INTEN = (inten & ~USB_INTEN_SLEEPEN_MASK) | USB_INTEN_RESUMEEN_MASK; - KHCI->USBTRC0 |= USB_USBTRC0_USBRESMEN_MASK; - KHCI->USBCTRL |= USB_USBCTRL_SUSP_MASK; - - dcd_event_bus_signal(rhport, DCD_EVENT_SUSPEND, true); -} - -static void process_bus_resume(uint8_t rhport) -{ - // Enable suspend & disable resume interrupt - const unsigned inten = KHCI->INTEN; - - KHCI->USBCTRL &= ~USB_USBCTRL_SUSP_MASK; // will also clear USB_USBTRC0_USB_RESUME_INT_MASK - KHCI->USBTRC0 &= ~USB_USBTRC0_USBRESMEN_MASK; - KHCI->INTEN = (inten & ~USB_INTEN_RESUMEEN_MASK) | USB_INTEN_SLEEPEN_MASK; - - dcd_event_bus_signal(rhport, DCD_EVENT_RESUME, true); -} - -/*------------------------------------------------------------------*/ -/* Device API - *------------------------------------------------------------------*/ -bool dcd_init(uint8_t rhport, const tusb_rhport_init_t* rh_init) { - (void) rhport; - (void) rh_init; - - // save crystal-less setting (if available) - #if defined(FSL_FEATURE_USB_KHCI_IRC48M_MODULE_CLOCK_ENABLED) && FSL_FEATURE_USB_KHCI_IRC48M_MODULE_CLOCK_ENABLED == 1 - uint32_t clk_recover_irc_en = KHCI->CLK_RECOVER_IRC_EN; - uint32_t clk_recover_ctrl = KHCI->CLK_RECOVER_CTRL; - #endif - - KHCI->USBTRC0 |= USB_USBTRC0_USBRESET_MASK; - while (KHCI->USBTRC0 & USB_USBTRC0_USBRESET_MASK); - - // restore crystal-less setting (if available) - #if defined(FSL_FEATURE_USB_KHCI_IRC48M_MODULE_CLOCK_ENABLED) && FSL_FEATURE_USB_KHCI_IRC48M_MODULE_CLOCK_ENABLED == 1 - KHCI->CLK_RECOVER_IRC_EN = clk_recover_irc_en; - KHCI->CLK_RECOVER_CTRL |= clk_recover_ctrl; - #endif - - tu_memclr(&_dcd, sizeof(_dcd)); - KHCI->USBTRC0 |= TU_BIT(6); /* software must set this bit to 1 */ - KHCI->BDTPAGE1 = (uint8_t)((uintptr_t)_dcd.bdt >> 8); - KHCI->BDTPAGE2 = (uint8_t)((uintptr_t)_dcd.bdt >> 16); - KHCI->BDTPAGE3 = (uint8_t)((uintptr_t)_dcd.bdt >> 24); - - KHCI->INTEN = USB_INTEN_USBRSTEN_MASK; - - dcd_connect(rhport); - NVIC_ClearPendingIRQ(USB0_IRQn); - - return true; -} - -void dcd_int_enable(uint8_t rhport) -{ - (void) rhport; - NVIC_EnableIRQ(USB0_IRQn); -} - -void dcd_int_disable(uint8_t rhport) -{ - (void) rhport; - NVIC_DisableIRQ(USB0_IRQn); -} - -void dcd_set_address(uint8_t rhport, uint8_t dev_addr) -{ - _dcd.addr = dev_addr & 0x7F; - /* Response with status first before changing device address */ - dcd_edpt_xfer(rhport, tu_edpt_addr(0, TUSB_DIR_IN), NULL, 0, false); -} - -void dcd_remote_wakeup(uint8_t rhport) -{ - (void) rhport; - - KHCI->CTL |= USB_CTL_RESUME_MASK; - - unsigned cnt = SystemCoreClock / 1000; - while (cnt--) __NOP(); - - KHCI->CTL &= ~USB_CTL_RESUME_MASK; -} - -void dcd_connect(uint8_t rhport) -{ - (void) rhport; - KHCI->USBCTRL = 0; - KHCI->CONTROL |= USB_CONTROL_DPPULLUPNONOTG_MASK; - KHCI->CTL |= USB_CTL_USBENSOFEN_MASK; -} - -void dcd_disconnect(uint8_t rhport) -{ - (void) rhport; - KHCI->CTL = 0; - KHCI->CONTROL &= ~USB_CONTROL_DPPULLUPNONOTG_MASK; -} - -void dcd_sof_enable(uint8_t rhport, bool en) -{ - (void) rhport; - (void) en; - - // TODO implement later -} - -//--------------------------------------------------------------------+ -// Endpoint API -//--------------------------------------------------------------------+ -static bool edpt_open(uint8_t rhport, uint8_t ep_addr, uint16_t max_packet_size, tusb_xfer_type_t xfer) { - (void)rhport; - - const unsigned epn = tu_edpt_number(ep_addr); - const unsigned dir = tu_edpt_dir(ep_addr); - endpoint_state_t *ep = &_dcd.endpoint[epn][dir]; - const unsigned odd = ep->odd; - buffer_descriptor_t *bd = _dcd.bdt[epn][dir]; - - /* No support for control transfer */ - TU_ASSERT(epn && (xfer != TUSB_XFER_CONTROL)); - - ep->max_packet_size = max_packet_size; - unsigned val = USB_ENDPT_EPCTLDIS_MASK; - val |= (xfer != TUSB_XFER_ISOCHRONOUS) ? USB_ENDPT_EPHSHK_MASK : 0; - val |= dir ? USB_ENDPT_EPTXEN_MASK : USB_ENDPT_EPRXEN_MASK; - KHCI->ENDPOINT[epn].ENDPT |= val; - - if (xfer != TUSB_XFER_ISOCHRONOUS) { - bd[odd].dts = 1; - bd[odd].data = 0; - bd[odd ^ 1].dts = 1; - bd[odd ^ 1].data = 1; - } - - return true; -} - -bool dcd_edpt_open(uint8_t rhport, const tusb_desc_endpoint_t *ep_desc) { - return edpt_open(rhport, ep_desc->bEndpointAddress, tu_edpt_packet_size(ep_desc), ep_desc->bmAttributes.xfer); -} - -bool dcd_edpt_iso_alloc(uint8_t rhport, uint8_t ep_addr, uint16_t largest_packet_size) { - return edpt_open(rhport, ep_addr, largest_packet_size, TUSB_XFER_ISOCHRONOUS); -} - -bool dcd_edpt_iso_activate(uint8_t rhport, const tusb_desc_endpoint_t *ep_desc) { - const unsigned epn = tu_edpt_number(ep_desc->bEndpointAddress); - const unsigned dir = tu_edpt_dir(ep_desc->bEndpointAddress); - endpoint_state_t *ep = &_dcd.endpoint[epn][dir]; - - dcd_int_disable(rhport); - ep->max_packet_size = tu_edpt_packet_size(ep_desc); - dcd_int_enable(rhport); - - return true; -} - -void dcd_edpt_close_all(uint8_t rhport) -{ - (void) rhport; - const unsigned ie = NVIC_GetEnableIRQ(USB0_IRQn); - NVIC_DisableIRQ(USB0_IRQn); - for (unsigned i = 1; i < 16; ++i) { - KHCI->ENDPOINT[i].ENDPT = 0; - } - if (ie) NVIC_EnableIRQ(USB0_IRQn); - buffer_descriptor_t *bd = _dcd.bdt[1][0]; - for (unsigned i = 2; i < sizeof(_dcd.bdt)/sizeof(*bd); ++i, ++bd) { - bd->head = 0; - } - endpoint_state_t *ep = &_dcd.endpoint[1][0]; - for (unsigned i = 2; i < sizeof(_dcd.endpoint)/sizeof(*ep); ++i, ++ep) { - /* Clear except the odd */ - ep->max_packet_size = 0; - ep->length = 0; - ep->remaining = 0; - } -} - -bool dcd_edpt_xfer(uint8_t rhport, uint8_t ep_addr, uint8_t * buffer, uint16_t total_bytes, bool is_isr) -{ - (void) rhport; - (void) is_isr; - const unsigned epn = tu_edpt_number(ep_addr); - const unsigned dir = tu_edpt_dir(ep_addr); - endpoint_state_t *ep = &_dcd.endpoint[epn][dir]; - buffer_descriptor_t *bd = &_dcd.bdt[epn][dir][ep->odd]; - TU_ASSERT(0 == bd->own); - - const unsigned ie = NVIC_GetEnableIRQ(USB0_IRQn); - NVIC_DisableIRQ(USB0_IRQn); - - ep->length = total_bytes; - ep->remaining = total_bytes; - - const unsigned mps = ep->max_packet_size; - if (total_bytes > mps) { - buffer_descriptor_t *next = ep->odd ? bd - 1: bd + 1; - /* When total_bytes is greater than the max packet size, - * it prepares to the next transfer to avoid NAK in advance. */ - next->bc = total_bytes >= 2 * mps ? mps: total_bytes - mps; - next->addr = buffer + mps; - next->own = 1; - } - bd->bc = total_bytes >= mps ? mps: total_bytes; - bd->addr = buffer; - __DSB(); - bd->own = 1; /* This bit must be set last */ - - if (ie) NVIC_EnableIRQ(USB0_IRQn); - return true; -} - -void dcd_edpt_stall(uint8_t rhport, uint8_t ep_addr) -{ - (void) rhport; - const unsigned epn = tu_edpt_number(ep_addr); - - if (0 == epn) { - KHCI->ENDPOINT[epn].ENDPT |= USB_ENDPT_EPSTALL_MASK; - } else { - const unsigned dir = tu_edpt_dir(ep_addr); - const unsigned odd = _dcd.endpoint[epn][dir].odd; - buffer_descriptor_t *bd = &_dcd.bdt[epn][dir][odd]; - TU_ASSERT(0 == bd->own,); - - const unsigned ie = NVIC_GetEnableIRQ(USB0_IRQn); - NVIC_DisableIRQ(USB0_IRQn); - - bd->bdt_stall = 1; - __DSB(); - bd->own = 1; /* This bit must be set last */ - - if (ie) NVIC_EnableIRQ(USB0_IRQn); - } -} - -void dcd_edpt_clear_stall(uint8_t rhport, uint8_t ep_addr) -{ - (void) rhport; - const unsigned epn = tu_edpt_number(ep_addr); - TU_VERIFY(epn,); - const unsigned dir = tu_edpt_dir(ep_addr); - const unsigned odd = _dcd.endpoint[epn][dir].odd; - buffer_descriptor_t *bd = _dcd.bdt[epn][dir]; - TU_VERIFY(bd[odd].own,); - - const unsigned ie = NVIC_GetEnableIRQ(USB0_IRQn); - NVIC_DisableIRQ(USB0_IRQn); - - bd[odd].own = 0; - __DSB(); - - // clear stall - bd[odd].bdt_stall = 0; - - // Reset data toggle - bd[odd ].data = 0; - bd[odd ^ 1].data = 1; - - // We already cleared this in ISR, but just clear it here to be safe - const unsigned endpt = KHCI->ENDPOINT[epn].ENDPT; - if (endpt & USB_ENDPT_EPSTALL_MASK) { - KHCI->ENDPOINT[epn].ENDPT = endpt & ~USB_ENDPT_EPSTALL_MASK; - } - - if (ie) NVIC_EnableIRQ(USB0_IRQn); -} - -//--------------------------------------------------------------------+ -// ISR -//--------------------------------------------------------------------+ -void dcd_int_handler(uint8_t rhport) -{ - uint32_t is = KHCI->ISTAT; - uint32_t msk = KHCI->INTEN; - - // clear non-enabled interrupts - KHCI->ISTAT = is & ~msk; - is &= msk; - - if (is & USB_ISTAT_ERROR_MASK) { - /* TODO: */ - uint32_t es = KHCI->ERRSTAT; - KHCI->ERRSTAT = es; - KHCI->ISTAT = is; /* discard any pending events */ - } - - if (is & USB_ISTAT_USBRST_MASK) { - KHCI->ISTAT = is; /* discard any pending events */ - process_bus_reset(rhport); - } - - if (is & USB_ISTAT_SLEEP_MASK) { - // TU_LOG3("Suspend: "); TU_LOG2_HEX(is); - - // Note Host usually has extra delay after bus reset (without SOF), which could falsely - // detected as Sleep event. Though usbd has debouncing logic so we are good - KHCI->ISTAT = USB_ISTAT_SLEEP_MASK; - process_bus_sleep(rhport); - } - -#if 0 // ISTAT_RESUME never trigger, probably for host mode ? - if (is & USB_ISTAT_RESUME_MASK) { - // TU_LOG2("ISTAT Resume: "); TU_LOG2_HEX(is); - KHCI->ISTAT = USB_ISTAT_RESUME_MASK; - process_bus_resume(rhport); - } -#endif - - if (KHCI->USBTRC0 & USB_USBTRC0_USB_RESUME_INT_MASK) { - // TU_LOG2("USBTRC0 Resume: "); TU_LOG2_HEX(is); TU_LOG2_HEX(KHCI->USBTRC0); - process_bus_resume(rhport); - } - - if (is & USB_ISTAT_SOFTOK_MASK) { - KHCI->ISTAT = USB_ISTAT_SOFTOK_MASK; - dcd_event_sof(rhport, tu_u16(KHCI->FRMNUMH, KHCI->FRMNUML), true); - } - - if (is & USB_ISTAT_STALL_MASK) { - KHCI->ISTAT = USB_ISTAT_STALL_MASK; - process_stall(rhport); - } - - if (is & USB_ISTAT_TOKDNE_MASK) { - process_tokdne(rhport); - } -} -#endif diff --git a/src/portable/nxp/lpc17_40/dcd_lpc17_40.c b/src/portable/nxp/lpc17_40/dcd_lpc17_40.c index 182710016..b577d0e9f 100644 --- a/src/portable/nxp/lpc17_40/dcd_lpc17_40.c +++ b/src/portable/nxp/lpc17_40/dcd_lpc17_40.c @@ -19,6 +19,12 @@ //--------------------------------------------------------------------+ #define DCD_ENDPOINT_MAX 32 +// The iso machinery (5th DD word + packet-size memory) costs USB RAM on every build; +// compile it only when a class that can open an iso endpoint is enabled. Keep this in +// sync with the classes that actually arm an iso endpoint: audio, video, BTH (voice), +// and vendor (its optional CFG_TUD_VENDOR_EP_ISO_* endpoints, exercised by usbtest). +#define DCD_ISO_ENABLED (CFG_TUD_AUDIO || CFG_TUD_VIDEO || CFG_TUD_VENDOR || CFG_TUD_BTH) + typedef struct TU_ATTR_ALIGNED(4) { //------------- Word 0 -------------// @@ -48,11 +54,37 @@ typedef struct TU_ATTR_ALIGNED(4) volatile uint16_t present_count; // For non-iso : The number of bytes transferred by the DMA engine // For iso : number of packets +#if DCD_ISO_ENABLED //------------- Word 4 -------------// - // uint32_t iso_packet_size_addr; // iso only, can be omitted for non-iso + volatile uint32_t iso_packet_size_addr; // iso only: pointer into iso packet-size memory, + // advanced by hardware after each packet +#endif }dma_desc_t; -TU_VERIFY_STATIC( sizeof(dma_desc_t) == 16, "size is not correct"); // TODO not support ISO for now +TU_VERIFY_STATIC( sizeof(dma_desc_t) == (DCD_ISO_ENABLED ? 20 : 16), "size is not correct"); + +// Hardware fixes endpoint type by number: 3, 6, 9, 12 are the iso-capable ones. +// Constant per ep_id (= 2*epnum + dir) — unlike dd->isochronous, which dcd_edpt_xfer +// transiently zeroes while rebuilding the DD, this is safe to dispatch on from the ISR. +// TU_ATTR_UNUSED: every caller is under #if DCD_ISO_ENABLED, so non-iso builds don't +// reference it and clang -Wunused-function (fatal) would otherwise reject the build. +TU_ATTR_UNUSED TU_ATTR_ALWAYS_INLINE static inline bool ep_id_is_iso(uint8_t ep_id) { + uint8_t const epnum = (uint8_t)(ep_id >> 1); + return (epnum % 3) == 0 && (epnum != 0) && (epnum != 15); +} + +#if DCD_ISO_ENABLED +// Isochronous packet-size memory (UM10562 12.15.6.3): one word per packet. +// IN : software fills Packet_length (bits 15:0), 0 = ZLP +// OUT: hardware writes Frame_number (31:17) | Packet_valid (16) | Packet_length (15:0) +// Iso-capable endpoint numbers are 3, 6, 9, 12 -> 8 slots (x2 directions). +// One packet moves per FRAME, so a deep queue only adds latency: 8 frames is plenty. +#define ISO_MAX_PACKETS 8 +#define ISO_SLOT_COUNT 8 +TU_ATTR_ALWAYS_INLINE static inline uint8_t iso_slot(uint8_t ep_id) { + return (uint8_t)(((ep_id / 6) - 1) * 2 + (ep_id & 1)); // ep_id = 2*epnum + dir, epnum in {3,6,9,12} +} +#endif typedef struct { @@ -66,11 +98,17 @@ typedef struct { uint8_t* out_buffer; uint8_t out_bytes; + volatile bool out_queued; // an OUT xfer is queued; out_buffer may legitimately be NULL (status ZLP) volatile bool out_received; // indicate if data is already received in endpoint uint8_t in_bytes; } control; +#if DCD_ISO_ENABLED + // iso packet-size memory, must be DMA-reachable like the DDs + volatile uint32_t iso_psize[ISO_SLOT_COUNT][ISO_MAX_PACKETS]; +#endif + } dcd_data_t; CFG_TUD_MEM_SECTION TU_ATTR_ALIGNED(128) static dcd_data_t _dcd; @@ -79,6 +117,29 @@ CFG_TUD_MEM_SECTION TU_ATTR_ALIGNED(128) static dcd_data_t _dcd; //--------------------------------------------------------------------+ // SIE Command //--------------------------------------------------------------------+ + +// The SIE command protocol (CmdCode + CCEMPTY/CDFULL handshake) and the +// slave-mode Ctrl/RxData/TxData registers are shared between thread-mode API +// calls and dcd_int_handler, and are not reentrant: an ISR preempting a +// thread-mode SIE sequence consumes its handshake flags and overwrites +// CmdCode (symptom: EP0 wedges/answers stale data right after SET_INTERFACE +// stall/clear-stall bursts overlapping bulk EOT interrupts). Mask only the +// USB interrupt around those sequences; safe to nest, including from the ISR. +static inline bool usb_irq_lock(void) +{ + bool const enabled = NVIC_GetEnableIRQ(USB_IRQn) != 0; + if (enabled) + { + NVIC_DisableIRQ(USB_IRQn); // CMSIS already ends this with DSB+ISB + } + return enabled; +} + +static inline void usb_irq_unlock(bool enabled) +{ + if (enabled) NVIC_EnableIRQ(USB_IRQn); +} + static void sie_cmd_code (sie_cmdphase_t phase, uint8_t code_data) { LPC_USB->DevIntClr = (DEV_INT_COMMAND_CODE_EMPTY_MASK | DEV_INT_COMMAND_DATA_FULL_MASK); @@ -92,19 +153,28 @@ static void sie_cmd_code (sie_cmdphase_t phase, uint8_t code_data) static void sie_write (uint8_t cmd_code, uint8_t data_len, uint8_t data) { + bool const lock = usb_irq_lock(); + sie_cmd_code(SIE_CMDPHASE_COMMAND, cmd_code); if (data_len) { sie_cmd_code(SIE_CMDPHASE_WRITE, data); } + + usb_irq_unlock(lock); } static uint8_t sie_read (uint8_t cmd_code) { + bool const lock = usb_irq_lock(); + sie_cmd_code(SIE_CMDPHASE_COMMAND , cmd_code); sie_cmd_code(SIE_CMDPHASE_READ , cmd_code); - return (uint8_t) LPC_USB->CmdData; + uint8_t const data = (uint8_t) LPC_USB->CmdData; + + usb_irq_unlock(lock); + return data; } //--------------------------------------------------------------------+ @@ -117,6 +187,11 @@ static inline uint8_t ep_addr2idx(uint8_t ep_addr) static void set_ep_size(uint8_t ep_id, uint16_t max_packet_size) { + // ReEp RMW + the EP_RLZED handshake share DevIntSt with the ISR: a bus reset + // from dcd_int_handler writes DevIntClr = 0xFFFFFFFF and would consume the + // flag this spin waits on, hanging it forever -> same lock as the SIE paths. + bool const lock = usb_irq_lock(); + // follows example in 11.10.4.2 LPC_USB->ReEp |= TU_BIT(ep_id); LPC_USB->EpInd = ep_id; // select index before setting packet size @@ -124,6 +199,8 @@ static void set_ep_size(uint8_t ep_id, uint16_t max_packet_size) while ((LPC_USB->DevIntSt & DEV_INT_ENDPOINT_REALIZED_MASK) == 0) {} LPC_USB->DevIntClr = DEV_INT_ENDPOINT_REALIZED_MASK; + + usb_irq_unlock(lock); } @@ -230,6 +307,7 @@ static inline uint8_t byte2dword(uint8_t bytes) static void control_ep_write(void const * buffer, uint8_t len) { uint32_t const * buf32 = (uint32_t const *) buffer; + bool const lock = usb_irq_lock(); // Ctrl/TxData + SIE sequence must not interleave with the ISR LPC_USB->Ctrl = USBCTRL_WRITE_ENABLE_MASK; // logical endpoint = 0 LPC_USB->TxPLen = (uint32_t) len; @@ -245,10 +323,14 @@ static void control_ep_write(void const * buffer, uint8_t len) // select control IN & validate the endpoint sie_write(SIE_CMDCODE_ENDPOINT_SELECT+1, 0, 0); sie_write(SIE_CMDCODE_BUFFER_VALIDATE , 0, 0); + + usb_irq_unlock(lock); } static uint8_t control_ep_read(void * buffer, uint8_t len) { + bool const lock = usb_irq_lock(); // Ctrl/RxData + SIE sequence must not interleave with the ISR + LPC_USB->Ctrl = USBCTRL_READ_ENABLE_MASK; // logical endpoint = 0 while ((LPC_USB->RxPLen & USBRXPLEN_PACKET_READY_MASK) == 0) {} // TODO blocking, should have timeout @@ -267,6 +349,7 @@ static uint8_t control_ep_read(void * buffer, uint8_t len) sie_write(SIE_CMDCODE_ENDPOINT_SELECT+0, 0, 0); sie_write(SIE_CMDCODE_BUFFER_CLEAR , 0, 0); + usb_irq_unlock(lock); return len; } @@ -281,8 +364,9 @@ bool dcd_edpt_open(uint8_t rhport, tusb_desc_endpoint_t const * p_endpoint_desc) uint8_t const epnum = tu_edpt_number(p_endpoint_desc->bEndpointAddress); uint8_t const ep_id = ep_addr2idx(p_endpoint_desc->bEndpointAddress); - // Endpoint type is fixed to endpoint number - // 1: interrupt, 2: Bulk, 3: Iso and so on + // Endpoint type is fixed to endpoint number (1 interrupt, 2 bulk, 3 iso, ...). + // Iso endpoints are armed via dcd_edpt_iso_alloc/activate, never through here + // (TUP_DCD_EDPT_ISO_ALLOC is defined for this IP), so only bulk/interrupt land here. switch ( p_endpoint_desc->bmAttributes.xfer ) { case TUSB_XFER_INTERRUPT: @@ -293,10 +377,6 @@ bool dcd_edpt_open(uint8_t rhport, tusb_desc_endpoint_t const * p_endpoint_desc) TU_ASSERT((epnum % 3) == 2 || (epnum == 15)); break; - case TUSB_XFER_ISOCHRONOUS: - TU_ASSERT((epnum % 3) == 0 && (epnum != 0) && (epnum != 15)); - break; - default: break; } @@ -307,9 +387,7 @@ bool dcd_edpt_open(uint8_t rhport, tusb_desc_endpoint_t const * p_endpoint_desc) //------------- first DD prepare -------------// dma_desc_t* const dd = &_dcd.dd[ep_id]; - tu_memclr(dd, sizeof(dma_desc_t)); - - dd->isochronous = (p_endpoint_desc->bmAttributes.xfer == TUSB_XFER_ISOCHRONOUS) ? 1 : 0; + tu_memclr(dd, sizeof(dma_desc_t)); // non-iso: isochronous stays 0 dd->max_packet_size = ep_size; dd->retired = 1; // invalid at first @@ -319,16 +397,54 @@ bool dcd_edpt_open(uint8_t rhport, tusb_desc_endpoint_t const * p_endpoint_desc) } bool dcd_edpt_iso_alloc(uint8_t rhport, uint8_t ep_addr, uint16_t largest_packet_size) { +#if DCD_ISO_ENABLED (void)rhport; - (void)ep_addr; - (void)largest_packet_size; + uint8_t const ep_id = ep_addr2idx(ep_addr); + + // hardware fixes iso to endpoint numbers 3, 6, 9, 12 + TU_ASSERT(ep_id_is_iso(ep_id)); + TU_ASSERT(largest_packet_size > 0); + + set_ep_size(ep_id, largest_packet_size); + + dma_desc_t* const dd = &_dcd.dd[ep_id]; + tu_memclr(dd, sizeof(dma_desc_t)); + dd->isochronous = 1; + dd->max_packet_size = largest_packet_size; + dd->retired = 1; // invalid at first + + sie_write(SIE_CMDCODE_ENDPOINT_SET_STATUS + ep_id, 1, 0); + return true; +#else + (void)rhport; (void)ep_addr; (void)largest_packet_size; return false; +#endif } bool dcd_edpt_iso_activate(uint8_t rhport, const tusb_desc_endpoint_t *desc_ep) { +#if DCD_ISO_ENABLED (void)rhport; - (void)desc_ep; + uint8_t const ep_id = ep_addr2idx(desc_ep->bEndpointAddress); + dma_desc_t* const dd = &_dcd.dd[ep_id]; + + // same fixed-number rule as alloc: without it a rejected-but-ignored alloc (classes + // discard that return) would set isochronous on a non-iso ep_id and underflow iso_slot() + TU_ASSERT(ep_id_is_iso(ep_id)); + + // kill any armed transfer from a previous alternate setting + LPC_USB->EpDMADis = TU_BIT(ep_id); + _dcd.udca[ep_id] = NULL; + + dd->isochronous = 1; + dd->max_packet_size = tu_edpt_packet_size(desc_ep); + dd->retired = 1; + + sie_write(SIE_CMDCODE_ENDPOINT_SET_STATUS + ep_id, 1, 0); + return true; +#else + (void)rhport; (void)desc_ep; return false; +#endif } void dcd_edpt_close_all (uint8_t rhport) @@ -369,19 +485,28 @@ static bool control_xact(uint8_t rhport, uint8_t dir, uint8_t * buffer, uint8_t control_ep_write(buffer, len); }else { + // guard the out_received/out_buffer handshake against the EP0 OUT ISR + bool const lock = usb_irq_lock(); + if ( _dcd.control.out_received ) { // Already received the DATA OUT packet _dcd.control.out_received = false; - _dcd.control.out_buffer = NULL; - _dcd.control.out_bytes = 0; uint8_t received = control_ep_read(buffer, len); + // event queued with in_isr=true, which skips the queue's own locking: keep the + // USB IRQ masked across it, or a real ISR completion could interleave the write dcd_event_xfer_complete(0, 0, received, XFER_RESULT_SUCCESS, true); + usb_irq_unlock(lock); }else { + // buffer is NULL for a status-stage ZLP: signal the pending xfer explicitly, + // NOT via out_buffer != NULL — a NULL-buffer queue mistaken for "nothing queued" + // leaves out_received stale and poisons the next control OUT data stage. _dcd.control.out_buffer = buffer; _dcd.control.out_bytes = len; + _dcd.control.out_queued = true; + usb_irq_unlock(lock); } } @@ -406,26 +531,68 @@ bool dcd_edpt_xfer(uint8_t rhport, uint8_t ep_addr, uint8_t * buffer, uint16_t t uint16_t const ep_size = dd->max_packet_size; uint8_t is_iso = dd->isochronous; - tu_memclr(dd, sizeof(dma_desc_t)); - dd->isochronous = is_iso; - dd->max_packet_size = ep_size; - dd->buffer = (uint32_t) buffer; - dd->buflen = total_bytes; +#if DCD_ISO_ENABLED + if ( is_iso ) + { + // iso: buflen counts packets; per-packet sizes live in the packet-size memory. + // One packet moves per frame (UM10562 12.15.6: DMA request is raised for + // DMA-enabled iso endpoints on every FRAME interrupt, both directions). + // Validate BEFORE touching the DD: bailing out mid-rebuild would leave a + // zeroed (retired=0 -> serviceable) descriptor armed for the frame engine. + TU_ASSERT(ep_size > 0); + uint16_t const packets = (total_bytes > 0) ? (uint16_t) tu_div_ceil(total_bytes, ep_size) : 1; + TU_ASSERT(packets <= ISO_MAX_PACKETS); + + uint8_t const slot = iso_slot(ep_id); + uint16_t remain = total_bytes; + for ( uint16_t i = 0; i < packets; i++ ) + { + uint16_t const pkt_len = tu_min16(remain, ep_size); + // IN: length to send (0 = ZLP). OUT: hardware writes back + // Frame_number|Packet_valid|Packet_length -- prefill 0 so a frame the + // hardware never wrote (missed/invalid) cannot read back as data. + _dcd.iso_psize[slot][i] = (ep_id & 1) ? pkt_len : 0; + remain = (uint16_t)(remain - pkt_len); + } - _dcd.udca[ep_id] = dd; + tu_memclr(dd, sizeof(dma_desc_t)); + dd->isochronous = 1; + dd->max_packet_size = ep_size; + dd->buffer = (uint32_t) buffer; + dd->buflen = packets; + dd->iso_packet_size_addr = (uint32_t) &_dcd.iso_psize[slot][0]; - if ( ep_id % 2 ) + _dcd.udca[ep_id] = dd; + LPC_USB->EpDMAEn = TU_BIT(ep_id); // frame-triggered: no DMARSet, no EpIntEn + } + else +#else + (void) is_iso; +#endif { - // Clear EP interrupt before Enable DMA - LPC_USB->EpIntEn &= ~TU_BIT(ep_id); - LPC_USB->EpDMAEn = TU_BIT(ep_id); + tu_memclr(dd, sizeof(dma_desc_t)); + dd->max_packet_size = ep_size; + dd->buffer = (uint32_t) buffer; + dd->buflen = total_bytes; - // endpoint IN need to actively raise DMA request - LPC_USB->DMARSet = TU_BIT(ep_id); - }else - { - // Enable DMA - LPC_USB->EpDMAEn = TU_BIT(ep_id); + _dcd.udca[ep_id] = dd; + + if ( ep_id % 2 ) + { + // Clear EP interrupt before Enable DMA + // EpIntEn read-modify-write races the ISR's own RMWs -> lock + bool const lock = usb_irq_lock(); + LPC_USB->EpIntEn &= ~TU_BIT(ep_id); + LPC_USB->EpDMAEn = TU_BIT(ep_id); + usb_irq_unlock(lock); + + // endpoint IN need to actively raise DMA request + LPC_USB->DMARSet = TU_BIT(ep_id); + }else + { + // Enable DMA + LPC_USB->EpDMAEn = TU_BIT(ep_id); + } } return true; @@ -451,13 +618,20 @@ static void control_xfer_isr(uint8_t rhport, uint32_t ep_int_status) uint8_t setup_packet[8]; control_ep_read(setup_packet, 8); // TODO read before clear setup above + // a new SETUP voids any half-finished control state + _dcd.control.out_queued = false; + _dcd.control.out_received = false; + _dcd.control.out_buffer = NULL; + _dcd.control.out_bytes = 0; + dcd_event_setup_received(rhport, setup_packet, true); } - else if ( _dcd.control.out_buffer ) + else if ( _dcd.control.out_queued ) { - // software queued transfer previously + // software queued transfer previously (out_buffer NULL = status ZLP) uint8_t received = control_ep_read(_dcd.control.out_buffer, _dcd.control.out_bytes); + _dcd.control.out_queued = false; _dcd.control.out_buffer = NULL; _dcd.control.out_bytes = 0; @@ -513,7 +687,32 @@ static void dd_complete_isr(uint8_t rhport, uint8_t ep_id) uint8_t result = (dd->status == DD_STATUS_NORMAL || dd->status == DD_STATUS_DATA_UNDERUN) ? XFER_RESULT_SUCCESS : XFER_RESULT_FAILED; uint8_t const ep_addr = (ep_id / 2) | ((ep_id & 0x01) ? TUSB_DIR_IN_MASK : 0); - dcd_event_xfer_complete(rhport, ep_addr, dd->present_count, result, true); + uint32_t xferred_bytes; +#if DCD_ISO_ENABLED + if ( ep_id_is_iso(ep_id) ) + { + // present_count is in packets; actual byte counts are in the packet-size memory + // (IN: as programmed by us, OUT: Packet_length written back by hardware, + // guarded by Packet_valid -- a frame with no packet must count as 0) + uint8_t const slot = iso_slot(ep_id); + uint16_t const packets = tu_min16(dd->present_count, ISO_MAX_PACKETS); + xferred_bytes = 0; + for (uint16_t i = 0; i < packets; i++) + { + uint32_t const psize = _dcd.iso_psize[slot][i]; + if ( (ep_id & 1) || (psize & TU_BIT(16)) ) + { + xferred_bytes += (psize & 0xFFFFu); + } + } + } + else +#endif + { + xferred_bytes = dd->present_count; + } + + dcd_event_xfer_complete(rhport, ep_addr, (uint16_t) xferred_bytes, result, true); } // main USB IRQ handler @@ -569,6 +768,16 @@ void dcd_int_handler(uint8_t rhport) { if ( tu_bit_test(eot, ep_id) ) { + // dispatch on the hardware's fixed ep-number/type map, NOT dd->isochronous: + // thread-mode dcd_edpt_xfer transiently zeroes the DD while rebuilding it +#if DCD_ISO_ENABLED + if ( ep_id_is_iso(ep_id) ) + { + // iso: last packet already left with its frame; complete both directions here + dd_complete_isr(rhport, ep_id); + } + else +#endif if ( ep_id & 0x01 ) { // IN enable EpInt for end of usb transfer diff --git a/src/portable/nxp/lpc_ip3511/dcd_lpc_ip3511.c b/src/portable/nxp/lpc_ip3511/dcd_lpc_ip3511.c index aa0307d25..42f6750b1 100644 --- a/src/portable/nxp/lpc_ip3511/dcd_lpc_ip3511.c +++ b/src/portable/nxp/lpc_ip3511/dcd_lpc_ip3511.c @@ -87,6 +87,10 @@ enum { DEVCMDSTAT_SUSPEND_CHANGE_MASK = TU_BIT(25), DEVCMDSTAT_RESET_CHANGE_MASK = TU_BIT(26), DEVCMDSTAT_VBUS_DEBOUNCED_MASK = TU_BIT(28), + + // write-1-to-clear latches + DEVCMDSTAT_W1C_MASK = DEVCMDSTAT_SETUP_RECEIVED_MASK | DEVCMDSTAT_CONNECT_CHANGE_MASK | + DEVCMDSTAT_SUSPEND_CHANGE_MASK | DEVCMDSTAT_RESET_CHANGE_MASK, }; enum { @@ -158,6 +162,10 @@ typedef struct // - 55 usb0 (FS) has 5x2 endpoints, usb1 (HS) has 6x2 endpoints #define MAX_EP_PAIRS 6 +// Bounded spin waiting for hardware to clear an EPSKIP bit when retiring a still-armed endpoint on +// reopen (dcd_edpt_open). Hardware clears it within a (micro)frame; the guard only avoids a hang. +#define IP3511_EPSKIP_SPIN 100000u + // NOTE data will be transferred as soon as dcd get request by dcd_pipe(_queue)_xfer using double buffering. // current_td is used to keep track of number of remaining & xferred bytes of the current request. typedef struct @@ -167,7 +175,9 @@ typedef struct ep_cmd_sts_t ep[2*MAX_EP_PAIRS][2]; xfer_dma_t dma[2*MAX_EP_PAIRS]; - TU_ATTR_ALIGNED(64) uint8_t setup_packet[8]; + // volatile: the controller DMAs a new setup packet into this buffer as soon as the SETUP + // latch is cleared, so reads of it must stay ordered against the register accesses around them + TU_ATTR_ALIGNED(64) volatile uint8_t setup_packet[8]; }dcd_data_t; // EP list must be 256-byte aligned @@ -176,8 +186,12 @@ typedef struct // Use CFG_TUD_MEM_SECTION to place it accordingly. CFG_TUD_MEM_SECTION TU_ATTR_ALIGNED(256) static dcd_data_t _dcd; -// Dummy buffer to fix ZLPs overwriting the buffer (probably an USB/DMA controller bug) -// TODO find way to save memory +// Dummy buffer to fix ZLPs overwriting the buffer: Errata LPC55S6x USB.5 / LPC55S2x USB.4 - the +// HS device controller always DMA-writes OUT data in 8-byte units, so up to 7 bytes land past the +// received length. This redirects the ZLP case; the general short-OUT case is unhandled here +// (TinyUSB's own endpoint buffers are sized/aligned so the spill stays inside them, but a tight +// caller buffer can be overrun by up to 7 bytes - the SDK's documented workaround is a bounce +// buffer). TODO find way to save memory CFG_TUD_MEM_SECTION TU_ATTR_ALIGNED(64) static uint8_t dummy[8]; //--------------------------------------------------------------------+ @@ -217,7 +231,7 @@ static const dcd_controller_t _dcd_controller[] = { // INTERNAL OBJECT & FUNCTION DECLARATION //--------------------------------------------------------------------+ -TU_ATTR_ALWAYS_INLINE static inline uint16_t get_buf_offset(void const * buffer) { +TU_ATTR_ALWAYS_INLINE static inline uint16_t get_buf_offset(void const volatile * buffer) { uint32_t addr = (uint32_t) buffer; TU_ASSERT( (addr & 0x3f) == 0, 0 ); return ( (addr >> 6) & 0xFFFFUL ) ; @@ -243,6 +257,16 @@ TU_ATTR_ALWAYS_INLINE static inline bool rhport_is_highspeed(uint8_t rhport) { return _dcd_controller[rhport].is_highspeed; } + +// DEVCMDSTAT mixes RW fields with write-1-to-clear latches (SETUP + the 3 change bits): a blind +// RMW writes a pending latch back as 1 and silently clears it (a SETUP eaten this way strands +// EP0). Mask the latches on every update; pass one in set_mask only to clear it. +TU_ATTR_ALWAYS_INLINE static inline void devcmdstat_update(dcd_registers_t* dcd_reg, + uint32_t clear_mask, uint32_t set_mask) { + const uint32_t v = dcd_reg->DEVCMDSTAT & ~(DEVCMDSTAT_W1C_MASK | clear_mask); + dcd_reg->DEVCMDSTAT = v | set_mask; +} + //--------------------------------------------------------------------+ // CONTROLLER API //--------------------------------------------------------------------+ @@ -280,8 +304,10 @@ bool dcd_init(uint8_t rhport, const tusb_rhport_init_t* rh_init) { dcd_reg->DATABUFSTART = tu_align((uint32_t) &_dcd, TU_BIT(22)); // 22-bit alignment dcd_reg->INTSTAT = dcd_reg->INTSTAT; // clear all pending interrupt dcd_reg->INTEN = INT_DEVICE_STATUS_MASK; - dcd_reg->DEVCMDSTAT |= DEVCMDSTAT_DEVICE_ENABLE_MASK | DEVCMDSTAT_DEVICE_CONNECT_MASK | - DEVCMDSTAT_RESET_CHANGE_MASK | DEVCMDSTAT_CONNECT_CHANGE_MASK | DEVCMDSTAT_SUSPEND_CHANGE_MASK; + // deliberately clear every latch (incl. a SETUP left by a bootloader/warm start) for a + // deterministic init state + devcmdstat_update(dcd_reg, 0, DEVCMDSTAT_DEVICE_ENABLE_MASK | DEVCMDSTAT_DEVICE_CONNECT_MASK | + DEVCMDSTAT_W1C_MASK); NVIC_ClearPendingIRQ(_dcd_controller[rhport].irqnum); @@ -305,8 +331,7 @@ void dcd_set_address(uint8_t rhport, uint8_t dev_addr) // Response with status first before changing device address dcd_edpt_xfer(rhport, tu_edpt_addr(0, TUSB_DIR_IN), NULL, 0, false); - dcd_reg->DEVCMDSTAT &= ~DEVCMDSTAT_DEVICE_ADDR_MASK; - dcd_reg->DEVCMDSTAT |= dev_addr; + devcmdstat_update(dcd_reg, DEVCMDSTAT_DEVICE_ADDR_MASK, dev_addr); } void dcd_remote_wakeup(uint8_t rhport) @@ -317,13 +342,13 @@ void dcd_remote_wakeup(uint8_t rhport) void dcd_connect(uint8_t rhport) { dcd_registers_t* dcd_reg = _dcd_controller[rhport].regs; - dcd_reg->DEVCMDSTAT |= DEVCMDSTAT_DEVICE_CONNECT_MASK; + devcmdstat_update(dcd_reg, 0, DEVCMDSTAT_DEVICE_CONNECT_MASK); } void dcd_disconnect(uint8_t rhport) { dcd_registers_t* dcd_reg = _dcd_controller[rhport].regs; - dcd_reg->DEVCMDSTAT &= ~DEVCMDSTAT_DEVICE_CONNECT_MASK; + devcmdstat_update(dcd_reg, DEVCMDSTAT_DEVICE_CONNECT_MASK, 0); } void dcd_sof_enable(uint8_t rhport, bool en) @@ -337,13 +362,37 @@ void dcd_sof_enable(uint8_t rhport, bool en) //--------------------------------------------------------------------+ // DCD Endpoint Port //--------------------------------------------------------------------+ +// Retire a still-armed (Active) endpoint before reconfiguring it (reopen across SET_INTERFACE). +// UM11126 §41.7.6/§41.8.3: write EPSKIP and wait for hardware to clear the bit, then Active is +// safe to clear. EPSKIP raises the endpoint interrupt as it clears Active, delivered as a +// (partial) transfer completion. Here that is sanctioned — usbd_edpt_close() documents "in +// progress transfers may be delivered after this call", and that completion is what clears the +// stale usbd busy flag (ISO_ALLOC close is a no-op) so the class can re-arm the reopened +// endpoint. NOT for the stall/iso-activate paths: there the class re-arms from the completion +// callback and the endpoint ends up Active+Stall, which never sends a STALL handshake (usbtest +// case 13 regression on LPC11u37) — those paths must clear Active directly instead. +// Bounded: hardware clears EPSKIP within a (micro)frame. +static void edpt_skip_active(uint8_t rhport, uint8_t ep_id) { + ep_cmd_sts_t* ep_cs = get_ep_cs(ep_id); + if ( ep_cs[0].cmd_sts.active || ep_cs[1].cmd_sts.active ) { + dcd_registers_t* dcd_reg = _dcd_controller[rhport].regs; + dcd_reg->EPSKIP |= TU_BIT(ep_id); + uint32_t guard = IP3511_EPSKIP_SPIN; + while ( (dcd_reg->EPSKIP & TU_BIT(ep_id)) && guard-- ) {} + } + ep_cs[0].cmd_sts.active = ep_cs[1].cmd_sts.active = 0; +} + void dcd_edpt_stall(uint8_t rhport, uint8_t ep_addr) { (void) rhport; - // TODO cannot able to STALL Control OUT endpoint !!!!! FIXME try some walk-around uint8_t const ep_id = ep_addr2id(ep_addr); - _dcd.ep[ep_id][0].cmd_sts.stall = 1; + // Clear Active directly before setting Stall (no EPSKIP — see edpt_skip_active): the hardware + // services an armed buffer instead of returning STALL, so a halt requested while a transfer is + // queued would not actually stall the endpoint (usbtest case 13). + _dcd.ep[ep_id][0].cmd_sts.active = 0; + _dcd.ep[ep_id][0].cmd_sts.stall = 1; } void dcd_edpt_clear_stall(uint8_t rhport, uint8_t ep_addr) @@ -352,9 +401,17 @@ void dcd_edpt_clear_stall(uint8_t rhport, uint8_t ep_addr) uint8_t const ep_id = ep_addr2id(ep_addr); + // Preserve rf_tv: for non-control endpoints it is a TYPE bit, not the toggle value (UM11126: + // T=1 + RF 1/0 = interrupt/iso). Zeroing it here turned HS periodic interrupt endpoints into + // isochronous - no handshake on OUT, dead IN (usbtest cases 25/26 on lpc55 HS port). + // TODO implement the Errata LPC546xx USB.13 work-around (same semantics in UM11126): with RF/TV preserved at 1, TR + // loads the toggle from TV, so an HS interrupt endpoint restarts on DATA1 after clear-halt and + // the host discards one packet as a retransmission. The documented workaround needs an + // interrupt-on-NAK state machine (park as generic TR=1/TV=0, wait for a NAKed token to latch + // toggle 0 via EPTOGGLE, restore the type) - deferred; one lost packet beats the fully broken + // endpoint the old rf_tv clear caused. _dcd.ep[ep_id][0].cmd_sts.stall = 0; _dcd.ep[ep_id][0].cmd_sts.toggle_reset = 1; - _dcd.ep[ep_id][0].cmd_sts.rf_tv = 0; } bool dcd_edpt_open(uint8_t rhport, tusb_desc_endpoint_t const * p_endpoint_desc) @@ -362,9 +419,15 @@ bool dcd_edpt_open(uint8_t rhport, tusb_desc_endpoint_t const * p_endpoint_desc) //------------- Prepare Queue Head -------------// uint8_t ep_id = ep_addr2id(p_endpoint_desc->bEndpointAddress); ep_cmd_sts_t* ep_cs = get_ep_cs(ep_id); + dcd_registers_t* dcd_reg = _dcd_controller[rhport].regs; - // Check if endpoint is available - TU_ASSERT( ep_cs[0].cmd_sts.disable && ep_cs[1].cmd_sts.disable ); + // usbd_edpt_close() is a no-op on ISO_ALLOC ports, so an endpoint a class closed then reopened + // across SET_INTERFACE (e.g. the video notification or audio streaming endpoint) is still armed + // here rather than disabled. Retire it (edpt_skip_active) before reconfiguring. + if ( !(ep_cs[0].cmd_sts.disable && ep_cs[1].cmd_sts.disable) ) { + edpt_skip_active(rhport, ep_id); + ep_cs[0].cmd_sts.disable = ep_cs[1].cmd_sts.disable = 1; + } edpt_reset(rhport, ep_id); @@ -389,7 +452,6 @@ bool dcd_edpt_open(uint8_t rhport, tusb_desc_endpoint_t const * p_endpoint_desc) } // Enable EP interrupt - dcd_registers_t* dcd_reg = _dcd_controller[rhport].regs; dcd_reg->INTEN |= TU_BIT(ep_id); return true; @@ -399,34 +461,40 @@ void dcd_edpt_close_all (uint8_t rhport) { for (uint8_t ep_id = 0; ep_id < 2*_dcd_controller[rhport].ep_pairs; ++ep_id) { - _dcd.ep[ep_id][0].cmd_sts.active = _dcd.ep[ep_id][0].cmd_sts.active = 0; // TODO proper way is to EPSKIP then wait ep[][].active then write ep[][].disable (see table 778 in LPC55S69 Use Manual) + _dcd.ep[ep_id][0].cmd_sts.active = _dcd.ep[ep_id][1].cmd_sts.active = 0; // TODO proper way is to EPSKIP then wait ep[][].active then write ep[][].disable (see table 778 in LPC55S69 Use Manual) _dcd.ep[ep_id][0].cmd_sts.disable = _dcd.ep[ep_id][1].cmd_sts.disable = 1; } } -void dcd_edpt_close(uint8_t rhport, uint8_t ep_addr) -{ - (void) rhport; - +bool dcd_edpt_iso_alloc(uint8_t rhport, uint8_t ep_addr, uint16_t largest_packet_size) { + (void) largest_packet_size; + // Reserve the endpoint command/status entry once (persists across altsetting changes); the + // buffer pointer is filled per-transfer, so nothing to pre-allocate. Mirrors the ISO branch of + // dcd_edpt_open(). uint8_t ep_id = ep_addr2id(ep_addr); - _dcd.ep[ep_id][0].cmd_sts.active = _dcd.ep[ep_id][0].cmd_sts.active = 0; // TODO proper way is to EPSKIP then wait ep[][].active then write ep[][].disable (see table 778 in LPC55S69 Use Manual) - _dcd.ep[ep_id][0].cmd_sts.disable = _dcd.ep[ep_id][1].cmd_sts.disable = 1; -} + ep_cmd_sts_t* ep_cs = get_ep_cs(ep_id); + TU_ASSERT( ep_cs[0].cmd_sts.disable && ep_cs[1].cmd_sts.disable ); -#if 0 -bool dcd_edpt_iso_alloc(uint8_t rhport, uint8_t ep_addr, uint16_t largest_packet_size) { - (void)rhport; - (void)ep_addr; - (void)largest_packet_size; - return false; + edpt_reset(rhport, ep_id); + ep_cs[0].cmd_sts.type = 1; // ISO + + dcd_registers_t* dcd_reg = _dcd_controller[rhport].regs; + dcd_reg->INTEN |= TU_BIT(ep_id); + return true; } bool dcd_edpt_iso_activate(uint8_t rhport, const tusb_desc_endpoint_t *desc_ep) { - (void)rhport; - (void)desc_ep; - return false; + // (Re)activate on altsetting selection: abort a transfer still armed from the previous + // altsetting (the hardware keeps servicing an Active buffer across SET_INTERFACE, fighting the + // fresh transfer the class queues), clear stall and reset the data toggle. Direct Active=0, not + // EPSKIP (see edpt_skip_active). The class re-arms via dcd_edpt_xfer(). + uint8_t ep_id = ep_addr2id(desc_ep->bEndpointAddress); + ep_cmd_sts_t* ep_cs = get_ep_cs(ep_id); + ep_cs[0].cmd_sts.active = 0; + ep_cs[1].cmd_sts.active = 0; + dcd_edpt_clear_stall(rhport, desc_ep->bEndpointAddress); + return true; } -#endif static void prepare_ep_xfer(uint8_t rhport, uint8_t ep_id, uint16_t buf_offset, uint16_t total_bytes) { uint16_t nbytes; @@ -499,7 +567,7 @@ static void bus_reset(uint8_t rhport) dcd_reg->EPSKIP = 0xFFFFFFFF; dcd_reg->INTSTAT = dcd_reg->INTSTAT; // clear all pending interrupt - dcd_reg->DEVCMDSTAT |= DEVCMDSTAT_SETUP_RECEIVED_MASK; // clear setup received interrupt + devcmdstat_update(dcd_reg, 0, DEVCMDSTAT_SETUP_RECEIVED_MASK); // clear setup received interrupt dcd_reg->INTEN = INT_DEVICE_STATUS_MASK | TU_BIT(0) | TU_BIT(1); // enable device status & control endpoints } @@ -558,18 +626,25 @@ void dcd_int_handler(uint8_t rhport) { dcd_registers_t* dcd_reg = _dcd_controller[rhport].regs; - uint32_t const cmd_stat = dcd_reg->DEVCMDSTAT; - uint32_t int_status = dcd_reg->INTSTAT; - int_status &= dcd_reg->INTEN; + int_status &= dcd_reg->INTEN; dcd_reg->INTSTAT = int_status; // Acknowledge handled interrupt if (int_status == 0) return; + // Snapshot after the INTSTAT ack: latch bits persist (RWC) so nothing is lost, while the reverse + // order could consume INTSTAT bit0 for a SETUP not yet visible in the snapshot - stranding the + // SETUP (INTSTAT is edge-latched) and feeding bit0 to process_xfer_isr as a bogus completion. + uint32_t const cmd_stat = dcd_reg->DEVCMDSTAT; + //------------- Device Status -------------// if ( int_status & INT_DEVICE_STATUS_MASK ) { - dcd_reg->DEVCMDSTAT |= DEVCMDSTAT_RESET_CHANGE_MASK | DEVCMDSTAT_CONNECT_CHANGE_MASK | DEVCMDSTAT_SUSPEND_CHANGE_MASK; + // clear only the change latches observed in the snapshot: one latched by hardware between the + // snapshot and this write would be acknowledged unseen (its DEV_INT re-latches and dispatches + // next pass instead) + devcmdstat_update(dcd_reg, 0, cmd_stat & + (DEVCMDSTAT_RESET_CHANGE_MASK | DEVCMDSTAT_CONNECT_CHANGE_MASK | DEVCMDSTAT_SUSPEND_CHANGE_MASK)); if ( cmd_stat & DEVCMDSTAT_RESET_CHANGE_MASK) // bus reset { @@ -614,15 +689,43 @@ void dcd_int_handler(uint8_t rhport) _dcd.ep[0][0].cmd_sts.active = _dcd.ep[1][0].cmd_sts.active = 0; _dcd.ep[0][0].cmd_sts.stall = _dcd.ep[1][0].cmd_sts.stall = 0; - dcd_reg->DEVCMDSTAT |= DEVCMDSTAT_SETUP_RECEIVED_MASK; + // UM flow: ack the latch FIRST, then read the payload. This IP has no setup lockout, so a + // back-to-back SETUP can overwrite _dcd.setup_packet at any time - but with the latch already + // released, any such overwrite re-latches SETUP_RECEIVED and is redelivered (worst case a + // superseded duplicate, absorbed by usbd's queued-setup counter). The reverse order can + // consume the newer SETUP's latch unseen and lose it. + devcmdstat_update(dcd_reg, 0, DEVCMDSTAT_SETUP_RECEIVED_MASK); - dcd_event_setup_received(rhport, _dcd.setup_packet, true); + // UM11126 Fig 163 (control EP0 flowchart) requires clearing the EP0IN interrupt here: a + // control IN completion latched before this SETUP must not reach usbd after it, where it + // would be applied to the new request and arm its status stage early. EP0OUT goes with it - + // bit0 is set by SETUP reception too, and left set it would replay next pass as a phantom + // completion. Neither can discard live work: the SETUP latch NAKs all EP0 traffic until the + // update above, and both EP0 Active bits were cleared a few lines up. + dcd_reg->INTSTAT = TU_BIT(0) | TU_BIT(1); + + // Copied a byte at a time rather than with memcpy: C orders volatile accesses only against + // each other, so a non-volatile copy of this buffer may be sunk below the guard read that + // follows - gcc does exactly that at -O2 and -O3, leaving only -Os correct. + uint8_t setup_copy[8]; + for (uint8_t i = 0; i < sizeof(setup_copy); i++) { + setup_copy[i] = _dcd.setup_packet[i]; + } + + // a SETUP that raced in after the acks (its bit0 consumed above) makes this copy suspect: + // its latch is visible again, so re-raise the endpoint interrupt and let the next pass + // deliver the newer payload rather than passing up bytes that may be torn between the two + if (dcd_reg->DEVCMDSTAT & DEVCMDSTAT_SETUP_RECEIVED_MASK) { + dcd_reg->INTSETSTAT = TU_BIT(0); + } else { + dcd_event_setup_received(rhport, setup_copy, true); + } // keep waiting for next setup prepare_setup_packet(rhport); - // clear bit0 - int_status = tu_bit_clear(int_status, 0); + // drop both EP0 bits: acked above, and neither belongs to the request this SETUP starts + int_status &= ~(TU_BIT(0) | TU_BIT(1)); } // Endpoint transfer complete interrupt diff --git a/src/portable/raspberrypi/rp2040/dcd_rp2040.c b/src/portable/raspberrypi/rp2040/dcd_rp2040.c index 63097cd0a..564ded535 100644 --- a/src/portable/raspberrypi/rp2040/dcd_rp2040.c +++ b/src/portable/raspberrypi/rp2040/dcd_rp2040.c @@ -387,6 +387,9 @@ bool dcd_deinit(uint8_t rhport) { reset_block(RESETS_RESET_USBCTRL_BITS); unreset_block_wait(RESETS_RESET_USBCTRL_BITS); + // Release allocated resources + rp2usb_deinit(); + return true; } @@ -550,9 +553,46 @@ void dcd_edpt_clear_stall(uint8_t rhport, uint8_t ep_addr) { if (epnum != 0) { struct hw_endpoint* ep = hw_endpoint_get(epnum, dir); - ep->next_pid = 0; // reset data toggle - io_rw_32 *buf_reg = get_buf_ctrl(epnum, dir); - *buf_reg = 0; + + if (ep->state == EPSTATE_ACTIVE) { + // Clear-halt on an endpoint with an in-flight transfer is used as a data-toggle reset + // (e.g. usbtest case 29) rather than to recover from a real stall (a stall aborts the + // transfer, leaving the endpoint IDLE). Abort and re-issue the transfer with the toggle + // reset to DATA0 so it still completes and releases the usbd claim, instead of silently + // dropping it and starving the endpoint. Save the buffer/length before the abort clears them. + uint8_t* user_buf = ep->user_buf; + uint16_t remaining = ep->remaining_len; + const uint16_t xferred = ep->xferred_len; // bytes already moved on this submission + io_rw_32 *ep_reg = get_ep_ctrl(epnum, dir); + io_rw_32 *buf_reg = get_buf_ctrl(epnum, dir); + // bufctrl_prepare16() subtracts each armed buffer's length from remaining_len when arming, + // for BOTH directions, before the host has drained (IN) or filled (OUT) it. The abort below + // discards those still-armed buffers, so rewind remaining_len by their lengths or the re-issue + // is short by 1-2 packets. IN additionally advances user_buf as packets are copied into DPRAM, + // so its pointer must rewind too; OUT copies out only on completion, so its pointer is intact. + const uint32_t bc = *buf_reg; + uint16_t staged = 0; + if (bc & USB_BUF_CTRL_AVAIL) { + staged = (uint16_t)(bc & USB_BUF_CTRL_LEN_MASK); + } + if ((bc >> 16) & USB_BUF_CTRL_AVAIL) { + staged = (uint16_t)(staged + ((bc >> 16) & USB_BUF_CTRL_LEN_MASK)); + } + remaining = (uint16_t)(remaining + staged); + if (dir == TUSB_DIR_IN) { + user_buf -= staged; + } + hw_endpoint_abort_xfer(ep); // safe abort (handles RP2040-E2), resets ep transfer state + ep->next_pid = 0; // DATA0 + rp2usb_xfer_start(ep, ep_reg, buf_reg, user_buf, NULL, remaining); + // rp2usb_xfer_start() zeroes xferred_len; add back what the aborted transfer already moved so + // the eventual completion reports the full length, not just the post-clear-halt remainder. + ep->xferred_len += xferred; + } else { + ep->next_pid = 0; // reset data toggle + io_rw_32 *buf_reg = get_buf_ctrl(epnum, dir); + *buf_reg = 0; // clear the stall response + } } } diff --git a/src/portable/raspberrypi/rp2040/hcd_rp2040.c b/src/portable/raspberrypi/rp2040/hcd_rp2040.c index 28dc7f93c..a04890835 100644 --- a/src/portable/raspberrypi/rp2040/hcd_rp2040.c +++ b/src/portable/raspberrypi/rp2040/hcd_rp2040.c @@ -427,6 +427,10 @@ bool hcd_deinit(uint8_t rhport) { irq_remove_handler(USBCTRL_IRQ, hcd_rp2040_irq); reset_block(RESETS_RESET_USBCTRL_BITS); unreset_block_wait(RESETS_RESET_USBCTRL_BITS); + + // Release allocated resources + rp2usb_deinit(); + return true; } diff --git a/src/portable/raspberrypi/rp2040/rp2040_usb.c b/src/portable/raspberrypi/rp2040/rp2040_usb.c index e4eb0184e..96b335bd3 100644 --- a/src/portable/raspberrypi/rp2040/rp2040_usb.c +++ b/src/portable/raspberrypi/rp2040/rp2040_usb.c @@ -85,6 +85,10 @@ void rp2usb_init(void) { critical_section_init(&rp2usb_lock); } +void rp2usb_deinit(void) { + critical_section_deinit(&rp2usb_lock); +} + void __tusb_irq_path_func(rp2usb_reset_transfer)(hw_endpoint_t *ep) { ep->state = EPSTATE_IDLE; ep->remaining_len = 0; @@ -176,10 +180,15 @@ void __tusb_irq_path_func(rp2usb_buffer_start)(hw_endpoint_t *ep, io_rw_32 *ep_r // Note: device EP0 does not have an endpoint control register if (ep_reg != NULL) { uint32_t ep_ctrl = *ep_reg; + // Isochronous endpoints get a single DPRAM buffer (hw_endpoint_open only double-sizes BULK), so + // they must never be double-buffered here even when a transfer spans multiple packets, or buffer + // 1 (at dpram_buf+64) would spill into the next endpoint's DPRAM. (Never true for BULK, so the + // double-buffered bulk path is unaffected.) + const bool is_iso = (((ep_ctrl >> EP_CTRL_BUFFER_TYPE_LSB) & 0x3u) == TUSB_XFER_ISOCHRONOUS); #if CFG_TUH_ENABLED - const bool force_single = (rp2usb_is_host_mode() && ep->interrupt_num > 0); + const bool force_single = is_iso || (rp2usb_is_host_mode() && ep->interrupt_num > 0); #else - const bool force_single = false; + const bool force_single = is_iso; #endif if (ep->remaining_len && !force_single) { diff --git a/src/portable/raspberrypi/rp2040/rp2040_usb.h b/src/portable/raspberrypi/rp2040/rp2040_usb.h index f4e85764d..e5a54007c 100644 --- a/src/portable/raspberrypi/rp2040/rp2040_usb.h +++ b/src/portable/raspberrypi/rp2040/rp2040_usb.h @@ -147,6 +147,7 @@ extern volatile uint32_t e15_last_sof; #endif void rp2usb_init(void); +void rp2usb_deinit(void); // if usb hardware is in host mode TU_ATTR_ALWAYS_INLINE static inline bool rp2usb_is_host_mode(void) { diff --git a/src/portable/renesas/rusb2/dcd_rusb2.c b/src/portable/renesas/rusb2/dcd_rusb2.c index f0ef9738b..c93bab05e 100644 --- a/src/portable/renesas/rusb2/dcd_rusb2.c +++ b/src/portable/renesas/rusb2/dcd_rusb2.c @@ -28,6 +28,9 @@ typedef struct { uint8_t ep; /* an assigned endpoint address */ uint8_t ff; /* `buf` is TU_FUFO or POD */ + bool queued; /* a transfer is submitted and not yet completed (independent of `buf`, which is + NULL for a zero-length read) -- used to decide clear-stall re-arm */ + bool zlp_pending; /* a zero-length IN packet couldn't be queued at submit (FIFO full); retry on BRDY */ } pipe_state_t; typedef struct @@ -40,6 +43,7 @@ typedef struct static dcd_data_t _dcd; + //--------------------------------------------------------------------+ // INTERNAL OBJECT & FUNCTION DECLARATION //--------------------------------------------------------------------+ @@ -121,9 +125,19 @@ static uint16_t edpt_max_packet_size(rusb2_reg_t *rusb, unsigned num) { return rusb->PIPEMAXP; } -static inline void pipe_wait_for_ready(rusb2_reg_t * rusb, unsigned num) { - while ( rusb->D0FIFOSEL_b.CURPIPE != num ) {} - while ( !rusb->D0FIFOCTR_b.FRDY ) {} +// Select the D0FIFO for `num` and wait until its buffer is ready for CPU access. Both flags +// normally settle within a few cycles (the pipe was just armed, or a BRDY freed a plane). But an +// IN pipe whose double buffer is already full stalls FRDY until the host drains it, and a +// no-handshake iso IN endpoint the host has stopped polling never drains at all — so FRDY would +// hang forever. This runs with the USB IRQ masked, so a naked spin freezes the whole stack; bound +// it and let the caller abort the FIFO access. Returns false on timeout. +#define RUSB2_FIFO_READY_SPIN 100000u +static inline bool pipe_wait_for_ready(rusb2_reg_t *rusb, unsigned num) { + uint32_t spin = RUSB2_FIFO_READY_SPIN; + while ( rusb->D0FIFOSEL_b.CURPIPE != num ) { if (!spin--) return false; } + spin = RUSB2_FIFO_READY_SPIN; + while ( !rusb->D0FIFOCTR_b.FRDY ) { if (!spin--) return false; } + return true; } //--------------------------------------------------------------------+ @@ -177,6 +191,15 @@ static bool pipe0_xfer_out(rusb2_reg_t *rusb) { pipe_state_t *pipe = &_dcd.pipe[0]; const unsigned rem = pipe->remaining; + // BRDY with no armed transfer: a back-to-back data-stage packet beat the PID=NAK below (the + // host has already ACKed it). Park it in the DCP buffer — an unread buffer NAKs further OUTs — + // and let process_pipe0_xfer deliver it when usbd arms the next chunk. BCLR here would silently + // drop the packet and shift every later chunk by one (usbtest ctrl_out corruption at ra4m1). + if (pipe->buf == NULL && rem == 0) { + rusb->DCPCTR = RUSB2_PIPE_CTR_PID_NAK; + return false; + } + const uint16_t mps = edpt0_max_packet_size(rusb); const uint16_t vld = rusb->CFIFOCTR_b.DTLN; const uint16_t len = tu_min16(tu_min16(rem, mps), vld); @@ -201,6 +224,12 @@ static bool pipe0_xfer_out(rusb2_reg_t *rusb) { pipe->remaining = rem - len; if ((len < mps) || (rem == len)) { pipe->buf = NULL; + // Flow-control the single-buffer control pipe: NAK further OUT until usbd arms the next + // data-stage chunk. usbd receives a multi-packet control-OUT one CFG_TUD_ENDPOINT0_SIZE + // packet per submit; without this the DCP auto-accepts the next back-to-back packet into the + // just-emptied buffer and the following BRDY (remaining==0) BCLR-discards it, dropping 64 + // bytes mid-transfer (e.g. usbtest ctrl_out 512B). RA4M1 UM R01UH0887 DCPCTR.PID. + rusb->DCPCTR = RUSB2_PIPE_CTR_PID_NAK; return true; } @@ -226,7 +255,12 @@ static bool pipe_xfer_in(rusb2_reg_t* rusb, unsigned num) } const uint16_t mps = edpt_max_packet_size(rusb, num); - pipe_wait_for_ready(rusb, num); + if (!pipe_wait_for_ready(rusb, num)) { + // Buffer never came ready (double-buffered IN pipe full, host not draining). Drop this load; + // the transfer stays pending and is retried when a BRDY frees a plane or the pipe is re-armed. + rusb->D0FIFOSEL = 0; + return false; + } uint16_t len = tu_min16(rem, mps); void *buf = pipe->buf; @@ -267,7 +301,10 @@ static bool pipe_xfer_out(rusb2_reg_t* rusb, unsigned num) rusb->D0FIFOSEL = fifo_sel; const uint16_t mps = edpt_max_packet_size(rusb, num); - pipe_wait_for_ready(rusb, num); + if (!pipe_wait_for_ready(rusb, num)) { + rusb->D0FIFOSEL = 0; + return false; // FIFO not ready; leave the receive pending (BRDY re-enters when data arrives) + } const uint16_t vld = (uint16_t)rusb->D0FIFOCTR_b.DTLN; const uint16_t len = tu_min16(tu_min16(rem, mps), vld); @@ -333,7 +370,16 @@ static void process_status_completion(uint8_t rhport) dcd_event_xfer_complete(rhport, ep_addr, 0, XFER_RESULT_SUCCESS, true); } -static bool process_pipe0_xfer(rusb2_reg_t *rusb, int buffer_type, uint8_t ep_addr, void *buffer, +// Report a completed transfer on `num` and reset its bookkeeping. Single completion path for the +// BRDY handler and the EP0 parked-packet drain, so they can't diverge (e.g. on clearing `queued`). +static void pipe_xfer_complete(uint8_t rhport, unsigned num, bool in_isr) { + pipe_state_t *pipe = &_dcd.pipe[num]; + pipe->queued = false; + dcd_event_xfer_complete(rhport, pipe->ep, pipe->length - pipe->remaining, + XFER_RESULT_SUCCESS, in_isr); +} + +static bool process_pipe0_xfer(uint8_t rhport, rusb2_reg_t *rusb, int buffer_type, uint8_t ep_addr, void *buffer, uint16_t total_bytes) { uint16_t fifo_sel = (rusb2_is_highspeed_reg(rusb) ? RUSB2_FIFOSEL_MBW_32BIT : RUSB2_FIFOSEL_MBW_16BIT) | FIFOSEL_BIGEND; @@ -359,6 +405,15 @@ static bool process_pipe0_xfer(rusb2_reg_t *rusb, int buffer_type, uint8_t ep_ad /* IN */ TU_ASSERT(rusb->DCPCTR_b.BSTS && (rusb->USBREQ & 0x80)); pipe0_xfer_in(rusb); + } else if (rusb->CFIFOCTR_b.DTLN > 0) { + /* OUT: a back-to-back packet parked by pipe0_xfer_out already sits in the DCP buffer (its + BRDY has fired and been cleared) — deliver it into this chunk now; no new BRDY will come + for it. Runs with the USB IRQ masked (dcd_edpt_xfer). Detected via the hardware DTLN + rather than a driver flag: the BCLR at SETUP/bus-reset then self-heals any parked state. */ + if (pipe0_xfer_out(rusb)) { + pipe_xfer_complete(rhport, 0, false); + return true; // PID stays NAK (set by pipe0_xfer_out) until the next chunk is armed + } } rusb->DCPCTR = RUSB2_PIPE_CTR_PID_BUF; } else { @@ -370,6 +425,24 @@ static bool process_pipe0_xfer(rusb2_reg_t *rusb, int buffer_type, uint8_t ep_ad return true; } +// Queue a zero-length IN packet. Returns false if the FIFO buffer wasn't free (double-buffered pipe +// full, host not draining) so BVAL couldn't be written -- the caller retries on the next BRDY. +static bool pipe_zlp_in(rusb2_reg_t *rusb, unsigned num) { + rusb->D0FIFOSEL = (uint16_t) num; + const bool ready = pipe_wait_for_ready(rusb, num); + if (ready) { + rusb->D0FIFOCTR = RUSB2_CFIFOCTR_BVAL_Msk; + } + rusb->D0FIFOSEL = 0; + // deselect completes within a few bus cycles (not host-dependent), but bound it anyway: this + // runs with the USB IRQ masked, where any stuck spin freezes the whole stack + uint32_t spin = RUSB2_FIFO_READY_SPIN; + while (rusb->D0FIFOSEL_b.CURPIPE) { + if (!spin--) { break; } + } + return ready; +} + static bool process_pipe_xfer(rusb2_reg_t* rusb, int buffer_type, uint8_t ep_addr, void* buffer, uint16_t total_bytes) { const unsigned epn = tu_edpt_number(ep_addr); @@ -379,23 +452,20 @@ static bool process_pipe_xfer(rusb2_reg_t* rusb, int buffer_type, uint8_t ep_add TU_ASSERT(num); pipe_state_t *pipe = &_dcd.pipe[num]; - pipe->ff = buffer_type; - pipe->buf = buffer; - pipe->length = total_bytes; - pipe->remaining = total_bytes; + pipe->ff = buffer_type; + pipe->buf = buffer; + pipe->length = total_bytes; + pipe->remaining = total_bytes; + pipe->queued = true; + pipe->zlp_pending = false; if (dir) { /* IN */ if (total_bytes) { pipe_xfer_in(rusb, num); } else { - /* ZLP */ - rusb->D0FIFOSEL = num; - pipe_wait_for_ready(rusb, num); - rusb->D0FIFOCTR = RUSB2_CFIFOCTR_BVAL_Msk; - rusb->D0FIFOSEL = 0; - /* if CURPIPE bits changes, check written value */ - while (rusb->D0FIFOSEL_b.CURPIPE) {} + /* ZLP: if the FIFO buffer isn't free yet, defer the queue to the next BRDY (see process_pipe_brdy) */ + pipe->zlp_pending = !pipe_zlp_in(rusb, num); } } else { // OUT @@ -418,11 +488,11 @@ static bool process_pipe_xfer(rusb2_reg_t* rusb, int buffer_type, uint8_t ep_add return true; } -static bool process_edpt_xfer(rusb2_reg_t* rusb, int buffer_type, uint8_t ep_addr, void* buffer, uint16_t total_bytes) +static bool process_edpt_xfer(uint8_t rhport, rusb2_reg_t* rusb, int buffer_type, uint8_t ep_addr, void* buffer, uint16_t total_bytes) { const unsigned epn = tu_edpt_number(ep_addr); if (0 == epn) { - return process_pipe0_xfer(rusb, buffer_type, ep_addr, buffer, total_bytes); + return process_pipe0_xfer(rhport, rusb, buffer_type, ep_addr, buffer, total_bytes); } else { return process_pipe_xfer(rusb, buffer_type, ep_addr, buffer, total_bytes); } @@ -448,7 +518,15 @@ static void process_pipe_brdy(uint8_t rhport, unsigned num) if (dir) { /* IN */ - completed = pipe_xfer_in(rusb, num); + if (pipe->zlp_pending) { + // The submit-time ZLP couldn't be queued (FIFO full); a freed buffer plane lets us queue it + // now. Don't report completion until the ZLP is actually queued (and then sent, next BRDY), + // otherwise a spurious BRDY would complete a zero-length IN the host never received. + pipe->zlp_pending = !pipe_zlp_in(rusb, num); + completed = false; + } else { + completed = pipe_xfer_in(rusb, num); + } } else { // OUT if (num) { @@ -458,9 +536,7 @@ static void process_pipe_brdy(uint8_t rhport, unsigned num) } } if (completed) { - dcd_event_xfer_complete(rhport, pipe->ep, - pipe->length - pipe->remaining, - XFER_RESULT_SUCCESS, true); + pipe_xfer_complete(rhport, num, true); // TU_LOG1("C %d %d\r\n", num, pipe->length - pipe->remaining); } } @@ -585,19 +661,8 @@ bool dcd_init(uint8_t rhport, const tusb_rhport_init_t* rh_init) { #ifdef RUSB2_SUPPORT_HIGHSPEED if ( rusb2_is_highspeed_rhport(rhport) ) { - rusb->SYSCFG_b.HSE = 1; - - // leave CLKSEL as default (0x11) 24Mhz - - // Power and reset UTMI Phy - uint16_t physet = (rusb->PHYSET | RUSB2_PHYSET_PLLRESET_Msk) & ~RUSB2_PHYSET_DIRPD_Msk; - rusb->PHYSET = physet; - R_BSP_SoftwareDelay((uint32_t) 1, BSP_DELAY_UNITS_MILLISECONDS); - rusb->PHYSET_b.PLLRESET = 0; - - // set UTMI to operating mode and wait for PLL lock confirmation - rusb->LPSTS_b.SUSPENDM = 1; - while (!rusb->PLLSTA_b.PLLLOCK) {} + rusb->SYSCFG_b.HSE = TUD_OPT_HIGH_SPEED ? 1 : 0; // FS-only build: no HS chirp + rusb2_utmi_phy_powerup(rusb); rusb->SYSCFG_b.DRPD = 0; rusb->SYSCFG_b.USBE = 1; @@ -702,10 +767,20 @@ bool dcd_edpt_open(uint8_t rhport, tusb_desc_endpoint_t const * ep_desc) if ( !rusb2_is_highspeed_rhport(rhport) && mps > 256) { return false; } + } else if (xfer == TUSB_XFER_INTERRUPT) { + // Interrupt pipes (6-9) have a fixed 64-byte buffer even in high speed (RA6M5 UM 29.1); + // a larger PIPEMAXP would enumerate, then silently truncate every transfer + TU_ASSERT(mps <= 64); } - const unsigned num = find_pipe(xfer); - TU_ASSERT(num); + // Re-opening an endpoint must reuse its pipe: usbd_edpt_close() is a no-op on ISO_ALLOC ports, + // so a class's close/open across SET_INTERFACE (e.g. video's notification endpoint) would + // otherwise allocate a second pipe with the same EPNUM and leak pipes until exhaustion. + unsigned num = _dcd.ep[dir][epn]; + if (num == 0) { + num = find_pipe(xfer); + TU_ASSERT(num); + } _dcd.pipe[num].ep = ep_addr; _dcd.ep[dir][epn] = num; @@ -713,13 +788,12 @@ bool dcd_edpt_open(uint8_t rhport, tusb_desc_endpoint_t const * ep_desc) /* setup pipe */ dcd_int_disable(rhport); + rusb->PIPESEL = num; if ( rusb2_is_highspeed_rhport(rhport) ) { - // FIXME shouldn't be after pipe selection and config, also the BUFNMB should be changed - // depending on the allocation scheme + // PIPEBUF is PIPESEL-windowed (RA6M5 UM 29.2.35): write it after selecting the pipe. + // FIXME BUFNMB is a fixed 0x08 for every pipe; a real per-pipe allocation scheme is needed. rusb->PIPEBUF = 0x7C08; } - - rusb->PIPESEL = num; rusb->PIPEMAXP = mps; volatile uint16_t *ctr = get_pipectr(rusb, num); *ctr = RUSB2_PIPE_CTR_ACLRM_Msk | RUSB2_PIPE_CTR_SQCLR_Msk; @@ -748,6 +822,8 @@ bool dcd_edpt_open(uint8_t rhport, tusb_desc_endpoint_t const * ep_desc) return true; } +static void edpt_close(uint8_t rhport, uint8_t ep_addr); + void dcd_edpt_close_all(uint8_t rhport) { unsigned i = TU_ARRAY_SIZE(_dcd.pipe); @@ -757,12 +833,14 @@ void dcd_edpt_close_all(uint8_t rhport) if (!ep_addr) { continue; } - dcd_edpt_close(rhport, (uint8_t)ep_addr); + edpt_close(rhport, (uint8_t)ep_addr); } dcd_int_enable(rhport); } -void dcd_edpt_close(uint8_t rhport, uint8_t ep_addr) +// Internal helper: on this (ISO_ALLOC) IP the stack no longer calls dcd_edpt_close(); only +// dcd_edpt_close_all() uses it to tear down each pipe. +static void edpt_close(uint8_t rhport, uint8_t ep_addr) { rusb2_reg_t * rusb = RUSB2_REG(rhport); const unsigned epn = tu_edpt_number(ep_addr); @@ -774,24 +852,73 @@ void dcd_edpt_close(uint8_t rhport, uint8_t ep_addr) *ctr = 0; rusb->PIPESEL = (uint16_t)num; rusb->PIPECFG = 0; - _dcd.pipe[num].ep = 0; + _dcd.pipe[num].ep = 0; + _dcd.pipe[num].queued = false; + _dcd.pipe[num].zlp_pending = false; _dcd.ep[dir][epn] = 0; } -#if 0 bool dcd_edpt_iso_alloc(uint8_t rhport, uint8_t ep_addr, uint16_t largest_packet_size) { - (void)rhport; - (void)ep_addr; - (void)largest_packet_size; - return false; + rusb2_reg_t * rusb = RUSB2_REG(rhport); + const unsigned epn = tu_edpt_number(ep_addr); + const unsigned dir = tu_edpt_dir(ep_addr); + + // Fullspeed ISO is limited to 256 bytes + if (!rusb2_is_highspeed_rhport(rhport) && largest_packet_size > 256) { + return false; + } + + // Reserve an ISO-capable pipe (1 or 2) once; it persists across altsetting changes so + // dcd_edpt_iso_activate() only has to re-arm it in place (no pipe free/realloc, which on this + // shared-register IP would churn PIPESEL/PIPECFG and disturb the other pipes). + const unsigned num = find_pipe(TUSB_XFER_ISOCHRONOUS); + TU_ASSERT(num); + _dcd.pipe[num].ep = ep_addr; + _dcd.ep[dir][epn] = num; + + dcd_int_disable(rhport); + rusb->PIPESEL = (uint16_t) num; + if (rusb2_is_highspeed_rhport(rhport)) { + // PIPEBUF is PIPESEL-windowed (RA6M5 UM 29.2.35): write it after selecting the pipe. + // FIXME (as in dcd_edpt_open): BUFNMB is a fixed 0x08 for every pipe; a real allocator is needed. + rusb->PIPEBUF = 0x7C08; + } + rusb->PIPEMAXP = largest_packet_size; + volatile uint16_t *ctr = get_pipectr(rusb, num); + *ctr = RUSB2_PIPE_CTR_ACLRM_Msk | RUSB2_PIPE_CTR_SQCLR_Msk; + *ctr = 0; // leave the pipe NAKing until activated + rusb->PIPECFG = (uint16_t) ((dir << 4) | epn | RUSB2_PIPECFG_TYPE_ISO | RUSB2_PIPECFG_DBLB_Msk); + rusb->BRDYSTS = (uint16_t) (0x3FFu ^ TU_BIT(num)); + rusb->BRDYENB |= TU_BIT(num); + dcd_int_enable(rhport); + return true; } bool dcd_edpt_iso_activate(uint8_t rhport, const tusb_desc_endpoint_t *desc_ep) { - (void)rhport; - (void)desc_ep; - return false; + rusb2_reg_t * rusb = RUSB2_REG(rhport); + const uint8_t ep_addr = desc_ep->bEndpointAddress; + const unsigned epn = tu_edpt_number(ep_addr); + const unsigned dir = tu_edpt_dir(ep_addr); + const unsigned num = _dcd.ep[dir][epn]; + TU_ASSERT(num); // must have been iso-alloc'd + + dcd_int_disable(rhport); + rusb->PIPESEL = (uint16_t) num; + rusb->PIPEMAXP = tu_edpt_packet_size(desc_ep); + volatile uint16_t *ctr = get_pipectr(rusb, num); + *ctr = RUSB2_PIPE_CTR_ACLRM_Msk | RUSB2_PIPE_CTR_SQCLR_Msk; // abort in-flight + reset data toggle + *ctr = 0; + // a transfer armed before SET_INTERFACE survives to here (no dcd close on this port): drop the + // stale bookkeeping so a BRDY firing before the class re-arms can't replay it + pipe_state_t *pipe = &_dcd.pipe[num]; + pipe->buf = NULL; + pipe->remaining = 0; + pipe->queued = false; + pipe->zlp_pending = false; + *ctr = RUSB2_PIPE_CTR_PID_BUF; // enable + dcd_int_enable(rhport); + return true; } -#endif bool dcd_edpt_xfer(uint8_t rhport, uint8_t ep_addr, uint8_t * buffer, uint16_t total_bytes, bool is_isr) { @@ -799,7 +926,7 @@ bool dcd_edpt_xfer(uint8_t rhport, uint8_t ep_addr, uint8_t * buffer, uint16_t t rusb2_reg_t* rusb = RUSB2_REG(rhport); dcd_int_disable(rhport); - bool r = process_edpt_xfer(rusb, 0, ep_addr, buffer, total_bytes); + bool r = process_edpt_xfer(rhport, rusb, 0, ep_addr, buffer, total_bytes); dcd_int_enable(rhport); return r; @@ -812,7 +939,7 @@ bool dcd_edpt_xfer_fifo(uint8_t rhport, uint8_t ep_addr, tu_fifo_t * ff, uint16_ rusb2_reg_t* rusb = RUSB2_REG(rhport); dcd_int_disable(rhport); - bool r = process_edpt_xfer(rusb, 1, ep_addr, ff, total_bytes); + bool r = process_edpt_xfer(rhport, rusb, 1, ep_addr, ff, total_bytes); dcd_int_enable(rhport); return r; @@ -847,7 +974,19 @@ void dcd_edpt_clear_stall(uint8_t rhport, uint8_t ep_addr) } else { const unsigned num = _dcd.ep[0][tu_edpt_number(ep_addr)]; rusb->PIPESEL = (uint16_t)num; - if (rusb->PIPECFG_b.TYPE != 1) { + // Drop any packet parked in the buffer while halted: a data-OUT packet the host sent before + // aborting its transfer would otherwise be delivered into the next read after recovery + // (BOT reset + clear-halt re-arms a 31-byte CBW read which then receives stale WRITE data, + // "SCSI CBW is not valid" -> stall -> reset loop; ra6m5 msc write wedge). + *ctr = RUSB2_PIPE_CTR_ACLRM_Msk; + *ctr = 0; + // Non-bulk OUT re-enables straight away. Bulk OUT is normally armed together with its transaction + // counter (TRE) by process_pipe_xfer(), so we don't blindly re-enable it here — but if a receive + // was already armed (still queued), SQCLR above just left it NAKing. Re-assert BUF so it keeps + // receiving; the class driver still considers that read submitted and never re-arms it, so + // otherwise the endpoint NAKs forever (usbtest toggle test 29 clears the halt on an armed pipe). + // `queued` (not `buf`) is the armed test: a zero-length OUT read has buf==NULL yet is armed. + if (rusb->PIPECFG_b.TYPE != 1 || _dcd.pipe[num].queued) { *ctr = RUSB2_PIPE_CTR_PID_BUF; } } diff --git a/src/portable/renesas/rusb2/hcd_rusb2.c b/src/portable/renesas/rusb2/hcd_rusb2.c index 849551d27..489162e54 100644 --- a/src/portable/renesas/rusb2/hcd_rusb2.c +++ b/src/portable/renesas/rusb2/hcd_rusb2.c @@ -454,11 +454,9 @@ bool hcd_init(uint8_t rhport, const tusb_rhport_init_t* rh_init) { if (rusb2_is_highspeed_rhport(rhport) ) { rusb->SYSCFG_b.HSE = 1; rusb->PHYSET_b.HSEB = 0; - rusb->PHYSET_b.DIRPD = 0; - R_BSP_SoftwareDelay((uint32_t) 1, BSP_DELAY_UNITS_MILLISECONDS); - rusb->PHYSET_b.PLLRESET = 0; - rusb->LPSTS_b.SUSPENDM = 1; - while ( !rusb->PLLSTA_b.PLLLOCK ); + // same PHY reference-clock + power-up requirements as dcd_init: without CLKSEL matching the + // board XTAL the PLL never locks and the wait below would spin forever (e.g. EK-RA8M1, 20 MHz) + rusb2_utmi_phy_powerup(rusb); rusb->SYSCFG_b.DRPD = 1; rusb->SYSCFG_b.DCFM = 1; rusb->SYSCFG_b.DPRPU = 0; diff --git a/src/portable/renesas/rusb2/rusb2_ra.h b/src/portable/renesas/rusb2/rusb2_ra.h index e5945ffe2..0954d0d25 100644 --- a/src/portable/renesas/rusb2/rusb2_ra.h +++ b/src/portable/renesas/rusb2/rusb2_ra.h @@ -49,6 +49,23 @@ typedef struct { #define rusb2_is_highspeed_rhport(_p) (_p == 1) #define rusb2_is_highspeed_reg(_reg) (_reg == RUSB2_REG(1)) + + // UTMI PHY reference clock is the main oscillator: PHYSET.CLKSEL must match the board XTAL + // before the PHY PLL is released (RA6M5 UM R01UH0891 29.2.17: 00=12 MHz, 10=20 MHz, + // 11=24 MHz reset default). EK-RA6M5 runs 24 MHz (default works); EK-RA8M1 runs 20 MHz and + // never locks/chirps on the default. A board with a non-standard USB clocking scheme can + // pre-define RUSB2_PHYSET_CLKSEL_VALUE to override this selection. + #ifndef RUSB2_PHYSET_CLKSEL_VALUE + #if BSP_CFG_XTAL_HZ == 12000000 + #define RUSB2_PHYSET_CLKSEL_VALUE 0u + #elif BSP_CFG_XTAL_HZ == 20000000 + #define RUSB2_PHYSET_CLKSEL_VALUE 2u + #elif BSP_CFG_XTAL_HZ == 24000000 + #define RUSB2_PHYSET_CLKSEL_VALUE 3u + #else + #error "USBHS UTMI PHY: no PHYSET.CLKSEL encoding for this BSP_CFG_XTAL_HZ; define RUSB2_PHYSET_CLKSEL_VALUE" + #endif + #endif #else #define RUSB2_CONTROLLER_COUNT 1 @@ -84,6 +101,31 @@ TU_ATTR_ALWAYS_INLINE static inline void rusb2_int_disable(uint8_t rhport) { TU_ATTR_ALWAYS_INLINE static inline void rusb2_phy_init(void) { } +#ifdef RUSB2_SUPPORT_HIGHSPEED +// UTMI PHY power-up per the FSP reference sequence (r_usb_preg_access.c), shared by dcd_init and +// hcd_init: program CLKSEL to the board XTAL while the PHY is powered down (DIRPD=1), 1 us, +// release DIRPD, 1 ms, release PLLRESET, then wait for PLL lock. Changing CLKSEL as the PHY +// powers up gets mis-sampled (EK-RA8M1, 20 MHz). +static inline void rusb2_utmi_phy_powerup(rusb2_reg_t* rusb) { + uint16_t physet = rusb->PHYSET | RUSB2_PHYSET_DIRPD_Msk; + rusb->PHYSET = physet; + #ifdef RUSB2_PHYSET_CLKSEL_VALUE + physet = (uint16_t) ((physet & ~RUSB2_PHYSET_CLKSEL_Msk) | + (RUSB2_PHYSET_CLKSEL_VALUE << RUSB2_PHYSET_CLKSEL_Pos)); + rusb->PHYSET = physet; + #endif + R_BSP_SoftwareDelay((uint32_t) 1, BSP_DELAY_UNITS_MICROSECONDS); + physet &= (uint16_t) ~RUSB2_PHYSET_DIRPD_Msk; + rusb->PHYSET = physet; + R_BSP_SoftwareDelay((uint32_t) 1, BSP_DELAY_UNITS_MILLISECONDS); + rusb->PHYSET_b.PLLRESET = 0; + + // set UTMI to operating mode and wait for PLL lock confirmation + rusb->LPSTS_b.SUSPENDM = 1; + while (!rusb->PLLSTA_b.PLLLOCK) {} +} +#endif + #ifdef __cplusplus } #endif diff --git a/src/portable/st/stm32_fsdev/dcd_stm32_fsdev.c b/src/portable/st/stm32_fsdev/dcd_stm32_fsdev.c index aecd689b3..ba05818b6 100644 --- a/src/portable/st/stm32_fsdev/dcd_stm32_fsdev.c +++ b/src/portable/st/stm32_fsdev/dcd_stm32_fsdev.c @@ -835,7 +835,17 @@ void dcd_edpt_clear_stall(uint8_t rhport, uint8_t ep_addr) { ep_reg &= U_EPREG_MASK | EP_STAT_MASK(dir) | EP_DTOG_MASK(dir); if (!ep_is_iso(ep_reg)) { - ep_change_status(&ep_reg, dir, EP_STAT_NAK); + // Only knock a genuinely STALLED endpoint down to NAK (the class then re-arms it). If the + // endpoint is armed (VALID) - e.g. a clear-halt used purely to reset the data toggle, as in + // usbtest case 29 - leave STAT untouched so the in-flight transfer isn't disarmed with no + // completion, which would leak the usbd claim and starve the endpoint. Masking the STAT bits + // to 0 writes no toggle, so an armed/idle endpoint keeps its current status. + const uint8_t stat_pos = (uint8_t) (U_EPTX_STAT_Pos + (dir == TUSB_DIR_IN ? 0u : 8u)); + if (((ep_reg >> stat_pos) & 0x3u) == EP_STAT_STALL) { + ep_change_status(&ep_reg, dir, EP_STAT_NAK); + } else { + ep_reg &= ~EP_STAT_MASK(dir); + } } ep_change_dtog(&ep_reg, dir, 0); // Reset to DATA0 ep_write(ep_idx, ep_reg, true); diff --git a/src/portable/st/stm32_fsdev/fsdev_apm32.h b/src/portable/st/stm32_fsdev/fsdev_apm32.h new file mode 100644 index 000000000..5031445e8 --- /dev/null +++ b/src/portable/st/stm32_fsdev/fsdev_apm32.h @@ -0,0 +1,77 @@ +/* + * The MIT License (MIT) + * + * Copyright (c) 2024, hathach (tinyusb.org) + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + */ +#ifndef TUSB_FSDEV_APM32_H +#define TUSB_FSDEV_APM32_H + +#include "common/tusb_compiler.h" + +#if CFG_TUSB_MCU == OPT_MCU_APM32F0XX + #include "apm32f0xx.h" +#endif + +#define FSDEV_USE_SBUF_ISO 0 +#define FSDEV_REG_BASE ((uint32_t)(USBD_BASE)) +#define FSDEV_PMA_BASE ((uint32_t)(USBD_BASE + 0x400UL)) + +#ifndef CFG_TUD_FSDEV_DOUBLE_BUFFERED_ISO_EP + #define CFG_TUD_FSDEV_DOUBLE_BUFFERED_ISO_EP 0 +#endif + +//--------------------------------------------------------------------+ +// +//--------------------------------------------------------------------+ + +static const IRQn_Type fsdev_irq[] = { + USBD_IRQn +}; +enum { FSDEV_IRQ_NUM = TU_ARRAY_SIZE(fsdev_irq) }; + +TU_ATTR_ALWAYS_INLINE static inline void fsdev_int_enable(uint8_t rhport) { + (void)rhport; + for (uint8_t i = 0; i < FSDEV_IRQ_NUM; i++) { + NVIC_EnableIRQ(fsdev_irq[i]); + } +} + +TU_ATTR_ALWAYS_INLINE static inline void fsdev_int_disable(uint8_t rhport) { + (void)rhport; + for (uint8_t i = 0; i < FSDEV_IRQ_NUM; i++) { + NVIC_DisableIRQ(fsdev_irq[i]); + } +} + +TU_ATTR_ALWAYS_INLINE static inline void fsdev_disconnect(uint8_t rhport) { + (void) rhport; + FSDEV_REG->CNTR |= U_CNTR_PDWN; + FSDEV_REG->BCDR &= ~U_BCDR_DPPU; +} + +TU_ATTR_ALWAYS_INLINE static inline void fsdev_connect(uint8_t rhport) { + (void) rhport; + FSDEV_REG->CNTR &= ~U_CNTR_PDWN; + FSDEV_REG->BCDR |= U_BCDR_DPPU; +} + +#endif diff --git a/src/portable/st/stm32_fsdev/fsdev_at32.h b/src/portable/st/stm32_fsdev/fsdev_at32.h index 212c3b86d..9138dc101 100644 --- a/src/portable/st/stm32_fsdev/fsdev_at32.h +++ b/src/portable/st/stm32_fsdev/fsdev_at32.h @@ -17,7 +17,7 @@ #define FSDEV_USE_SBUF_ISO 0 #define FSDEV_REG_BASE (APB1PERIPH_BASE + 0x00005C00UL) -#define FSDEV_PMA_BASE (APB1PERIPH_BASE + 0x00006000UL) +#define FSDEV_PMA_BASE (APB1PERIPH_BASE + 0x00007800UL) #ifndef CFG_TUD_FSDEV_DOUBLE_BUFFERED_ISO_EP #define CFG_TUD_FSDEV_DOUBLE_BUFFERED_ISO_EP 0 diff --git a/src/portable/st/stm32_fsdev/fsdev_common.c b/src/portable/st/stm32_fsdev/fsdev_common.c index 2b573899a..9c9233d12 100644 --- a/src/portable/st/stm32_fsdev/fsdev_common.c +++ b/src/portable/st/stm32_fsdev/fsdev_common.c @@ -33,6 +33,11 @@ void fsdev_core_reset(void) { // Clear pending interrupts FSDEV_REG->ISTR = 0; + + #if (CFG_TUSB_MCU == OPT_MCU_AT32F403A_407) || (CFG_TUSB_MCU == OPT_MCU_AT32F413) + // Enable larger PMA area + CRM->misc1_bit.usbbufs = 1; + #endif } // De-initialize the USB Core diff --git a/src/portable/st/stm32_fsdev/fsdev_common.h b/src/portable/st/stm32_fsdev/fsdev_common.h index ebde44bf7..00d9b513a 100644 --- a/src/portable/st/stm32_fsdev/fsdev_common.h +++ b/src/portable/st/stm32_fsdev/fsdev_common.h @@ -284,6 +284,8 @@ typedef struct { #include "fsdev_ch32.h" #elif defined(TUP_USBIP_FSDEV_AT32) #include "fsdev_at32.h" +#elif defined(TUP_USBIP_FSDEV_APM32) + #include "fsdev_apm32.h" #else #error "Unknown USB IP" #endif diff --git a/src/portable/synopsys/dwc2/dcd_dwc2.c b/src/portable/synopsys/dwc2/dcd_dwc2.c index 6c88b4f27..b2f1a93a4 100644 --- a/src/portable/synopsys/dwc2/dcd_dwc2.c +++ b/src/portable/synopsys/dwc2/dcd_dwc2.c @@ -393,10 +393,6 @@ static void edpt_schedule_packets(uint8_t rhport, const uint8_t epnum, const uin } dep->diepdma = (uintptr_t) xfer->buffer; dep->diepctl = depctl.value; // enable endpoint - // Advance buffer pointer for EP0 - if (epnum == 0) { - xfer->buffer += total_bytes; - } } else #endif { @@ -732,6 +728,8 @@ static void handle_bus_reset(uint8_t rhport) { tu_memclr(xfer_status, sizeof(xfer_status)); + _dcd_data.ep0_pending[TUSB_DIR_OUT] = 0; + _dcd_data.ep0_pending[TUSB_DIR_IN] = 0; _dcd_data.sof_en = false; _dcd_data.allocated_epin_count = 0; @@ -1009,6 +1007,10 @@ static void handle_epout_dma(uint8_t rhport, uint8_t epnum, dwc2_doepint_t doepi if (edpt_is_enabled(epin0)) { edpt_disable(rhport, 0x80, false); } + // a new SETUP aborts any in-progress control transfer: drop leftover EP0 chunking state so a + // stale latched completion cannot re-arm from it + _dcd_data.ep0_pending[TUSB_DIR_OUT] = 0; + _dcd_data.ep0_pending[TUSB_DIR_IN] = 0; dcd_dcache_invalidate(_dcd_usbbuf.setup_buffer, sizeof(_dcd_usbbuf.setup_buffer)); @@ -1029,24 +1031,37 @@ static void handle_epout_dma(uint8_t rhport, uint8_t epnum, dwc2_doepint_t doepi // only handle data skip if it is setup or status related // Normal OUT transfer complete if (!doepint_bm.status_phase_rx && !doepint_bm.setup_packet_rx) { + xfer_ctl_t* xfer = XFER_CTL_BASE(epnum, TUSB_DIR_OUT); if ((epnum == 0) && _dcd_data.ep0_pending[TUSB_DIR_OUT]) { - // EP0 can only handle one packet Schedule another packet to be received. + // EP0 can only handle one packet: invalidate and advance past the received bytes, then + // schedule the next. + if (xfer->buffer != NULL) { + dcd_dcache_invalidate(xfer->buffer, CFG_TUD_ENDPOINT0_SIZE); + xfer->buffer += CFG_TUD_ENDPOINT0_SIZE; + } edpt_schedule_packets(rhport, epnum, TUSB_DIR_OUT); } else { dwc2_dep_t* epout = &dwc2->epout[epnum]; - xfer_ctl_t* xfer = XFER_CTL_BASE(epnum, TUSB_DIR_OUT); // determine actual received bytes const dwc2_ep_tsize_t tsiz = {.value = epout->tsiz}; const uint16_t remain = tsiz.xfer_size; xfer->total_len -= remain; + // EP0 invalidates only this (final) chunk's DMA-written bytes: DOEPDMA "is incremented on + // every AHB transaction" (databook 7.1.83), i.e. it points past the last word written. + // Read it before dma_setup_prepare() re-targets it at the setup buffer + uint16_t inval_len = xfer->total_len; + if (epnum == 0) { + inval_len = (uint16_t)(epout->doepdma - (uintptr_t)xfer->buffer); + } + // prepare EP0 for next setup if(epnum == 0) { dma_setup_prepare(rhport); } - dcd_dcache_invalidate(xfer->buffer, xfer->total_len); + dcd_dcache_invalidate(xfer->buffer, inval_len); dcd_event_xfer_complete(rhport, epnum, xfer->total_len, XFER_RESULT_SUCCESS, true); } } @@ -1058,7 +1073,10 @@ static void handle_epin_dma(uint8_t rhport, uint8_t epnum, dwc2_diepint_t diepin if (diepint_bm.xfer_complete) { if ((epnum == 0) && _dcd_data.ep0_pending[TUSB_DIR_IN]) { - // EP0 can only handle one packet. Schedule another packet to be transmitted. + // EP0 can only handle one packet: advance past the sent bytes, then schedule the next. + if (xfer->buffer != NULL) { + xfer->buffer += CFG_TUD_ENDPOINT0_SIZE; + } edpt_schedule_packets(rhport, epnum, TUSB_DIR_IN); } else { dcd_event_xfer_complete(rhport, epnum | TUSB_DIR_IN_MASK, xfer->total_len, XFER_RESULT_SUCCESS, true); @@ -1125,7 +1143,12 @@ static void handle_incomplete_iso_in(uint8_t rhport) { xfer_ctl_t *xfer = XFER_CTL_BASE(epnum, TUSB_DIR_IN); if (xfer->iso_retry > 0) { xfer->iso_retry--; - // Restart ISO transfe: re-write TSIZ and CTL + // Restart ISO transfer: re-write DMA address, TSIZ, and CTL + #if CFG_TUD_DWC2_DMA_ENABLE + if (dma_device_enabled(dwc2)) { + epin->diepdma = (uintptr_t) xfer->buffer; + } + #endif dwc2_ep_tsize_t deptsiz = {.value = 0}; deptsiz.xfer_size = xfer->total_len; deptsiz.packet_count = tu_div_ceil(xfer->total_len, xfer->max_size); diff --git a/src/portable/wch/ch32_usbfs_reg.h b/src/portable/wch/ch32_usbfs_reg.h index 8bac103fe..5b037281f 100644 --- a/src/portable/wch/ch32_usbfs_reg.h +++ b/src/portable/wch/ch32_usbfs_reg.h @@ -179,6 +179,14 @@ #endif #endif +// CH32V20x/V30x/F20x USBFS gives endpoint 3 a 1023-byte isochronous packet (CH32FV2x_V3xRM ch23: +// every endpoint is 64 B except EP3 = 1023 B, from EP3's 10-bit R16_UEP3_T_LEN field plus a single +// contiguous >=1023 B DMA buffer — NOT double-buffering, which only yields 2x64 B). +// CH32V103/X035/CH58x cap every endpoint at 64 B. dcd_ch32_usbfs.c reads this to size EP3's buffer. +#if CFG_TUSB_MCU == OPT_MCU_CH32V20X || CFG_TUSB_MCU == OPT_MCU_CH32V307 || CFG_TUSB_MCU == OPT_MCU_CH32F20X + #define CH32_USBFS_EP3_1023_BUFSIZE 1 +#endif + #ifdef __GNUC__ #pragma GCC diagnostic pop #endif diff --git a/src/portable/wch/dcd_ch32_usbfs.c b/src/portable/wch/dcd_ch32_usbfs.c index a6458748a..ec521224e 100644 --- a/src/portable/wch/dcd_ch32_usbfs.c +++ b/src/portable/wch/dcd_ch32_usbfs.c @@ -16,6 +16,17 @@ /* private defines */ #define EP_MAX (8) + // EP3 IN buffer size. CH32V20x/V30x/F20x USBFS support full-speed iso packets up to 1023 B on + // endpoint 3 (every other endpoint is 64 B); those parts set CH32_USBFS_EP3_1023_BUFSIZE in + // ch32_usbfs_reg.h. V103/X035/CH58x cap every endpoint at 64 B. Overridable per project. + #ifndef CFG_TUD_WCH_USBFS_EP3_BUFSIZE + #ifdef CH32_USBFS_EP3_1023_BUFSIZE + #define CFG_TUD_WCH_USBFS_EP3_BUFSIZE 1023 + #else + #define CFG_TUD_WCH_USBFS_EP3_BUFSIZE 64 + #endif + #endif + // Struct-based EP register access (uniform layout). CH58X has a different register map and // defines EP_DMA/EP_TX_LEN/EP_CTRL itself in ch32_usbfs_reg.h. #if CFG_TUSB_MCU == OPT_MCU_CH583 @@ -107,7 +118,7 @@ struct usb_xfer { static struct { bool ep0_tog; - bool isochronous[EP_MAX]; + bool isochronous[EP_MAX][2]; // per [ep][dir]: an ep number may be iso in one direction struct usb_xfer xfer[EP_MAX][2]; #ifdef CH32_USBFS_EP4_SHARES_EP0 // CH58X buffers laid out by hand so EP0/EP4 don't burn two unused buffer[] slots. EP0 and EP4 @@ -123,21 +134,23 @@ static struct { TU_ATTR_ALIGNED(4) uint8_t ep6_buffer[2][64]; TU_ATTR_ALIGNED(4) uint8_t ep7_buffer[2][64]; #else + // Every endpoint gets a 64-byte OUT + 64-byte IN buffer. TU_ATTR_ALIGNED(4) uint8_t buffer[EP_MAX][2][64]; - // EP3 IN gets an enlarged buffer for full-speed isochronous (packets up to 1023 B). + #if CFG_TUD_WCH_USBFS_EP3_BUFSIZE > 64 + // ...except EP3, which supports full-speed iso packets up to 1023 B on CH32V20x/V30x/F20x, so its + // IN buffer is enlarged (OUT stays 64 B; an OUT transfer >64 B on EP3 would overwrite queued IN). TU_ATTR_ALIGNED(4) struct { - // OUT transfers >64 bytes will overwrite queued IN data! uint8_t out[64]; - uint8_t in[1023]; + uint8_t in[CFG_TUD_WCH_USBFS_EP3_BUFSIZE]; uint8_t pad; } ep3_buffer; + #endif #endif } data; // DMA / copy buffer pointers per endpoint. The WCH USBFS buffer holds OUT (RX) at offset 0 and -// IN (TX) at +64; EP0 is half-duplex and reuses its OUT chunk for IN; EP3 has an enlarged IN -// buffer for throughput. On CH58X, EP0/EP4 share ep0_ep4_buffer and the regular endpoints use -// their own named buffer (see the struct above). +// IN (TX) at +64; EP0 is half-duplex and reuses its OUT chunk for IN. On CH58X, EP0/EP4 share +// ep0_ep4_buffer and the regular endpoints use their own named buffer (see the struct above). #ifdef CH32_USBFS_EP4_SHARES_EP0 // OUT base of the regular CH58X endpoints (EP1/2/3/5/6/7; EP0/EP4 share ep0_ep4_buffer). static inline uint8_t* ch58x_ep_buffer(uint8_t ep) { @@ -157,7 +170,9 @@ static inline uint32_t ep_dma_addr(uint8_t ep) { if (ep == 0 || ep == 4) { return (uint32_t) &data.ep0_ep4_buffer[0]; } // EP4 shares EP0's DMA return (uint32_t) ch58x_ep_buffer(ep); #else - if (ep == 3) { return (uint32_t) &data.ep3_buffer.out[0]; } + #if CFG_TUD_WCH_USBFS_EP3_BUFSIZE > 64 + if (ep == 3) { return (uint32_t) &data.ep3_buffer.out[0]; } // EP3 has an enlarged IN buffer + #endif return (uint32_t) &data.buffer[ep][0]; #endif } @@ -168,7 +183,9 @@ static inline uint8_t* ep_out_buf(uint8_t ep) { if (ep == 4) { return &data.ep0_ep4_buffer[64]; } return ch58x_ep_buffer(ep); #else + #if CFG_TUD_WCH_USBFS_EP3_BUFSIZE > 64 if (ep == 3) { return data.ep3_buffer.out; } + #endif return data.buffer[ep][TUSB_DIR_OUT]; #endif } @@ -180,7 +197,9 @@ static inline uint8_t* ep_in_buf(uint8_t ep) { return ch58x_ep_buffer(ep) + 64; // IN at +64 within the endpoint's 128-byte buffer #else if (ep == 0) { return data.buffer[0][TUSB_DIR_OUT]; } // EP0 half-duplex: IN reuses OUT chunk - if (ep == 3) { return data.ep3_buffer.in; } + #if CFG_TUD_WCH_USBFS_EP3_BUFSIZE > 64 + if (ep == 3) { return data.ep3_buffer.in; } // enlarged IN buffer for full-speed iso + #endif return data.buffer[ep][TUSB_DIR_IN]; #endif } @@ -202,9 +221,8 @@ static void update_in(uint8_t rhport, uint8_t ep, bool force) { if (force || xfer->len) { size_t len = TU_MIN(xfer->max_size, xfer->len); #if CFG_TUSB_MCU == OPT_MCU_CH583 - // Every CH58x endpoint buffer is 64 bytes. Isochronous (which would push max_size up to 1023) - // is refused in dcd_edpt_iso_alloc(), but some classes (e.g. video) ignore that result, so cap - // the copy here to guarantee we never write past the buffer into a neighbouring endpoint's. + // Every CH58x endpoint buffer is 64 bytes; cap the copy so an iso mps a class mistakenly set + // larger can't write past the buffer into a neighbouring endpoint's. len = TU_MIN(len, 64u); #endif memcpy(ep_in_buf(ep), xfer->buffer, len); @@ -216,7 +234,7 @@ static void update_in(uint8_t rhport, uint8_t ep, bool force) { if (ep == 0) { ep_tx_ctrl_set(0, USBFS_EP_T_RES_ACK | (data.ep0_tog ? USBFS_EP_T_TOG : 0)); data.ep0_tog = !data.ep0_tog; - } else if (data.isochronous[ep]) { + } else if (data.isochronous[ep][TUSB_DIR_IN]) { ep_tx_set_response(ep, USBFS_EP_T_RES_NYET); } else { ep_tx_set_response(ep, USBFS_EP_T_RES_ACK); @@ -225,7 +243,7 @@ static void update_in(uint8_t rhport, uint8_t ep, bool force) { xfer->valid = false; if (ep == 0) { ep_tx_ctrl_set(0, USBFS_EP_T_RES_NAK | (data.ep0_tog ? USBFS_EP_T_TOG : 0)); - } else if (!data.isochronous[ep]) { + } else if (!data.isochronous[ep][TUSB_DIR_IN]) { ep_tx_set_response(ep, USBFS_EP_T_RES_NAK); } dcd_event_xfer_complete(rhport, ep | TUSB_DIR_IN_MASK, xfer->processed_len, XFER_RESULT_SUCCESS, true); @@ -254,7 +272,7 @@ static void update_out(uint8_t rhport, uint8_t ep, size_t rx_len) { ep_rx_set_response(0, USBFS_EP_R_RES_NAK); } else { uint8_t rx_res = - data.isochronous[ep] ? USBFS_EP_R_RES_NYET : (xfer->valid ? USBFS_EP_R_RES_ACK : USBFS_EP_R_RES_NAK); + data.isochronous[ep][TUSB_DIR_OUT] ? USBFS_EP_R_RES_NYET : (xfer->valid ? USBFS_EP_R_RES_ACK : USBFS_EP_R_RES_NAK); ep_rx_set_response(ep, rx_res); } } @@ -319,12 +337,14 @@ void dcd_int_handler(uint8_t rhport) { // Drop an OUT packet whose data toggle doesn't match what we expect -- a host retransmit // after a lost ACK, or a host that doesn't alternate DATA0/DATA1. The hardware auto-toggle // does not reject these on its own, so the check is needed on every variant. EP0 keeps its - // own toggle via the SETUP/status flow and is exempt. - if (ep != 0 && !(int_st & USBFS_INT_ST_TOG_OK)) { break; } + // own toggle via the SETUP/status flow and is exempt; isochronous is DATA0-only (no toggle), + // so its packets must not be toggle-checked. + if (ep != 0 && !data.isochronous[ep][TUSB_DIR_OUT] && !(int_st & USBFS_INT_ST_TOG_OK)) { break; } #ifdef CH32_USBFS_EP_MANUAL_TOG // CH58x has no hardware auto-toggle: advance the expected RX toggle after each accepted packet // (EP0 included -- it also has no auto-toggle and a control-OUT data stage can span packets). - EP_CTRL(ep) ^= USBFS_EPC_R_TOG; + // Iso endpoints are DATA0-only, so leave them alone (matches the PID_IN path). + if (!data.isochronous[ep][TUSB_DIR_OUT]) { EP_CTRL(ep) ^= USBFS_EPC_R_TOG; } #endif update_out(rhport, ep, rx_len); break; @@ -333,7 +353,8 @@ void dcd_int_handler(uint8_t rhport) { case PID_IN: #ifdef CH32_USBFS_EP_MANUAL_TOG // Manual toggle: flip the TX toggle after each ACK'd IN packet (EP0 manages its own). - if (ep != 0) { EP_CTRL(ep) ^= USBFS_EPC_T_TOG; } + // Isochronous transfers are DATA0-only (no toggle), so leave iso endpoints alone. + if (ep != 0 && !data.isochronous[ep][TUSB_DIR_IN]) { EP_CTRL(ep) ^= USBFS_EPC_T_TOG; } #endif update_in(rhport, ep, false); break; @@ -443,6 +464,7 @@ bool dcd_edpt_open(uint8_t rhport, const tusb_desc_endpoint_t *desc_ep) { uint8_t dir = tu_edpt_dir(desc_ep->bEndpointAddress); TU_ASSERT(ep < EP_MAX); + data.isochronous[ep][dir] = false; // (re)opening as a non-iso endpoint clears any stale iso flag data.xfer[ep][dir].max_size = tu_edpt_packet_size(desc_ep); if (ep != 0) { @@ -464,31 +486,38 @@ void dcd_edpt_close_all(uint8_t rhport) { bool dcd_edpt_iso_alloc(uint8_t rhport, uint8_t ep_addr, uint16_t largest_packet_size) { (void)rhport; - (void)ep_addr; - (void)largest_packet_size; -#if CFG_TUSB_MCU == OPT_MCU_CH583 - // No isochronous support on CH58x: its 8-bit T_LEN caps a packet at 255B and the endpoints use - // plain 64-byte buffers, so accepting an iso max_size (up to 1023) would let update_in()/ - // update_out() run off the end of the buffer into neighbouring ones. Refuse it outright. - return false; -#else uint8_t ep = tu_edpt_number(ep_addr); uint8_t dir = tu_edpt_dir(ep_addr); + TU_ASSERT(ep < EP_MAX); - data.isochronous[ep] = true; + // Endpoint buffers are 64 B, except EP3 IN which is enlarged for full-speed iso on the parts that + // support 1023-byte EP3 packets (CH32V20x/V30x/F20x; CFG_TUD_WCH_USBFS_EP3_BUFSIZE). Reject a + // larger mps rather than running off the end into the neighbouring endpoint's memory. + uint16_t max_packet = 64; +#if CFG_TUD_WCH_USBFS_EP3_BUFSIZE > 64 + if (ep == 3 && dir == TUSB_DIR_IN) { max_packet = CFG_TUD_WCH_USBFS_EP3_BUFSIZE; } +#endif + TU_VERIFY(largest_packet_size <= max_packet); + + data.isochronous[ep][dir] = true; data.xfer[ep][dir].max_size = largest_packet_size; return true; -#endif } bool dcd_edpt_iso_activate(uint8_t rhport, const tusb_desc_endpoint_t *desc_ep) { (void)rhport; - (void)desc_ep; -#if CFG_TUSB_MCU == OPT_MCU_CH583 - return false; // CH58x has no isochronous support (see dcd_edpt_iso_alloc) -#else + const uint8_t ep = tu_edpt_number(desc_ep->bEndpointAddress); + const uint8_t dir = tu_edpt_dir(desc_ep->bEndpointAddress); + + // a transfer armed before SET_INTERFACE survives to here (no dcd close on this port): drop the + // stale descriptor and NAK the endpoint so the ISR can't complete it against the old buffer + data.xfer[ep][dir].valid = false; + if (dir == TUSB_DIR_IN) { + ep_tx_set_response(ep, USBFS_EP_T_RES_NAK); + } else { + ep_rx_set_response(ep, USBFS_EP_R_RES_NAK); + } return true; -#endif } bool dcd_edpt_xfer(uint8_t rhport, uint8_t ep_addr, uint8_t *buffer, uint16_t total_bytes, bool is_isr) { @@ -510,7 +539,7 @@ bool dcd_edpt_xfer(uint8_t rhport, uint8_t ep_addr, uint8_t *buffer, uint16_t to if (dir == TUSB_DIR_IN) { update_in(rhport, ep, true); } else { - uint8_t rx_res = data.isochronous[ep] ? USBFS_EP_R_RES_NYET : USBFS_EP_R_RES_ACK; + uint8_t rx_res = data.isochronous[ep][TUSB_DIR_OUT] ? USBFS_EP_R_RES_NYET : USBFS_EP_R_RES_ACK; ep_rx_set_response(ep, rx_res); } dcd_int_enable(rhport); @@ -546,9 +575,15 @@ void dcd_edpt_clear_stall(uint8_t rhport, uint8_t ep_addr) { ep_rx_ctrl_set(0, USBFS_EP_R_RES_ACK); } } else { - // clear-stall resets the toggle to DATA0 (USB spec); manual-toggle parts then re-sync via ISR + // clear-stall resets the toggle to DATA0 (USB spec); manual-toggle parts then re-sync via ISR. + // Preserve an in-flight receive: if a read is still armed (the class driver considers it + // submitted and won't re-arm), fall back to ACK, not NAK, or the endpoint NAKs forever and the + // host times out (usbtest toggle test 29 clears the halt between bulk writes on an armed EP). if (dir == TUSB_DIR_OUT) { - ep_rx_ctrl_set(ep, EP_R_AUTO_TOG | USBFS_EP_R_RES_NAK); + uint8_t res = data.xfer[ep][TUSB_DIR_OUT].valid + ? (data.isochronous[ep][TUSB_DIR_OUT] ? USBFS_EP_R_RES_NYET : USBFS_EP_R_RES_ACK) + : USBFS_EP_R_RES_NAK; + ep_rx_ctrl_set(ep, EP_R_AUTO_TOG | res); } else { ep_tx_ctrl_set(ep, EP_T_AUTO_TOG | USBFS_EP_T_RES_NAK); } diff --git a/src/portable/wch/dcd_ch32_usbhs.c b/src/portable/wch/dcd_ch32_usbhs.c index 0c154f5ce..577f86582 100644 --- a/src/portable/wch/dcd_ch32_usbhs.c +++ b/src/portable/wch/dcd_ch32_usbhs.c @@ -348,8 +348,16 @@ void dcd_edpt_clear_stall(uint8_t rhport, uint8_t ep_addr) { const tusb_dir_t dir = tu_edpt_dir(ep_addr); if (dir == TUSB_DIR_OUT) { - EP_RX_CTRL(ep_num) = USBHS_EP_R_RES_NAK | USBHS_EP_R_TOG_0; - ep_data_tog[ep_num][TUSB_DIR_OUT] = false; + ep_data_tog[ep_num][TUSB_DIR_OUT] = false; // clear-halt resets the toggle to DATA0 + xfer_ctl_t *xfer = XFER_CTL_BASE(ep_num, TUSB_DIR_OUT); + if (xfer->valid) { + // A receive is still armed (the class driver considers it submitted and won't re-arm it); + // re-queue it (ACK/NYET) instead of leaving it NAKing, or the endpoint NAKs forever after + // clear-halt (usbtest toggle test 29 clears the halt on an armed bulk-OUT pipe). + queue_out_packet(ep_num, xfer); + } else { + EP_RX_CTRL(ep_num) = USBHS_EP_R_RES_NAK | USBHS_EP_R_TOG_0; + } } else { EP_TX_CTRL(ep_num) = USBHS_EP_T_RES_NAK | USBHS_EP_T_TOG_0; ep_data_tog[ep_num][TUSB_DIR_IN] = false; diff --git a/src/tusb_option.h b/src/tusb_option.h index 7a636657f..8fafde699 100644 --- a/src/tusb_option.h +++ b/src/tusb_option.h @@ -204,6 +204,12 @@ // HPMicro #define OPT_MCU_HPM 2600 ///< HPMicro +// Puya +#define OPT_MCU_PY32F0 2700 ///< Puya PY32F0 + +// Geehy +#define OPT_MCU_APM32F0XX 2800 ///< Geehy APM32F0xx + // Check if configured MCU is one of listed // Apply TU_MCU_IS_EQUAL with || as separator to list of input #define TU_MCU_IS_EQUAL(_m) (CFG_TUSB_MCU == (_m)) @@ -240,6 +246,16 @@ #define OPT_MODE_SPEED_MASK 0xff00u //--------------------------------------------------------------------+ +// Validation Level +// Optional validation of data received from the USB peer, traded against code size. Coverage is parser-specific +// and expanded incrementally. CFG_TUSB_VALIDATION_LEVEL sets the default for both device and host; use the +// CFG_TUD_VALIDATION_LEVEL / CFG_TUH_VALIDATION_LEVEL overrides when the roles need different policies. +//--------------------------------------------------------------------+ +#define TUSB_VALIDATION_NONE 0 ///< trusted peers, minimal code size +#define TUSB_VALIDATION_BASIC 1 ///< structural memory-safety checks where supported +#define TUSB_VALIDATION_STRICT 2 ///< additional USB and class-specific conformance checks + +//--------------------------------------------------------------------+ // Include tusb_config.h //--------------------------------------------------------------------+ @@ -253,6 +269,46 @@ #include "common/tusb_mcu.h" //--------------------------------------------------------------------+ +// Validation Options +//--------------------------------------------------------------------+ + +#ifndef CFG_TUSB_VALIDATION_LEVEL + #define CFG_TUSB_VALIDATION_LEVEL TUSB_VALIDATION_BASIC +#endif + +#ifndef CFG_TUD_VALIDATION_LEVEL + #define CFG_TUD_VALIDATION_LEVEL CFG_TUSB_VALIDATION_LEVEL +#endif + +#ifndef CFG_TUH_VALIDATION_LEVEL + #define CFG_TUH_VALIDATION_LEVEL CFG_TUSB_VALIDATION_LEVEL +#endif + +#if (CFG_TUSB_VALIDATION_LEVEL < TUSB_VALIDATION_NONE) || \ + (CFG_TUSB_VALIDATION_LEVEL > TUSB_VALIDATION_STRICT) + #error "CFG_TUSB_VALIDATION_LEVEL must be TUSB_VALIDATION_NONE, TUSB_VALIDATION_BASIC, or TUSB_VALIDATION_STRICT" +#endif + +#if (CFG_TUD_VALIDATION_LEVEL < TUSB_VALIDATION_NONE) || \ + (CFG_TUD_VALIDATION_LEVEL > TUSB_VALIDATION_STRICT) + #error "CFG_TUD_VALIDATION_LEVEL must be TUSB_VALIDATION_NONE, TUSB_VALIDATION_BASIC, or TUSB_VALIDATION_STRICT" +#endif + +#if (CFG_TUH_VALIDATION_LEVEL < TUSB_VALIDATION_NONE) || \ + (CFG_TUH_VALIDATION_LEVEL > TUSB_VALIDATION_STRICT) + #error "CFG_TUH_VALIDATION_LEVEL must be TUSB_VALIDATION_NONE, TUSB_VALIDATION_BASIC, or TUSB_VALIDATION_STRICT" +#endif + +// Validation conditions are short-circuited at lower levels and compile out when the result is unused. +#define TUD_VALIDATION_CHECK(_level, _cond) ((CFG_TUD_VALIDATION_LEVEL < (_level)) || (_cond)) +#define TUH_VALIDATION_CHECK(_level, _cond) ((CFG_TUH_VALIDATION_LEVEL < (_level)) || (_cond)) + +#define TUD_VALIDATE_BASIC(_cond) TUD_VALIDATION_CHECK(TUSB_VALIDATION_BASIC, _cond) +#define TUH_VALIDATE_BASIC(_cond) TUH_VALIDATION_CHECK(TUSB_VALIDATION_BASIC, _cond) +#define TUD_VALIDATE_STRICT(_cond) TUD_VALIDATION_CHECK(TUSB_VALIDATION_STRICT, _cond) +#define TUH_VALIDATE_STRICT(_cond) TUH_VALIDATION_CHECK(TUSB_VALIDATION_STRICT, _cond) + +//--------------------------------------------------------------------+ // USBIP //--------------------------------------------------------------------+ @@ -350,7 +406,9 @@ #if defined(TUP_USBIP_MUSB) #define CFG_TUD_EDPT_DEDICATED_HWFIFO 1 #define CFG_TUSB_FIFO_HWFIFO_DATA_STRIDE 4 // 32 bit data - #define CFG_TUSB_FIFO_HWFIFO_DATA_ODD_16BIT_ACCESS // allow odd 16bit access + #if !defined(TUP_USBIP_MUSB_PY32) + #define CFG_TUSB_FIFO_HWFIFO_DATA_ODD_16BIT_ACCESS // allow odd 16bit access + #endif #define CFG_TUSB_FIFO_HWFIFO_DATA_ODD_8BIT_ACCESS // allow odd 8bit access #define CFG_TUSB_FIFO_HWFIFO_ADDR_STRIDE 0 // fixed hwfifo #endif @@ -694,6 +752,7 @@ #ifndef CFG_TUH_ENUMERATION_BUFSIZE #define CFG_TUH_ENUMERATION_BUFSIZE 256 #endif + #endif // CFG_TUH_ENABLED // Attribute to place data in accessible RAM for host controller (default: CFG_TUSB_MEM_SECTION) diff --git a/test/fuzz/device/net_ncm/fuzz.c b/test/fuzz/device/net_ncm/fuzz.c index a93c144f7..3052636d8 100644 --- a/test/fuzz/device/net_ncm/fuzz.c +++ b/test/fuzz/device/net_ncm/fuzz.c @@ -47,6 +47,9 @@ bool usbd_open_edpt_pair(uint8_t rhport, uint8_t const *p_desc, uint8_t ep_count (void) rhport; (void) p_desc; (void) ep_count; (void) xfer_type; (void) ep_out; (void) ep_in; return true; } +void usbd_defer_func(osal_task_func_t func, void *param, bool in_isr) { + (void) func; (void) param; (void) in_isr; +} bool tud_control_xfer(uint8_t rhport, tusb_control_request_t const *request, void *buffer, uint16_t len) { (void) rhport; (void) request; (void) buffer; (void) len; return true; diff --git a/test/hil/board_lock.py b/test/hil/board_lock.py deleted file mode 100755 index c35e13705..000000000 --- a/test/hil/board_lock.py +++ /dev/null @@ -1,254 +0,0 @@ -#!/usr/bin/env python3 -"""Per-board advisory locks for the HIL rig. - -Arbitrates board access between dev sessions and CI's hil_test.py without -stopping the actions-runner. Locks are kernel flocks: the kernel releases -them automatically when the holder process dies, and holders clear their -lock-file record on release so records stay truthful (/tmp also clears on -reboot). - -Usage: - board_lock.py hold BOARD [BOARD...] --reason TEXT - board_lock.py hold --all [--config CONFIG.json] --reason TEXT - board_lock.py release BOARD [BOARD...] | release --all - board_lock.py status - -A holder process holds ALL boards given in one `hold` call; releasing any of -them kills that holder and releases all of its boards. -""" -import argparse -import fcntl -import json -import os -import select -import signal -import sys -import time - -LOCK_DIR = '/tmp/tinyusb-hil-locks' - - -def lock_path(board: str) -> str: - return os.path.join(LOCK_DIR, f'{board}.lock') - - -def boards_from_config(config: str) -> list: - try: - with open(config) as f: - return [b['name'] for b in json.load(f)['boards']] - except (OSError, ValueError, KeyError) as e: - print(f'ERROR: cannot read board roster {config}: {e}', file=sys.stderr) - sys.exit(1) - - -def read_info(board: str): - try: - with open(lock_path(board)) as f: - return json.load(f) - except (OSError, ValueError): - return None - - -def is_locked(board: str) -> bool: - """True if the recorded holder process is still alive. - - Deliberately never touches the flock: even a momentary probe lock would - make a concurrent acquirer's LOCK_NB attempt fail spuriously. The flock - taken by acquirers themselves stays the only authority.""" - info = read_info(board) - pid = info.get('pid') if isinstance(info, dict) else None - if not isinstance(pid, int) or pid <= 0: - return False - try: - os.kill(pid, 0) - except ProcessLookupError: - return False - except PermissionError: - return True # alive but owned by another user (e.g. the CI runner) - return True - - -def cmd_hold(boards, reason): - os.makedirs(LOCK_DIR, exist_ok=True) - # No pre-check: the holder's own LOCK_NB flock is the only authority — a - # recorded pid may be stale or recycled (e.g. a live hil_test.py worker - # that already released this board's flock but not its record). - # The holder signals success through this pipe. A generic is_locked() - # poll would be fooled by a RIVAL invocation's flock — only the holder - # itself knows whether it won every board. - r_fd, w_fd = os.pipe() - pid = os.fork() - if pid > 0: - os.close(w_fd) - os.waitpid(pid, 0) # reap intermediate child - ready, _, _ = select.select([r_fd], [], [], 10) - ok = bool(ready) and os.read(r_fd, 1) == b'1' - os.close(r_fd) - if ok: - print(f'held: {", ".join(boards)}') - return 0 - for b in boards: - info = read_info(b) - if info: - print(f'ERROR: {b} locked: {info}', file=sys.stderr) - print('ERROR: holder failed to acquire locks', file=sys.stderr) - return 1 - # intermediate child: detach, then spawn the actual holder - os.setsid() - if os.fork() > 0: - os._exit(0) - # holder (grandchild): acquire all flocks, signal the parent, sleep until killed - os.close(r_fd) - # Keep the success pipe clear of fds 0-2: invoked with stdio closed, - # os.pipe() can land there and the dup2 loop below would clobber it. - if w_fd <= 2: - w_fd = fcntl.fcntl(w_fd, fcntl.F_DUPFD, 3) - # Detach stdio: a `hold` whose output is captured must see EOF when the - # front-end exits — the immortal holder must not keep that pipe open. - devnull = os.open(os.devnull, os.O_RDWR) - for std_fd in (0, 1, 2): - os.dup2(devnull, std_fd) - if devnull > 2: - os.close(devnull) - try: - handles = [] - for b in boards: - # O_RDWR without O_TRUNC: never truncate before the flock is - # held — a losing racer must not wipe the winner's holder info. - fd = os.open(lock_path(b), os.O_RDWR | os.O_CREAT, 0o666) - fh = os.fdopen(fd, 'r+') - fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB) - fh.truncate(0) - fh.seek(0) - json.dump({'pid': os.getpid(), 'reason': reason, - 'since': time.strftime('%Y-%m-%dT%H:%M:%S%z')}, fh) - fh.flush() - handles.append(fh) - except OSError: - try: - os.write(w_fd, b'0') - except OSError: - pass - os._exit(1) # lost a race; parent reports the failure - os.write(w_fd, b'1') - os.close(w_fd) - - def _bow_out(*_): - # clear the records before dying so read_info/status stay truthful - # (the kernel drops the flocks themselves on exit either way) - for h in handles: - try: - h.truncate(0) - except OSError: - pass - os._exit(0) - - signal.signal(signal.SIGTERM, _bow_out) - while True: - signal.pause() - - -def cmd_release(boards): - rc = 0 - victims = set() - for b in boards: - try: - fd = os.open(lock_path(b), os.O_RDWR) - except OSError: - continue # no lock file (or another user's): nothing we can release - fh = os.fdopen(fd, 'r+') - try: - fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB) - except OSError: - # flock genuinely held — never SIGTERM on a mere pid record: the - # pid may be recycled, or a live worker that already moved on. - fh.close() - info = read_info(b) or {} - pid = info.get('pid') - if info.get('reason') == 'hil_test.py': - print(f'ERROR: {b} is mid-test by hil_test.py (pid {pid}) — not killing a ' - 'CI run; wait for it to finish', file=sys.stderr) - rc = 1 - elif isinstance(pid, int) and pid > 0: - victims.add(pid) - else: - print(f'ERROR: {b} is held but its record is unreadable', file=sys.stderr) - rc = 1 - continue - # flock was free: only a stale record remained — clear it - try: - fh.truncate(0) - except OSError: - pass - fh.close() - for holder in sorted(victims): - try: - os.kill(holder, signal.SIGTERM) - print(f'released holder pid {holder}') - except ProcessLookupError: - pass - except PermissionError: - print(f'ERROR: holder pid {holder} belongs to another user — cannot signal it', - file=sys.stderr) - rc = 1 - time.sleep(0.3) - still = [b for b in boards if is_locked(b)] - if still: - print(f'ERROR: still locked: {", ".join(still)}', file=sys.stderr) - return 1 - return rc - - -def cmd_status(): - if not os.path.isdir(LOCK_DIR): - print('no locks') - return 0 - any_locked = False - for fn in sorted(os.listdir(LOCK_DIR)): - if not fn.endswith('.lock'): - continue - b = fn[:-5] - if is_locked(b): - any_locked = True - print(f'{b}: {read_info(b)}') - if not any_locked: - print('no locks') - return 0 - - -def main(): - ap = argparse.ArgumentParser(description=__doc__, - formatter_class=argparse.RawDescriptionHelpFormatter) - sub = ap.add_subparsers(dest='cmd', required=True) - p_hold = sub.add_parser('hold') - p_hold.add_argument('boards', nargs='*') - p_hold.add_argument('--all', action='store_true') - p_hold.add_argument('--config', - default=os.path.join(os.path.dirname(os.path.abspath(__file__)), - 'tinyusb.json'), - help='board roster JSON (default: tinyusb.json beside this script)') - p_hold.add_argument('--reason', required=True) - p_rel = sub.add_parser('release') - p_rel.add_argument('boards', nargs='*') - p_rel.add_argument('--all', action='store_true') - sub.add_parser('status') - a = ap.parse_args() - if a.cmd == 'hold': - boards = boards_from_config(a.config) if a.all else a.boards - if not boards: - ap.error('no boards given (name boards or use --all)') - sys.exit(cmd_hold(boards, a.reason)) - if a.cmd == 'release': - if a.all: - boards = ([fn[:-5] for fn in os.listdir(LOCK_DIR) if fn.endswith('.lock')] - if os.path.isdir(LOCK_DIR) else []) - else: - boards = a.boards - if not boards: - ap.error('no boards given (name boards or use --all)') - sys.exit(cmd_release(boards)) - sys.exit(cmd_status()) - - -if __name__ == '__main__': - main() diff --git a/test/hil/helper/__init__.py b/test/hil/helper/__init__.py new file mode 100644 index 000000000..a080a2f55 --- /dev/null +++ b/test/hil/helper/__init__.py @@ -0,0 +1,4 @@ +# Marks helper/ as a REGULAR package. Without this it is only a PEP 420 namespace portion, +# and a regular package named `helper` anywhere on sys.path wins over it even though +# test/hil is sys.path[0] -- one transitive pip install would break every HIL entry point +# at import. `helper` is a real distribution name on PyPI. diff --git a/test/hil/helper/hil_health.py b/test/hil/helper/hil_health.py new file mode 100644 index 000000000..92f0accc8 --- /dev/null +++ b/test/hil/helper/hil_health.py @@ -0,0 +1,380 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +"""Shutting a wedged HIL run down: kill what the workers spawned, then report. + +A device whose usbfs node is held by a D-state process cannot be freed -- SIGKILL is not +delivered in uninterruptible sleep -- so the goal is never to fix the rig from here. It is +to free the runner's single job slot and leave a report naming what survived, instead of +letting the job sit until GitHub cancels it with nothing to show. + +Deliberately shallow. We SIGKILL the process groups the workers spawned, wait a grace, +and report whoever is still alive; we do not re-scan groups, prove pid ownership or +escalate through sudo. A root-owned survivor is named in the report for hil_pool_check +and the usb-kernel-recover skill to deal with -- signalling a pid we cannot prove is ours +is the worse failure, and the job ceiling backstops whatever this misses. + +Everything here is stdlib-only and reads /proc unprivileged (dmesg is restricted on the +rig), which keeps it importable -- and testable -- on a bare runner. +""" +import os +import signal +import threading +import time +from pathlib import Path + +PROC = Path('/proc') + + +# How long to let a SIGKILL land before calling a process a survivor. Generous enough to +# cover scheduling delay on a loaded rig, short enough that a fleet-wide sweep stays quick. +CONFIRM_KILL_GRACE = 2.0 + + +def _p(*args, **kwargs) -> None: + # These run on the free-the-runner path, where stdout can already be a dead pipe (a + # dropped ssh session). An unguarded print would raise BrokenPipeError out of + # hil_test's inner finally, skipping shutdown_pool AND the report writing. + try: + print(*args, **kwargs) + except (OSError, ValueError): + # ValueError, not just OSError: printing to a CLOSED stream raises + # "ValueError: I/O operation on closed file", and both hil_pool_check and + # hil_test redirect stdout into a StringIO that can be closed under us. Escaping + # here skips shutdown_pool/kill_pool_children/os._exit -- stranding the runner, + # the exact failure this wrapper exists to prevent. + pass + + +def _state(pid_dir: Path) -> str: + """The state letter from /proc/<pid>/stat. comm can contain ')', so the field is + located from the right rather than by splitting.""" + # bytes, not read_text(): read_text decodes with the LOCALE encoding, so under LANG=C + # (systemd services, self-hosted runners) a non-ASCII comm raises UnicodeDecodeError + # and the entry silently vanishes from the scan. + stat = (pid_dir / 'stat').read_bytes() + return chr(stat[stat.rindex(b')') + 2]) + + +def _pids(): + """/proc pid entries. Yields nothing rather than raising if /proc is unreadable.""" + try: + entries = list(PROC.iterdir()) + except OSError: + return + for entry in entries: + if entry.name.isdigit(): + yield entry + + +def d_state_note() -> str: + """Pids in uninterruptible sleep, for the report. Never aborts, never blocks. + + A D-state process at start-up is NOT a fault on its own -- a healthy in-flight testusb + looks exactly like this, and the rig supports a dev run alongside CI. It is a hint for + whoever reads a red cell below. Diagnosis proper is hil_pool_check and the + usb-kernel-recover skill; this is one line, not a probe.""" + stuck = [] + for d in PROC.glob('[0-9]*'): + try: + if _state(d) == 'D': + stuck.append(d.name) + except (OSError, ValueError, IndexError): + pass # raced with exit, or /proc is restricted: not our problem here + if not stuck: + return '' + return (f'{len(stuck)} process(es) in D state when this run started: ' + f'{sorted(stuck)[:10]}') + + +def shutdown_pool(pool, grace: float = 30) -> bool: + """terminate() a worker Pool without ever blocking forever. + + multiprocessing joins its workers unbounded (util.py _exit_function terminate()s the + daemonic ones, then calls p.join() -- no timeout -- on every remaining active child, + CPython 3.13.5), and a worker in uninterruptible sleep never + reaps -- so terminate() itself hangs, taking the runner's only job slot with it. False + when the pool refuses to die within `grace` (the caller must then abandon it); a + terminate() that *raises* counts as failure too, the pool being just as alive.""" + outcome = {} + + def _term(): + try: + pool.terminate() + outcome['ok'] = True + except BaseException as e: # noqa: BLE001 - any failure means the pool is still up + # Say what happened: Pool._terminate_pool really can raise (CPython: + # AssertionError 'Cannot have cache with result_handler not alive'), and a + # swallowed one is indistinguishable from an unkillable D-state worker. + outcome['err'] = e + _p(f'warning: Pool.terminate() raised {type(e).__name__}: {e}', flush=True) + + t = threading.Thread(target=_term, daemon=True) + t.start() + t.join(grace) + # Decide on the thread, not the dict: _term may set outcome['ok'] after join(grace) + # expired, reporting a merely-slow terminate as success on one read and abandoned on + # another. Still inside terminate() == not shut down. + if t.is_alive(): + return False + return outcome.get('ok', False) + + +def child_procs(pids) -> dict: + """{ancestor pid in `pids`: [(descendant pid, its pgid), ...]}, from ONE walk of /proc. + + DESCENDANTS, not direct children: a worker's usbtest.py spawns its recovery flasher + through run_cmd (own session), so it is a GRANDCHILD that a direct-child sweep misses + and a kill mid-recovery would orphan on the probe. pgid comes back too because the two + kinds of child need different signals (see kill_pool_children).""" + wanted = set(pids) + by_parent: dict = {} # ppid -> [(pid, pgid), ...] for EVERY process + for entry in _pids(): + try: + stat = (entry / 'stat').read_bytes() + except OSError: + continue # exited between the scan and the read, or not readable + # comm (field 2) is parenthesised and may contain spaces and ')' -- so split only + # what follows the LAST ')': state, ppid, pgrp, ... + try: + fields = stat[stat.rindex(b')') + 2:].split() + ppid, pgid = int(fields[1]), int(fields[2]) + except (ValueError, IndexError): + continue # truncated or unparsable stat line + by_parent.setdefault(ppid, []).append((int(entry.name), pgid)) + out: dict = {} + for root in wanted: + todo = list(by_parent.get(root, [])) + while todo: + pid, pgid = todo.pop() + out.setdefault(root, []).append((pid, pgid)) + todo += by_parent.get(pid, []) + return out + + +def _pool_procs(pool, extra) -> list: + """The pool's worker Process objects, plus each extra's own process. + + Manager() runs in its own child process and inherits the same descriptors as the + workers, so leaving it behind defeats the point: os._exit skips its finalizer.""" + procs = list(getattr(pool, '_pool', []) or []) + for e in extra: + procs.append(getattr(e, '_process', e)) + return procs + + + + +def kill_worker_children(pool, *extra) -> int: + """SIGKILL what the pool's workers spawned; returns how many SURVIVED. + + For the TIMEOUT path only. On the normal path each worker has already run + kill_own_children() and retired (maxtasksperchild=1), so this walks fresh idle workers + and finds nothing -- measured: 4 tasks, zero overlap with the pool at sweep time. + + Call it BEFORE shutdown_pool(): terminate() reaps the (interruptible) worker and its + flasher is reparented to init, erasing the ppid link this matches on. Signalling the + worker's own group instead cannot work -- a forked pool worker inherits OUR group + (CPython 3.13.5 multiprocessing never setsid/setpgid) and run_cmd gives every flasher + a session of its own. + + TWO passes because our SIGKILL can fail an in-flight flash and the worker then retries + in a fresh session, which one /proc snapshot misses. `seen` stops a pid signalled in + pass 1 being confirmed twice. + """ + seen: set = set() + total = 0 + for i in range(2): + if i: + time.sleep(0.5) + procs = _pool_procs(pool, extra) + total += _kill_kids( + child_procs(getattr(p, 'pid', None) for p in procs if p is not None), seen) + return total + + +def kill_own_children() -> int: + """SIGKILL what THIS process spawned. Returns how many survived. + + For the worker to call before it returns. maxtasksperchild=1 retires it the moment the + task ends, reparenting its children to init, so main()'s sweep walks fresh idle workers + and finds nothing (measured over 4 tasks: zero overlap, sweep 0, 4 strays alive). + Inside the worker the ppid link is still live. + """ + return _kill_kids(child_procs([os.getpid()]), set()) + + +def _kill_kids(kids: dict, seen: set) -> int: + """SIGKILL every pid in a ppid-tree snapshot; return how many survived. + + Every pid here is a DESCENDANT of a process we own, so it is ours by construction -- no + argv identity check, because we never signal anything we did not discover through our + own ppid tree. + """ + try: + own = os.getpgid(0) + except OSError: + own = None # cannot tell our own group apart: never killpg, signal pids only + # One list: every pid here is a DESCENDANT of one of our own workers, so it is ours by + # construction -- no argv identity check needed, because we never signal anything we + # did not discover through our own ppid tree. + touched: list = [] + for children in kids.values(): + for cpid, cpgid in children: + if cpid in seen: + continue # a previous pass already signalled it + seen.add(cpid) + try: + if own is not None and cpgid != own: + # A run_cmd child: its own session, so one killpg also reaps what it + # spawned. Recorded because killpg cannot report a partial kill. + os.killpg(cpgid, signal.SIGKILL) + else: + # Shares our group (a plain subprocess.run), so killpg would take + # down the whole run -- it is signalled by pid in _kill_and_confirm. + pass + touched.append(cpid) + except PermissionError: + # NOT "already gone": the signal did not land, so this pid MUST still be + # confirmed, or the one case this handler exists for (an all-root session: + # the sudo wrapper died, its root members did not) is the one case that + # never reaches the report. + touched.append(cpid) + except ProcessLookupError: + pass # already gone + except OSError: + pass + # Both paths need confirming: a killpg'd flasher and a same-group mtype blocked on a + # wedged device are both in D state, and os.kill reported success on either. + denied = _kill_and_confirm(touched) + if denied: + _p(f'warning: could not kill {sorted(denied)}; they still hold whatever they ' + f'had open (probe, usbfs node) into the next job', flush=True) + # SURVIVORS, not the signalled-child count: the caller needs to know the rig is dirty + # for the next job, and a count of what we successfully signalled cannot tell it that. + # (They are different units anyway -- a killpg is counted once per child sharing the + # group -- so the old return was never comparable to anything.) + return len(denied) + + +def _kill_and_confirm(pids) -> list: + """SIGKILL every pid, then return those STILL alive after ONE grace window. + + SIGKILL is QUEUED, not delivered, for a task in uninterruptible sleep -- and testusb + waits in a plain wait_for_completion() with no timeout (v6.12.96 usbtest.c:1404; + usb_sg_wait, message.c:765), so that is the normal state of a healthy in-flight case + too. os.kill returning success proves nothing; only the recheck does. It is also + asynchronous, so probing immediately reports a process we just killed as a survivor + (measured: 11 of 20 plain `sleep`s with no grace). + + Signal all, then poll the set against ONE shared deadline: per-pid windows made this + scale with stray count, minutes on a convoy. A pid we cannot signal is reported, never + sudo-killed. + """ + pending = [] + for pid in pids: + try: + os.kill(pid, signal.SIGKILL) + except ProcessLookupError: + continue # already gone + except OSError: + pass # EPERM (root-owned): it stays, and the poll below reports it + pending.append(pid) + + deadline = time.monotonic() + CONFIRM_KILL_GRACE + while True: + alive = [] + for pid in pending: + try: + os.kill(pid, 0) + except ProcessLookupError: + continue # ESRCH: genuinely gone + except OSError: + pass # EPERM: it exists; the state check decides + try: + # a ZOMBIE answers kill(pid, 0) too: dead, merely unreaped. Not a survivor. + if _state(PROC / str(pid)) == 'Z': + continue + except (OSError, ValueError, IndexError): + continue # unreadable: assume gone rather than cry wolf + alive.append(pid) + pending = alive + if not pending or time.monotonic() >= deadline: + return pending # outlasted SIGKILL: D state, or not ours to kill + time.sleep(0.02) + + +def kill_pool_children(pool, *extra) -> int: + """SIGKILL the pool's worker processes themselves. Returns how many are STILL ALIVE + after the grace -- not how many were signalled. + + Survivors, not signals: the caller turns this number into "power-cycle the host", so + counting signals would send someone to a hypervisor over workers that all died. + + Call after a shutdown_pool() that returned False, and after kill_worker_children(). + A D-state worker ignores SIGKILL, but every other worker dies and drops the inherited + descriptors -- a survivor holds the runner's stdout pipe open and the runner waits for + EOF even after we exit, so the early exit would not free the job slot.""" + killed_procs: list = [] + for proc in _pool_procs(pool, extra): + try: + # Process.kill(), never a raw pid: multiprocessing's _send_signal re-checks + # `self.returncode is None` first, so once shutdown_pool's thread has reaped a + # worker this is a no-op instead of signalling a pid the OS may have recycled. + # os.pidfd_open(proc.pid) is worse: it skips that guard entirely. + if proc is None or not proc.is_alive(): + continue + proc.kill() + killed_procs.append(proc) + except (OSError, AttributeError, ValueError): + continue # already reaped, never started, or not a real process + # Re-check the Process objects, never the pids collected a moment ago: shutdown_pool's + # thread is STILL join()ing workers, so a pid killed here can be reaped and RECYCLED + # before _kill_and_confirm signals it -- and on EPERM that escalates to `sudo -n kill + # -9 <stale pid>`, killing an unrelated ROOT process as the last act before os._exit. + killed_pids = [] + for proc in killed_procs: + try: + if proc.is_alive() and proc.pid is not None: + killed_pids.append(proc.pid) + except (OSError, AttributeError, ValueError): + continue + # SIGKILL is asynchronous and a D-state task ignores it: only a confirmed survivor + # justifies the caller's power-cycle wording + return len(_kill_and_confirm(killed_pids)) if killed_pids else 0 + + +def write_timeout_report(report_dir: Path, boards, secs: int, md_name: str, + banner: str = '', prefix: str = '') -> None: + """Leave a report behind when the worker pool has to be abandoned. + + map_async is all-or-nothing, so a timeout loses every per-board result and the report + dir would stay empty with no reason for the failure. Any prior attempt's markdown is + kept below the banner.""" + # `prefix` carries the preflight rig-health verdict: the timeout aborts before + # accumulate_report, so without it the report loses the one line saying WHY the pool + # never finished. The '\n' stops Markdown lazy continuation pulling the banner into + # the blockquote. + try: + # Built INSIDE the try: a roster entry without a 'name' key raises KeyError while + # assembling the board list, and outside the try that escaped and stranded the + # runner -- which is exactly what the broad handler below exists to prevent. + head = (prefix + '\n' if prefix else '') + (banner or ( + f'**HIL run abandoned: worker pool timed out after {secs}s.**\n\n' + f'No per-board results could be collected for this attempt, so the ' + f'table below (if any) is from an earlier one. Boards dispatched:\n\n' + + '\n'.join(f'- {b.get("name", "?")}' for b in boards) + '\n')) + report_dir.mkdir(parents=True, exist_ok=True) + md_path = report_dir / md_name + # Its own handler so it cannot take the write down with it: a report torn by an + # attempt killed mid-write raises UnicodeDecodeError (a ValueError, and prior + # reports always contain status emoji), which under a shared try skipped the write + # entirely. Losing the old table is a nicety; losing the banner is the failure. + try: + prior = md_path.read_text(encoding='utf-8') if md_path.is_file() else '' + except (OSError, ValueError): + prior = '' + md_path.write_text(head + (f'\n{prior}' if prior else ''), encoding='utf-8') + except Exception as e: # noqa: BLE001 + # Deliberately broad: this is the first statement of the pool-abandon path, so ANY + # escape skips kill_pool_children and os._exit and strands the runner. + _p(f'warning: cannot write {md_name} to {report_dir}: {e}', flush=True) diff --git a/test/hil/helper/hil_lock.py b/test/hil/helper/hil_lock.py new file mode 100755 index 000000000..7757ef17d --- /dev/null +++ b/test/hil/helper/hil_lock.py @@ -0,0 +1,525 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +"""Board locks + controller permits for the TinyUSB HIL rig. + +Board locks are kernel flocks in BOARD_LOCK_DIR arbitrating hardware access +between dev sessions and CI's hil_test.py (never stop the actions-runner). +Controller permits are in-process semaphores budgeting flashes and usbtest +batteries per host controller; they have no CLI meaning. The CLI below +(hold/release/status) manages board locks only. +""" +import argparse +import fcntl +import json +import os +import re +import select +import signal +import sys +import time + +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # helper/ scripts import via the test/hil root +from helper import hil_util + +BOARD_LOCK_DIR = '/tmp/tinyusb-hil-locks' +CI_REASON = 'hil_test.py' # release-protected holder tag (release refuses to kill it) +PROTECTED_REASONS = {CI_REASON, 'pool_check'} # cmd_release refuses to SIGTERM these holders +PROFILE = os.environ.get('HIL_PROFILE') == '1' + + +def lock_path(board: str) -> str: + return os.path.join(BOARD_LOCK_DIR, f'{board}.lock') + + +def flock_nb(board: str): + """Open-or-create the lock file WITHOUT truncating (a losing racer must not + wipe the winner's record) and take LOCK_EX|LOCK_NB. Returns the open handle; + raises OSError when the flock is held elsewhere (handle already closed).""" + fd = os.open(lock_path(board), os.O_RDWR | os.O_CREAT, 0o666) + fh = os.fdopen(fd, 'r+') + try: + fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB) + except OSError: + fh.close() + raise + return fh + + +def write_record(fh, reason: str) -> bool: + """Holder record; the flock itself is already held. Returns False on a write failure: + acquire_board_lock stays best-effort (the flock is the authority), but cmd_hold aborts + -- a hold whose record is missing is invisible to status/release.""" + try: + fh.truncate(0) + fh.seek(0) + json.dump({'pid': os.getpid(), 'reason': reason, + 'since': time.strftime('%Y-%m-%dT%H:%M:%S%z')}, fh) + fh.flush() + return True + except OSError: + return False + + +def clear_record(fh) -> None: + """Clear our record before dropping the flock so records stay truthful.""" + try: + fh.truncate(0) + except OSError: + pass + + +def read_record(board: str): + try: + with open(lock_path(board)) as f: + return json.load(f) + except (OSError, ValueError): + return None + + +# --- per-board dev-session locks ------------------------------------------ +def acquire_board_lock(board_name, reason=CI_REASON): + """Take this board's flock for the duration of its flash+test. + Returns an open file handle (keep it referenced; closing releases it), + or None when HIL_NO_BOARD_LOCK=1 or the lock dir is unusable (fail-open: + locking must never break a test run by itself). + Raises RuntimeError only when another session holds the board.""" + import fcntl + if os.environ.get('HIL_NO_BOARD_LOCK') == '1': + return None # user-authorized bypass — see hil skill + try: + os.makedirs(BOARD_LOCK_DIR, exist_ok=True) + fd = os.open(os.path.join(BOARD_LOCK_DIR, f'{board_name}.lock'), + os.O_RDWR | os.O_CREAT, 0o666) + fh = os.fdopen(fd, 'r+') + except OSError as e: + # odd lock dir (perms, path collision): proceed unlocked, but say so — + # a silent fail-open is indistinguishable from the intentional bypass + print(f'warning: board lock unavailable for {board_name} ({e}); proceeding unlocked', + flush=True) + return None + try: + fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB) + except OSError: + try: + info = fh.read(500).strip() + except (OSError, UnicodeDecodeError): + info = '' + fh.close() + raise RuntimeError(f'board locked: {info or "unknown holder"}') + # announce ourselves so the other side's conflict message is truthful; + # best-effort — the flock itself is already held + try: + fh.truncate(0) + fh.seek(0) + json.dump({'pid': os.getpid(), 'reason': reason, + 'since': time.strftime('%Y-%m-%dT%H:%M:%S%z')}, fh) + fh.flush() + except OSError: + pass + return fh + + +# Per-host-controller concurrency (see controller_of/controller_slot below): a usbtest +# battery saturates its DUT's host controller, so batteries and flashes are budgeted per +# controller. The 4/2 defaults trade ~3.5 min on the usbtest leg for bandwidth margin on +# the shared leaf-hub uplinks, where battery case failures were observed from 12/8 +# (profiled 2026-07-13/14: 22.2/14.3/12.5/10.8 min at usbtest width 1/2/3/4, plateau +# after). Raise per run via HIL_FLASH_PARALLEL/HIL_USBTEST_PARALLEL. +# - uPD720201 cards need firmware >= 2.0.2.6 (RAM-uploaded, reloads every power cycle): +# the ROM firmware dies under battery + re-enumeration churn. +# - a marginal DUT port bouncing during concurrent batteries can kill a uPD720201 ("xHCI +# host not responding to stop endpoint command"): fix the port/cable or pull the board +# -- lowering the widths does not fix a bad port (2026-07-16, every death). +FLASH_PARALLEL = hil_util.pos_int_env('HIL_FLASH_PARALLEL', 4) +USBTEST_PARALLEL = hil_util.pos_int_env('HIL_USBTEST_PARALLEL', 2) +CONTROLLER_SLOTS = 12 # lock slots; controllers are assigned to slots on first sight +# Bound on ONE permit wait. Generous: a real queue behind a slow board is normal, +# and this only has to beat the pool guard so a leaked permit cannot consume it. +PERMIT_TIMEOUT = hil_util.pos_int_env('HIL_PERMIT_TIMEOUT', 900) +# CONTROLLER_SLOTS + 1 entries each, built by make_permit_sems: UNKNOWN_SLOT indexes the +# extra one. Sized to CONTROLLER_SLOTS instead, the first unresolved board IndexErrors +# inside a pool worker -- which now surfaces through drain_pool as a worker-raise (the +# finished boards survive), but still loses this board and aborts the run. +usbtest_sems = None # per-slot usbtest-battery permits +flash_sems = None # per-slot flash permits +controller_map = None # shared dict: 'pci:<addr>' -> slot, 'uid:<uid>' -> pci addr cache +controller_meta = None # guards slot assignment in controller_map +controller_hints = {} # static uid -> pci from the last run's cache (read-only per worker) + + +log = print # hil_test.init_worker points this at log_line via init_scheduling + + +def init_scheduling(b_sems, f_sems, cmap, cmeta, hints, log_fn=None): + """Install per-worker scheduling state (called from hil_test.init_worker).""" + global usbtest_sems, flash_sems, controller_map, controller_meta, controller_hints, log + usbtest_sems, flash_sems = b_sems, f_sems + controller_map, controller_meta, controller_hints = cmap, cmeta, hints + if log_fn is not None: + log = log_fn + + +# ------------------------------------------------------------- +# Per-controller scheduling +# ------------------------------------------------------------- +def controller_of(uid: str): + """Resolve a DUT uid to its root host controller's PCI address, or None when it cannot + be resolved — the device is not enumerated (e.g. parked in board_test firmware with USB + off), or sysfs would not answer. Successful resolutions are cached — cabling does not + change mid-run. Dual-port parts (e.g. CH32V307 usbhs/usbfs variants) share one uid and + one cache entry: budgeting is only exact when both ports sit on the same controller + (true on this rig).""" + if controller_map is None: + return None + cached = controller_map.get(f'uid:{uid}') + if cached: + return cached + # vid='cafe' first: the target is always a TinyUSB DUT, and the VID is a lock-free + # descriptor field. Without it this read every probe's and hub's `serial` -- the + # attribute served under device_lock -- so a HEALTHY peer mid-usbtest would strand a + # reader here and spend one of this worker's four blindness credits. + devs, _ = hil_util.usb_scan(vid='cafe', serial=uid) + for dev in devs: + busnum = hil_util.read_sysfs(os.path.join(dev['dir'], 'busnum')) + if busnum is None or busnum is hil_util.SYSFS_UNKNOWN: + continue + try: + root = os.path.realpath(f'/sys/bus/usb/devices/usb{int(busnum)}') + except ValueError: + continue + m = re.findall(r'[0-9a-f]{4}:[0-9a-f]{2}:[0-9a-f]{2}\.[0-9a-f]', root) + if m: + controller_map[f'uid:{uid}'] = m[-1] + return m[-1] + return None + + +def controller_slot(pci: str) -> int: + """Map a controller PCI address to a lock slot (assigned on first sight).""" + key = f'pci:{pci}' + with controller_meta: + slot = controller_map.get(key) + if slot is None: + slot = controller_map.get('nslots', 0) + if slot >= CONTROLLER_SLOTS: + slot = 0 # more controllers than slots: overflow shares slot 0 (safe, over-serialized) + else: + controller_map['nslots'] = slot + 1 + controller_map[key] = slot + return slot + + +# Unresolved boards budget in a slot of their OWN, one past the real ones, and that slot +# holds exactly ONE permit whatever the per-controller width is. Neither neighbour works: +# a permit on every slot (the old fail-closed rule) serialized the whole fleet the moment +# a worker went blind, while a full private budget let unknown boards run a second +# controller's worth of batteries on top of the resolved ones -- doubling the load on +# whichever physical controller they actually sit on, which is the saturation the +# uPD720201 deaths above are attributed to. Width 1 caps the over-subscription at +1. +UNKNOWN_SLOT = CONTROLLER_SLOTS + + +def make_permit_sems(semaphore, width: int) -> list: + """One semaphore per controller slot at `width`, plus the unknown bucket at 1.""" + return [semaphore(width) for _ in range(CONTROLLER_SLOTS)] + [semaphore(1)] + + +class controller_permit: + """Context manager: one permit from `sems` on the board's controller slot. An + unresolved controller budgets in UNKNOWN_SLOT, which admits one at a time: unresolved + boards serialize against each other, never against the whole rig, and never add a + second full budget to a controller. `warn_unknown` logs that fallback (used by + usbtest, where the device is expected to be enumerated by the caller).""" + def __init__(self, sems, uid: str, warn_unknown: bool = False): + self.sems = sems + self.slots = None + self.uid = uid + # what __enter__ actually ACQUIRED. Not the same as self.slots: a bounded acquire + # that times out is skipped on purpose, and releasing it anyway would add a permit + # that was never taken -- multiprocessing semaphores are unbounded, so the width + # grows for the rest of the run, on the controller throttle that exists to keep + # concurrent batteries from killing the uPD720201 xHCI. + self.taken: list = [] + if sems is None: + return + # Hint FIRST for flash budgeting: a mis-budgeted flash is harmless, and the board + # is usually parked in board_test with USB off at this point, so controller_of + # cannot resolve it anyway -- it just walks the whole bus to say so, once per + # flash permit (~14 examples x ~21 boards a leg), each walk spawning a bounded + # reader per device. usbtest still resolves for real (warn_unknown), and by then + # the DUT is enumerated, so that walk succeeds and caches. + pci = None if warn_unknown else controller_hints.get(uid) + if pci is None: + pci = controller_of(uid) + if pci is None and warn_unknown: + log(f'warning: cannot resolve {uid} to a host controller' + f'{hil_util.sysfs_blind_note()}; budgeting it in the unknown bucket') + self.slots = [controller_slot(pci) if pci else UNKNOWN_SLOT] + + def __enter__(self): + if self.slots: + t0 = time.monotonic() + taken = self.taken = [] + try: + for s in self.slots: + # BOUNDED. multiprocessing semaphores are NOT released when a holder + # dies, and the pool sweep SIGKILLs workers -- so a permit lost that + # way would block every later worker on this controller forever, and + # boards unrelated to the wedge would burn the whole pool guard. On + # expiry proceed over-subscribed and say so: a slower controller is a + # far better failure than a hung run. + if not self.sems[s].acquire(timeout=PERMIT_TIMEOUT): + log(f'warning: waited {PERMIT_TIMEOUT}s for a permit on slot {s} ' + f'(uid {self.uid}); a holder probably died without releasing ' + f'it -- proceeding over-subscribed') + continue + taken.append(s) + # inside the try: a failed __enter__ never gets its __exit__, so a raise + # here (e.g. broken stdout) must still release the permits + if PROFILE and time.monotonic() - t0 > 1.0: + log(f'[prof] permit wait {time.monotonic() - t0:.1f}s ' + f'(uid {self.uid}, slots {self.slots})') + except BaseException: + for s in reversed(taken): + self.sems[s].release() + raise + return self + + def __exit__(self, *exc): + if self.slots: + for s in reversed(self.taken): + self.sems[s].release() + self.taken = [] + return False + + +def flash_permit(uid: str) -> controller_permit: + return controller_permit(flash_sems, uid) + + +def usbtest_permit(uid: str) -> controller_permit: + return controller_permit(usbtest_sems, uid, warn_unknown=True) + + +# --- operator CLI (hold/release/status) ------------------------------------ +def boards_from_config(config: str) -> list: + """All board names, INCLUDING boards-skip: `hold --all` guards rig-wide + operations, and parked boards can still be touched (pool_check -b names them + explicitly), so a rig-wide hold that skipped them would leave a gap.""" + try: + with open(config) as f: + cfg = json.load(f) + return [b['name'] for b in cfg['boards'] + cfg.get('boards-skip', [])] + except (OSError, ValueError, KeyError) as e: + print(f'ERROR: cannot read board roster {config}: {e}', file=sys.stderr) + sys.exit(1) + + +def is_locked(board: str) -> bool: + """True if the recorded holder process is still alive. + + Deliberately never touches the flock: even a momentary probe lock would + make a concurrent acquirer's LOCK_NB attempt fail spuriously. The flock + taken by acquirers themselves stays the only authority.""" + info = read_record(board) + pid = info.get('pid') if isinstance(info, dict) else None + if not isinstance(pid, int) or pid <= 0: + return False + try: + os.kill(pid, 0) + except ProcessLookupError: + return False + except PermissionError: + return True # alive but owned by another user (e.g. the CI runner) + return True + + +def cmd_hold(boards, reason): + os.makedirs(BOARD_LOCK_DIR, exist_ok=True) + # No pre-check: the holder's own LOCK_NB flock is the only authority, since a recorded + # pid may be stale or recycled. The holder signals success through this pipe because a + # generic is_locked() poll would be fooled by a RIVAL invocation's flock — only the + # holder knows whether it won every board. + r_fd, w_fd = os.pipe() + pid = os.fork() + if pid > 0: + os.close(w_fd) + os.waitpid(pid, 0) # reap intermediate child + ready, _, _ = select.select([r_fd], [], [], 10) + ok = bool(ready) and os.read(r_fd, 1) == b'1' + os.close(r_fd) + if ok: + print(f'held: {", ".join(boards)}') + return 0 + for b in boards: + info = read_record(b) + if info: + print(f'ERROR: {b} locked: {info}', file=sys.stderr) + print('ERROR: holder failed to acquire locks', file=sys.stderr) + return 1 + # intermediate child: detach, then spawn the actual holder + os.setsid() + if os.fork() > 0: + os._exit(0) + # holder (grandchild): acquire all flocks, signal the parent, sleep until killed + os.close(r_fd) + # Keep the success pipe clear of fds 0-2: invoked with stdio closed, os.pipe() can + # land there and the dup2 loop below would clobber it. + if w_fd <= 2: + w_fd = fcntl.fcntl(w_fd, fcntl.F_DUPFD, 3) + # Detach stdio: a `hold` whose output is captured must see EOF when the front-end + # exits — the immortal holder must not keep that pipe open. + devnull = os.open(os.devnull, os.O_RDWR) + for std_fd in (0, 1, 2): + os.dup2(devnull, std_fd) + if devnull > 2: + os.close(devnull) + try: + handles = [] + for b in boards: + fh = flock_nb(b) + if not write_record(fh, reason): + raise OSError(f'cannot write holder record for {b}') + handles.append(fh) + except OSError: + try: + os.write(w_fd, b'0') + except OSError: + pass + os._exit(1) # lost a race; parent reports the failure + os.write(w_fd, b'1') + os.close(w_fd) + + def _bow_out(*_): + # clear the records before dying so read_record/status stay truthful (the kernel + # drops the flocks themselves on exit either way) + for h in handles: + clear_record(h) + os._exit(0) + + signal.signal(signal.SIGTERM, _bow_out) + while True: + signal.pause() + + +def cmd_release(boards): + rc = 0 + victims = set() + for b in boards: + try: + fd = os.open(lock_path(b), os.O_RDWR) + except OSError: + continue # no lock file (or another user's): nothing we can release + fh = os.fdopen(fd, 'r+') + try: + fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB) + except OSError: + # flock genuinely held — never SIGTERM on a mere pid record: the pid may be + # recycled, or a live worker that already moved on. + fh.close() + info = read_record(b) or {} + pid = info.get('pid') + reason = info.get('reason') + if reason in PROTECTED_REASONS: + print(f'ERROR: {b} is mid-test by {reason} (pid {pid}) — not killing it; ' + 'wait for it to finish', file=sys.stderr) + rc = 1 + elif isinstance(pid, int) and pid > 0: + victims.add(pid) + else: + print(f'ERROR: {b} is held but its record is unreadable', file=sys.stderr) + rc = 1 + continue + # flock was free: only a stale record remained — clear it + clear_record(fh) + fh.close() + for holder in sorted(victims): + try: + os.kill(holder, signal.SIGTERM) + print(f'released holder pid {holder}') + except ProcessLookupError: + pass + except PermissionError: + print(f'ERROR: holder pid {holder} belongs to another user — cannot signal it', + file=sys.stderr) + rc = 1 + time.sleep(0.3) + still = [b for b in boards if is_locked(b)] + if still: + print(f'ERROR: still locked: {", ".join(still)}', file=sys.stderr) + return 1 + return rc + + +def cmd_status(): + if not os.path.isdir(BOARD_LOCK_DIR): + print('no locks') + return 0 + any_locked = False + for fn in sorted(os.listdir(BOARD_LOCK_DIR)): + if not fn.endswith('.lock'): + continue + b = fn[:-5] + if is_locked(b): + any_locked = True + print(f'{b}: {read_record(b)}') + if not any_locked: + print('no locks') + return 0 + + +_CLI_USAGE = """Per-board advisory locks for the HIL rig. + +Arbitrates board access between dev sessions and CI's hil_test.py without +stopping the actions-runner. Locks are kernel flocks: the kernel releases +them automatically when the holder process dies, and holders clear their +lock-file record on release so records stay truthful (/tmp also clears on +reboot). + +Usage: + hil_lock.py hold BOARD [BOARD...] --reason TEXT + hil_lock.py hold --all [--config CONFIG.json] --reason TEXT + hil_lock.py release BOARD [BOARD...] | release --all + hil_lock.py status + +A holder process holds ALL boards given in one `hold` call; releasing any of +them kills that holder and releases all of its boards. +""" + + +def main(): + ap = argparse.ArgumentParser(description=_CLI_USAGE, + formatter_class=argparse.RawDescriptionHelpFormatter) + sub = ap.add_subparsers(dest='cmd', required=True) + p_hold = sub.add_parser('hold') + p_hold.add_argument('boards', nargs='*') + p_hold.add_argument('--all', action='store_true') + p_hold.add_argument('--config', + default=os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), + 'tinyusb.json'), + help='board roster JSON (default: tinyusb.json in test/hil, one level above this script)') + p_hold.add_argument('--reason', required=True) + p_rel = sub.add_parser('release') + p_rel.add_argument('boards', nargs='*') + p_rel.add_argument('--all', action='store_true') + sub.add_parser('status') + a = ap.parse_args() + if a.cmd == 'hold': + boards = boards_from_config(a.config) if a.all else a.boards + if not boards: + ap.error('no boards given (name boards or use --all)') + sys.exit(cmd_hold(boards, a.reason)) + if a.cmd == 'release': + if a.all: + boards = ([fn[:-5] for fn in os.listdir(BOARD_LOCK_DIR) if fn.endswith('.lock')] + if os.path.isdir(BOARD_LOCK_DIR) else []) + else: + boards = a.boards + if not boards: + ap.error('no boards given (name boards or use --all)') + sys.exit(cmd_release(boards)) + sys.exit(cmd_status()) + + +if __name__ == '__main__': + main() diff --git a/test/hil/helper/hil_pool_check.py b/test/hil/helper/hil_pool_check.py new file mode 100644 index 000000000..371aff1e1 --- /dev/null +++ b/test/hil/helper/hil_pool_check.py @@ -0,0 +1,1019 @@ +#!/usr/bin/env python3 +"""Quick HIL pool health check. + +For every board in the rig's HIL config: is the flash probe on the USB bus, does a +light example flash, and does the board's USB device (uid) come back up? Missing +firmware is BUILT on the spot (tools/build.py, idf.py for espressif; one get_deps +retry) — never skipped; --no-build opts out. Applies only per-device-safe recovery +(probe authorized-toggle, board reset/re-flash) and prints a markdown summary +table. Row statuses: ok (flashed and verified; under --scan-only: probe present — +the scan checks presence only), flash-failed (firmware delivery failed: probe +missing, build failed, flasher error, silent flash no-op, park not verified), +failed (the check ran but did not verify: flashed with no enumeration/serial, or +the check itself errored), locked (board flock held by another process — +reported, never waited on or bypassed). + +Config is picked by hostname unless given: ci -> tinyusb.json, tusb (hifiphile +rig) -> hfp.json, anything else is a dev PC -> local.json. + +Lives in test/hil/helper/ beside hil_lock.py; imports it and hil_flash; board +recovery uses the repo's .claude/skills/usb-kernel-recover/scripts/usb_recover.sh. +""" + +import argparse +import io +import json +import glob +import os +import re +import shlex +import shutil +import socket +import sys +import threading +import time +from concurrent.futures import ThreadPoolExecutor +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) # hil_flash + the helper package +import hil_flash +from helper import hil_lock, hil_util + +REPO_ROOT = hil_util.TINYUSB_ROOT +USB_RECOVER = REPO_ROOT / '.claude' / 'skills' / 'usb-kernel-recover' / 'scripts' / 'usb_recover.sh' +SEEN_CACHE = Path.home() / '.cache' / 'tinyusb-hil' / 'pool_seen.json' +CONFIG_BY_HOST = {'ci': 'tinyusb.json', 'tusb': 'hfp.json'} # anything else: dev PC -> local.json + +# light-example preference; first built wins +DEVICE_CANDIDATES = ['device/dfu_runtime', 'device/cdc_msc', 'device/cdc_msc_freertos', + 'device/hid_composite_freertos', 'device/cdc_dual_ports'] +HOST_CANDIDATES = ['host/device_info', 'host/cdc_msc_hid', 'host/msc_file_explorer_freertos'] + +ENUM_WAIT = 12 # s, uid wait after flash +ENUM_WAIT_RETRY = 8 # s, uid wait after a recovery reset/re-flash +SERIAL_WAIT = 6 # s, host-board serial-output wait + +print_mutex = threading.Lock() +_UNKNOWN_WARNED = False # scan_usb's caveat: once per process, not once per poll +t0 = time.monotonic() + + +def say(msg: str) -> None: + with print_mutex: + print(f'[{time.monotonic() - t0:6.1f}s] {msg}', file=sys.__stdout__, flush=True) + + +def scan_usb() -> dict: + """busport -> {'serial', 'vidpid', 'ino'} for every enumerated USB device. Only + <bus>-<port>[.<port>...] dirs match; root hubs ('usbN', no dash) are excluded because + their 'serial' is a fabricated PCI address, and including them measured 6-7s/scan slower + (an observation; NOT an autosuspend wake -- that read is cached and does no I/O). + Keyed by busport, not serial: two devices can share a serial (an Espressif + USB-Serial-JTAG bridge and the cafe device it flashes both derive it from the same + MAC), and one dict slot would silently drop whichever lost the race.""" + found = {} + # `unknown` matters BEFORE the blindness latch trips: one wedged device is the normal + # reason this tool is run, and its serial read stranding makes it absent from `devs`. + # Reported as fact, that is "probe MISSING" for hardware that is physically present. + devs, unknown = hil_util.usb_scan() + # ONCE per process: this is called from 0.5s poll loops across 4 worker threads and + # ~26 boards, so warning per call buried the table it exists to qualify under 600+ + # identical lines. The memo in read_sysfs makes the condition sticky, so one line is + # as true as six hundred. + global _UNKNOWN_WARNED + if unknown and not _UNKNOWN_WARNED: + _UNKNOWN_WARNED = True + say('WARNING: at least one device did not answer a bounded read; rows below that ' + 'say a probe or board is missing may be this scan losing sight of healthy ' + 'hardware. Find the wedged device (usb-kernel-recover) and re-run.') + for dev in devs: + try: + found[dev['busport']] = { + 'serial': dev['serial'].lower(), + 'vidpid': f"{dev['vid']}:{dev['pid']}", + 'ino': os.stat(dev['dir'] + '/').st_ino} + except OSError: + continue + return found + + +def find_usb(uid: str, devs: dict | None = None): + """Locate a flasher probe by uid, excluding VID cafe (TinyUSB DUT firmware): a + probe's uid can coincidentally equal its DUT's (Espressif USB-Serial-JTAG + bridges derive both from the same MAC), and the DUT is never the probe. + + J-Link zero-pads numeric serials (681295394 -> 000681295394): an all-digit uid + matches an all-digit serial only when that serial equals the uid zero-padded to + the serial's own length (leading zeros only) — never when the zero-stripped uid + is empty, so a placeholder serial (metro_m4_express's probe legitimately reports + '123456') can't be mistaken for an unrelated device.""" + devs = devs if devs is not None else scan_usb() + u = uid.lower() + candidates = [(bp, dev) for bp, dev in devs.items() if not dev['vidpid'].startswith('cafe:')] + for bp, dev in candidates: + if dev['serial'] == u: + return bp, dev['vidpid'], dev['ino'] + stripped = u.lstrip('0') + if u.isdigit() and stripped: + for bp, dev in candidates: + s = dev['serial'] + if s.isdigit() and s == stripped.zfill(len(s)): + return bp, dev['vidpid'], dev['ino'] + return None + + +def find_device(uid: str, pid: str | None): + """Board-online check: TinyUSB device (idVendor cafe) with this uid, optionally + PID-pinned. VID cafe keeps an Espressif USB-Serial-JTAG (303a) that shares the MAC + serial from false-passing.""" + for busport, dev in scan_usb().items(): + if (dev['serial'] == uid.lower() and dev['vidpid'].startswith('cafe:') + and (pid is None or dev['vidpid'].endswith(pid))): + return busport, dev['vidpid'], dev['ino'] + return None + + +def wait_device(uid: str, pid: str | None, old_ino, budget: float): + """Wait for the board's device with a NEW sysfs inode (flash resets the MCU, so a + genuine flash must re-enumerate; the inode is the re-enumeration marker).""" + deadline = time.monotonic() + budget + while time.monotonic() < deadline: + hit = find_device(uid, pid) + if hit and hit[2] != old_ino: + return hit + time.sleep(0.5) + return None + + +def lock_board(name: str): + """Nonblocking flock per hil_lock.py protocol. Returns the handle, or a str with the + holder's info when the board is locked elsewhere. Board locks are ALWAYS respected: a + held board is reported and skipped, never waited on, and there is no bypass here.""" + os.makedirs(hil_lock.BOARD_LOCK_DIR, exist_ok=True) + try: + fh = hil_lock.flock_nb(name) + except OSError: + # NB: conflates a held flock with open() failures (EACCES/EROFS/ENOSPC) — benign + # while everything on the rig runs as one uid + info = hil_lock.read_record(name) + return json.dumps(info) if info else 'unknown holder' + if not hil_lock.write_record(fh, 'pool_check'): + # an invisible lock (flock held, no record) is worse than no lock: status cannot + # show us and release cannot recognize the protected holder + hil_lock.clear_record(fh) + fh.close() + return 'ERROR: holder record write failed (lock dir unwritable?)' + return fh + + +def unlock_board(fh) -> None: + hil_lock.clear_record(fh) + fh.close() + + +def can_recover() -> bool: + if not USB_RECOVER.is_file(): + return False + try: + # run_cmd, not subprocess.run: run's post-timeout reap is an UNBOUNDED wait(), and + # our kill bounces off a setuid-root sudo with EPERM, leaving communicate() on a + # pipe that never closes. run_cmd killpgs, escalates through sudo, reaps bounded. + r = hil_util.run_cmd('sudo -n true', timeout=10, quiet=True) + except OSError: # sudo not installed + return False + return r.returncode == 0 + + +def recover_probe(uid: str, busport: str) -> bool: + """Soft-replug an enumerated-but-wedged probe: deauthorize+reauthorize (no VBUS cut, + touches only this device). Success = the probe re-enumerated (new sysfs inode), not the + helper's exit code, which flakes while the toggle works. J-Links respond with a full + disconnect and can stay off the bus for >8 s.""" + pre = find_usb(uid) + # Bounded through run_cmd (same reason as can_recover): the sysfs authorized store can + # block in D state on a wedged device, and this runs while the board's release- + # PROTECTED flock is held -- a hang here would lock the board until the host reboots. + cmd = ' '.join(shlex.quote(a) for a in + ['sudo', '-n', str(USB_RECOVER), 'authorized', busport]) + if hil_util.run_cmd(cmd, timeout=30, quiet=True).returncode == 124: + return False + deadline = time.monotonic() + 20 + while time.monotonic() < deadline: + post = find_usb(uid) + if post and (pre is None or post[2] != pre[2]): + return True + time.sleep(0.5) + return False + + +def resolve_variant(board: dict, example: str, note: list | None = None) -> str: + """Build-dir variant name for `example`: the first of the board's variants with + already-built firmware, falling back to the board name. Notes the pick when it + differs from the board name (e.g. nanoch32v203's build dir is variant + 'nanoch32v203-fsdev', not the board name).""" + name = board['name'] + for v in board.get('variant') or [{'name': name}]: + vn = v['name'] + if hil_flash.find_firmware(vn, example, flasher=board['flasher']['name']): + if vn != name and note is not None and f'variant: {vn}' not in note: + note.append(f'variant: {vn}') + return vn + return name + + +def pick_example(board: dict, note: list, build_missing: bool = True): + """(example, kind, variant, fw) with built firmware for this board; kind is + 'device' (uid check) or 'host' (serial-output check); variant is the resolved + build-dir variant that has it (see resolve_variant); fw is the firmware path to + flash, extension included. When nothing is built and build_missing is set (the default — + never skip a board for lack of a build), the preferred candidate is built on + the spot via ensure_fw.""" + tests = board.get('tests', {}) + only = tests.get('only', []) + skip = set(tests.get('skip', [])) # config's known-broken examples: never pick one + is_device = tests.get('device') or any(t.startswith('device/') for t in only) + if is_device: + cand = DEVICE_CANDIDATES + [t for t in only if t.startswith('device/') and t != 'device/usbtest'] + kind = 'device' + else: + cand = HOST_CANDIDATES + [t for t in only if t.startswith('host/')] + kind = 'host' + for ex in dict.fromkeys(cand): + if ex in skip: + continue + variant = resolve_variant(board, ex, note) + fw = hil_flash.find_firmware(variant, ex, flasher=board['flasher']['name']) + if fw: + return ex, kind, variant, fw + if not build_missing: + return None, kind, None, None + # nothing built anywhere: build the preferred candidate (an only-list board + # must get one of its own examples — dfu_runtime etc. may not even configure) + pref = [c for c in dict.fromkeys(cand) if c not in skip and (not only or c in only)] + if not pref: + return None, kind, None, None + variant = (board.get('variant') or [{'name': board['name']}])[0]['name'] + for ex in pref[:2]: # the second candidate covers a preferred example that fails to build + fw = ensure_fw(board, variant, ex, note) + if fw: + return ex, kind, variant, fw + return None, kind, None, None + + +_pid_cache: dict[str, str | None] = {} + + +def get_expected_pid(example: str) -> str | None: + """USB_PID for `example`'s device descriptor (examples/<example>/src/ + usb_descriptors.c, '#define USB_PID 0x....'), lowercased and without the 0x + prefix to match sysfs idProduct. Cached per example; None (also cached) when + the file or define isn't there — host examples have no usb_descriptors.c, and + the caller must stay quiet rather than false-warn.""" + if example not in _pid_cache: + pid = None + try: + text = (REPO_ROOT / 'examples' / example / 'src' / 'usb_descriptors.c').read_text() + # optional parens as in tools/check_example_pids.py's parser + m = re.search(r'#define\s+USB_PID\s+\(?\s*(0x[0-9a-fA-F]+)', text) + if m: + pid = m.group(1)[2:].lower() + except OSError: + pass + _pid_cache[example] = pid + return _pid_cache[example] + + +def call_flasher(fn, *fn_args) -> tuple[int, str]: + """Run a hil_flash flash_*/reset_* backend, normalizing raises to a failure: several + backends raise instead of returning nonzero (get_serial_dev when a bridge's + /dev/serial/by-id node vanishes, a missing config.env, a .jlink script OSError), and an + exception must not skip the caller's retry/recovery ladder. Returns (rc, error line).""" + try: + ret = fn(*fn_args) + if ret.returncode == 0: + return 0, '' + err = flash_error_line(hil_util.cmd_stdout_text(ret.stdout)) + return ret.returncode, err or f'rc={ret.returncode}' + except Exception as e: + return -1, repr(e)[:90] + + +def flash(board: dict, fw, allow_recovery: bool, probe_port: str, note: list) -> bool: + """Flash the resolved firmware with one retry; on repeated failure soft-replug the + probe and always make one final attempt afterward, confirmed replug or not — some + probes (WCH-Link, ST-Link, CP210x, picoprobe) keep their sysfs kobject across an + authorized toggle instead of dropping off the bus. Returns True on success. + + `fw` comes from pick_example: a re-resolve here would use the global search policy and + miss a firmware ensure_fw just built into cmake-build/ under an exclusive -B.""" + fn = getattr(hil_flash, f'flash_{board["flasher"]["name"].lower()}') + for attempt in range(3): + if attempt == 2: + if not (allow_recovery and probe_port): + return False + cur = find_usb(board['flasher']['uid']) + if cur is None: + # probe gone from the bus: its old busport may now hold an UNRELATED device + # (bus renumbering) and the helper only checks occupancy, so toggling would + # deauthorize an innocent fixture + note.append('probe vanished before toggle') + else: + say(f'{board["name"]:26} recovery: replugging probe {cur[0]} (authorized toggle)') + if recover_probe(board['flasher']['uid'], cur[0]): + note.append('probe replugged') + time.sleep(2) # udev recreates /dev/serial/by-id symlinks after re-enumeration + else: + note.append('probe toggle unconfirmed') + rc, err = call_flasher(fn, board, str(fw)) + if rc == 0: + return True + if rc == 127: # flasher binary missing: retries/probe recovery can't fix env + note.append(f'flasher tool missing ({err}) — esptool needs the ESP-IDF env (get-idf)' + if board['flasher']['name'].lower() == 'esptool' else + f'flasher tool missing: {err}') + return False + if attempt == 0: + say(f'{board["name"]:26} flash retry: {err}') + else: + note.append(f'flash: {err}') + return False + + +def flash_error_line(out: str) -> str: + """Most informative line of a failed flash's output: last error-looking line, + else the last non-empty one.""" + lines = [l.strip() for l in out.splitlines() if l.strip()] + for l in reversed(lines): + if any(k in l.lower() for k in ('error', 'fail', 'unknown', 'cannot', 'timeout', + 'no valid', 'not found', 'unable')): + return l[:90] + return lines[-1][:90] if lines else '' + + +def check_host_serial(board: dict, do_reset: bool = True, want_hello: bool = False) -> bytes | None: + """Host-only boards never enumerate their uid (their USB port is the host side); + aliveness = output on the flasher's UART bridge after a reset. A probe byte is + written each poll so an echo-only firmware (board_test) also answers. Returns + the first output chunk (b'' when silent, None when the port is absent/drops) so + the caller can also judge WHAT answered — see boardtest_output(). + + do_reset=False listens to the firmware as-is: used right after a flash whose + own reset already started it — a second openocd/JLink session back-to-back on + the same probe can fail transiently and leave the target halted.""" + import serial + try: + port = hil_util.get_serial_dev(board['flasher']['uid'], None, None, 0) + ser = serial.Serial(port, baudrate=115200, timeout=0.3, write_timeout=1) + except Exception as e: + say(f'{board["name"]:26} no flasher serial port: {e}') + return None + try: + # flush BEFORE the reset: this drops the pre-reset CDC backlog (which must not + # count as life) while keeping the post-reset boot banner, which prints while the + # reset tool is still tearing down and a post-reset flush would eat + ser.reset_input_buffer() + if do_reset: + getattr(hil_flash, f'reset_{board["flasher"]["name"].lower()}')(board) + # judge the WHOLE window, not the first chunk: the probe's CDC bridge has its own + # FIFO, so stale pre-flash output (e.g. board_test hellos) can arrive after our + # host-side flush and must not decide the verdict alone. + data = b'' + deadline = time.monotonic() + SERIAL_WAIT + while time.monotonic() < deadline: + try: + ser.write(b'U') + data += ser.read(256) + except serial.SerialTimeoutException: + pass + except serial.SerialException: + return None # port dropped mid-poll (bridge re-enumerating) + # early-exit on the caller's positive signal (board_test hello for park + # verification, any non-board_test output for example liveness): stale + # bridge-FIFO backlog of the OTHER kind must not end the window + if want_hello: + if b'Hello from TinyUSB' in data: + return data + elif data and not boardtest_output(data): + return data + return data + finally: + ser.close() + + +def boardtest_output(data: bytes) -> bool: + """True when (non-empty) serial output is recognizably ONLY board_test's: its + periodic HELLO_STR and echoes of our b'U' pokes, nothing else. Any residue + beyond that (an example banner, log lines) proves other firmware is talking, + however much stale board_test backlog surrounds it. Used as a negative + identity marker — after flashing a host example, board_test-only chatter + means the flash silently didn't take (the host analog of the PID check).""" + residue = data.replace(b'Hello from TinyUSB', b'') + for junk in (b'U', b'\r', b'\n'): + residue = residue.replace(junk, b'') + return len(residue) == 0 + + +def build_example(board: dict, variant: str, example: str) -> int: + """Build one example for this board: tools/build.py (same invocation shape as + hil_test.build_board), or idf.py directly for espressif (tools/build.py's esp branch + ignores -T and builds everything; variant flags travel as -DCFLAGS_CLI, the channel + tools/build.py uses). Bounded and process-group-killed via run_cmd; 600 s covers a + first configure+build of an SDK-heavy family (pico, nrf, esp). Builds normally run + pre-lock, so a board flock is not held here except on rare recovery paths. Per-build + compile parallelism is capped at cpu/-j so -j concurrent builds cannot swamp sibling + workers' verification windows. Returns the returncode (127 = ESP-IDF env missing).""" + name = board['name'] + variants = board.get('variant') or [{'name': name}] + vcfg = next((v for v in variants if v['name'] == variant), variants[0]) + if board['flasher']['name'].lower() == 'esptool': + if not shutil.which('idf.py'): + return 127 # ESP-IDF env not sourced in this shell + # -B keyed off the VARIANT so ensure_fw's post-build lookup finds it + cmd = ['idf.py', '-C', f'examples/{example}', + '-B', f'cmake-build/cmake-build-{vcfg["name"]}/{example}', + '-G', 'Ninja', f'-DBOARD={name}', 'build'] + for d in board.get('build', {}).get('args', []) + vcfg.get('defines', []): + cmd.insert(-1, f'-D{d}') + if vcfg.get('flags'): + cmd.insert(-1, f'-DCFLAGS_CLI={vcfg["flags"]}') + # the IDF component manager writes examples/<ex>/dependencies.lock in the + # SOURCE tree (idf.py -B relocates only the build dir), so concurrent esp + # builds of one example for different targets corrupt each other's solve + with _esp_lock, _build_sem: + return hil_util.run_cmd(shlex.join(cmd), cwd=str(hil_util.TINYUSB_ROOT), + timeout=600).returncode + cmd = [sys.executable, str(hil_util.TINYUSB_ROOT / 'tools' / 'build.py'), + '-b', name, '-T', Path(example).name, + '-j', str(max(1, (os.cpu_count() or _jobs) // _jobs))] + for d in board.get('build', {}).get('args', []): + cmd += ['-D', d] + if vcfg['name'] != name: + cmd += ['--build-name', vcfg['name']] + for d in vcfg.get('defines', []): + cmd += ['-D', d] + for tok in vcfg.get('flags', '').split(): + cmd += [f'--cflag={tok}'] + with _build_sem: + return hil_util.run_cmd(shlex.join(cmd), cwd=str(hil_util.TINYUSB_ROOT), + timeout=600).returncode + + +_deps_lock = threading.Lock() # one get_deps at a time (it also drains _build_sem) +_esp_lock = threading.Lock() # idf.py mutates source-tree dependencies.lock per example +_no_build = False # --no-build: ensure_fw never invokes a build +_jobs = 4 # mirrors -j; set in main before the pool starts +_build_sem = threading.BoundedSemaphore(4) # build slots; get_deps drains ALL (exclusive) +_builds: dict = {} # (variant, example) -> (fw|None, reason): one attempt per run + + +def ensure_fw(board: dict, variant: str, example: str, note: list): + """Firmware for `example`, building it when absent — never skip a board for lack of a + build (--no-build opts out). One retry with deps fetched and the CMake caches dropped + when the first build fails (fresh checkouts lack the family deps; a cache configured + in a broken env poisons every later attempt). Returns the firmware path, or None with + the failure noted. Call BEFORE taking the board lock: builds are long. One attempt per + (variant, example) per run, memoized in _builds, so a repeat call (park, under the + held flock) resolves instantly even when an exclusive -B hides the fresh artifact.""" + fw = hil_flash.find_firmware(variant, example, flasher=board['flasher']['name']) + if fw: + return fw + key, base = (variant, example), Path(example).name + if key in _builds: + return _builds[key][0] + if _no_build: + _builds[key] = (None, 'disabled') + note.append(f'build skipped (--no-build): {base}') + return None + rc = build_example(board, variant, example) + if rc == 127 and board['flasher']['name'].lower() == 'esptool': + _builds[key] = (None, 'no-env') + note.append(f'cannot build {base}: ESP-IDF env missing (get-idf)') + return None + if rc == 124: # hung build: a deps/cache retry cannot cure it, don't double the stall + _builds[key] = (None, 'timeout') + note.append(f'build timeout: {base}') + return None + if rc != 0: + # retry once with deps fetched and the CMake caches dropped (cache only — a tree + # wipe would destroy every other example's firmware). get_deps git-resets shared + # deps that are already present, so it drains ALL build slots first. + with _deps_lock: + for _ in range(_jobs): + _build_sem.acquire() + try: + r = hil_util.run_cmd(shlex.join([sys.executable, str(hil_util.TINYUSB_ROOT / 'tools' / 'get_deps.py'), + '-b', board['name']]), + cwd=str(hil_util.TINYUSB_ROOT), timeout=600) + finally: + for _ in range(_jobs): + _build_sem.release() + if r.returncode != 0: + note.append('get_deps failed') + bd = hil_util.TINYUSB_ROOT / 'cmake-build' / f'cmake-build-{variant}' + # esp configures one level deeper (<variant>/<example>/): wipe both layouts + for d in (bd, bd / example): + shutil.rmtree(d / 'CMakeFiles', ignore_errors=True) + (d / 'CMakeCache.txt').unlink(missing_ok=True) + rc = build_example(board, variant, example) + if rc != 0: + _builds[key] = (None, 'fail') + note.append(f'build failed: {base}') + return None + # both build paths write to cmake-build/, so look there even when an explicit -B + # narrowed the global search — this is OUR fresh build, not a stale fallback + fw = hil_flash.find_firmware(variant, example, + roots=[hil_flash.build_dir, 'cmake-build'], + flasher=board['flasher']['name']) + _builds[key] = (fw, 'ok' if fw else 'no-fw') + note.append(f'built {base}' if fw else f'build produced no firmware: {base}') + return fw + + +def ensure_board_test(board: dict, variant: str, note: list): + """board_test firmware for parking, building it if absent (via ensure_fw). + Espressif included — tools/build.py builds board_test for that family too; + the build just needs the ESP-IDF env (127 → noted, park is then skipped).""" + fw = hil_flash.find_firmware(variant, 'device/board_test', flasher=board['flasher']['name']) + if fw: + return fw + variants = board.get('variant') or [{'name': board['name']}] + if not any(v['name'] == variant for v in variants): + variant = variants[0]['name'] + return ensure_fw(board, variant, 'device/board_test', note) + + +def verdict(row: dict, ok: bool) -> str: + """Row status for a verification result, preserving a 'flash-failed' a deeper + layer already recorded (silent flash no-op, board_test delivery failure).""" + return 'ok' if ok else ('flash-failed' if row['status'] == 'flash-failed' else 'failed') + + +def host_alive(board: dict, note: list, row: dict, flashed_example: bool = False) -> bool: + """Serial aliveness with recovery: silent -> (build and) flash board_test (it + hellos every second and echoes) -> recheck. Also cures a silent flash no-op + that left the board crashed. + + With flashed_example=True (a host example was just flashed), board_test-shaped + output FAILS the check: the parked image still talking means the example flash + silently didn't take — the host analog of the device path's PID check. + + Side effect: delivery-class failures (silent no-op, board_test build/flash + failure) set row['status'] = 'flash-failed' so verdict() preserves the cause; + the caller derives the final status from the return value via verdict().""" + data = check_host_serial(board) + if data: + if flashed_example and boardtest_output(data): + note.append('board_test output after example flash: silent flash no-op') + row['status'] = 'flash-failed' + return False + return True + variant = resolve_variant(board, 'device/board_test', note) + fw = ensure_board_test(board, variant, note) + if fw is None: + note.append('serial silent; board_test unavailable') + row['status'] = 'flash-failed' + return False + say(f'{board["name"]:26} recovery: serial silent, flashing board_test') + rc, err = call_flasher(getattr(hil_flash, f'flash_{board["flasher"]["name"].lower()}'), board, str(fw)) + if rc != 0: + note.append(f'serial silent; board_test flash failed: {err}') + row['status'] = 'flash-failed' + return False + if not check_host_serial(board): + return False + if flashed_example: + # board_test talking proves the BOARD is alive, but the just-flashed + # example never produced serial — that verification still fails + note.append('example silent; board alive via board_test reflash') + return False + note.append('recovered via board_test reflash') + return True + + +def device_recover_and_check(board: dict, example: str, variant: str, old_ino, note: list, row: dict, seen: dict) -> bool: + """Wait for the flashed board's uid to re-enumerate; on timeout, try one board + reset (skipped for flashers with no hardware reset — see hil_flash.RESET_NOOP, + it would just burn the wait) and wait again. + + The PID policy is deliberately asymmetric. Pre-reset, the re-enumeration was + caused by the flash itself, so a PID mismatch most likely means the build dir + is stale (the flash DID write what find_firmware found) — warn, don't fail — + UNLESS the firmware was built this very run: then 'stale build' is impossible + and the mismatch can only be a silent flash no-op, which fails. Post-reset, + the re-enumeration proves nothing about the flash (the reset alone explains + it), so a mismatch is treated as a silent flash no-op and fails; an unknown + expected PID scores ok with a 'pid unverified' note in both paths.""" + name = board['name'] + expected_pid = get_expected_pid(example) + built_this_run = _builds.get((variant, example), (None, ''))[1] == 'ok' + + def seen_hit(hit): + seen[board['uid']] = {'name': name, 'busport': hit[0], 'when': time.strftime('%Y-%m-%d %H:%M')} + + hit = wait_device(board['uid'], None, old_ino, ENUM_WAIT) + if hit: + if expected_pid is not None and not hit[1].endswith(expected_pid): + if built_this_run: + row['device'] = f'❌ {hit[1]}' + note.append(f'pid {hit[1]}, this run built {expected_pid}: silent flash no-op') + row['status'] = 'flash-failed' + return False + note.append(f'⚠ pid {hit[1]}, source says {expected_pid}: stale build or silent flash no-op') + elif expected_pid is None: + note.append('pid unverified') + row['device'] = f'✅ {hit[1]}' + seen_hit(hit) + return True + + flasher_name = board['flasher']['name'].lower() + if flasher_name in hil_flash.RESET_NOOP: + note.append(f'no hardware reset available for {flasher_name}') + row['device'] = '❌ not enumerated' + return False + + say(f'{name:26} recovery: uid not up, resetting board') + rc, err = call_flasher(getattr(hil_flash, f'reset_{flasher_name}'), board) + if rc != 0: + note.append(f'reset failed: {err}') + hit = wait_device(board['uid'], None, old_ino, ENUM_WAIT_RETRY) + if not hit: + row['device'] = '❌ not enumerated' + note.append('reset did not help') + return False + if expected_pid is None: + row['device'] = f'✅ {hit[1]}' + note.append('reset recovered (pid unverified)') + seen_hit(hit) + return True + if hit[1].endswith(expected_pid): + row['device'] = f'✅ {hit[1]}' + note.append('reset recovered') + seen_hit(hit) + return True + row['device'] = f'❌ {hit[1]}' + note.append(f'reset recovered wrong pid, expected {expected_pid}: silent flash no-op') + row['status'] = 'flash-failed' + return False + + +def check_board(board: dict, args, allow_recovery: bool, seen: dict) -> dict: + name = board['name'] + row = {'name': name, 'probe': '❌ missing', 'flash': '–', 'device': '–', 'note': [], 'status': 'failed'} + note = row['note'] + + probe = find_usb(board['flasher']['uid']) + if probe: + row['probe'] = f'✅ {probe[0]}' + seen[board['flasher']['uid']] = {'name': f'{name} probe', 'busport': probe[0], + 'when': time.strftime('%Y-%m-%d %H:%M')} + else: + last = seen.get(board['flasher']['uid']) + note.append(f'probe last seen {last["busport"]} {last["when"]}' if last + else 'probe never seen by pool_check') + say(f'{name:26} probe MISSING ({board["flasher"]["name"]} {board["flasher"]["uid"]})') + + # existing firmware only; a missing build is built further down (after a lock peek), + # except in scan/no-build modes and never for a missing probe + example, kind, variant, fw = pick_example(board, note, build_missing=False) + if kind == 'host': + note.append('host-only board') + + if args.scan_only: + hit = find_device(board['uid'], None) + # report the BOARD's usb state too: enumerated (with busport), off-bus (normal + # when parked in board_test), or n/a for host-only boards + if hit: + row['device'] = f'✅ {hit[1]} @{hit[0]}' + elif kind == 'host': + row['device'] = '– n/a (host-only)' + else: + row['device'] = '⚫ off bus (parked?)' + # scan verifies probe presence only, so probe present is ok; a missing probe means + # no firmware could be delivered → flash-failed + row['status'] = 'ok' if probe else 'flash-failed' + if probe: + say(f'{name:26} probe ✅ {probe[0]}' + (f' device {hit[1]}' if hit else '')) + return row + if not probe: + row['status'] = 'flash-failed' + return row + + bt_variant = resolve_variant(board, 'device/board_test', note) + need_example = example is None and not args.no_build + # board_test is also host_alive's recovery image, so host boards pre-build it + # even under --no-park; --no-build gates EVERY build, board_test included + need_bt = (not args.no_build + and (not args.no_park or kind == 'host') + and hil_flash.find_firmware(bt_variant, 'device/board_test', + flasher=board['flasher']['name']) is None) + if need_example or need_bt: + # builds are long and run BEFORE locking (park must never hold the flock through + # one); peek first so minutes of building are not wasted on — or a rebuilt tree + # swapped under — a board CI holds right now + peek = lock_board(name) + if isinstance(peek, str): + if peek.startswith('ERROR:'): # environment failure, not a held lock + row['flash'] = '❌ lock' + row['status'] = 'failed' + else: + row['flash'] = '🔒 locked' + row['status'] = 'locked' + note.append(peek) + say(f'{name:26} locked: {peek}') + return row + unlock_board(peek) + if need_example: + example, kind, variant, fw = pick_example(board, note, build_missing=True) + if need_bt and (example is not None or kind == 'host'): + # skip the park build when the example build already failed on a device board: + # the row returns before any flash/park could use it + ensure_board_test(board, bt_variant, note) + + if example is None: + if not any(n.startswith(('build failed', 'build timeout', 'build produced', + 'build skipped', 'cannot build')) for n in note): + note.append('no firmware built') + if kind != 'host': + row['status'] = 'flash-failed' + say(f'{name:26} probe ✅ {probe[0]} (no firmware to flash)') + return row + # host-only board: aliveness is still checkable without flashing — reset and listen + # to whatever is on it (parked board_test echoes and hellos on the flasher UART) + + lk = lock_board(name) + if isinstance(lk, str): + if lk.startswith('ERROR:'): # environment failure, not a held lock + row['flash'] = '❌ lock' + row['status'] = 'failed' + else: + row['flash'] = '🔒 locked' + row['status'] = 'locked' + note.append(lk) + say(f'{name:26} locked: {lk}') + return row + try: + if example is None: # host-only without firmware: UART-only aliveness check + ok = host_alive(board, note, row) + row['device'] = '✅ serial out' if ok else '❌ no serial out' + row['status'] = verdict(row, ok) + say(f'{name:26} – {row["device"]} (existing firmware)') + return row + + pre = find_device(board['uid'], None) + old_ino = pre[2] if pre else None + + try: + if not flash(board, fw, allow_recovery, probe[0], note): + row['flash'] = f'❌ {Path(example).name}' + row['status'] = 'flash-failed' + say(f'{name:26} flash FAILED ({example})') + return row + row['flash'] = f'✅ {Path(example).name}' + + if kind == 'host': + ok = host_alive(board, note, row, flashed_example=True) + row['device'] = '✅ serial out' if ok else '❌ no serial out' + else: + ok = device_recover_and_check(board, example, variant, old_ino, note, row, seen) + row['status'] = verdict(row, ok) + say(f'{name:26} {row["flash"]} {row["device"]}') + return row + finally: + # teardown for EVERY path that attempted a flash (a failed programmer op can + # still have erased/half-written the target), while the lock is still held + if not args.no_park: + park_board(board, kind, row, note) + finally: + unlock_board(lk) + + +def park_board(board: dict, kind: str, row: dict, note: list) -> None: + """Re-park with board_test, building it if absent (ensure_board_test), and + VERIFY it took: board_test never enumerates USB, so a device board's cafe + device must drop off the bus, and a host board must answer with board_test's + own output — a rc=0 park that changed nothing (silent no-op) must not pass. + A board left unparked marks an ok row flash-failed (never downgrading a + 'failed' verify verdict — that is the more diagnostic signal), with one + exception: an espressif board without the ESP-IDF env cannot build + board_test — noted, not a board fault.""" + # capture BEFORE the park flash: uid-disappearance only verifies the park if the + # device was on the bus to begin with + on_bus_before = kind != 'host' and find_device(board['uid'], None) is not None + variant = resolve_variant(board, 'device/board_test', note) + fw = ensure_board_test(board, variant, note) + if fw is None: + if any(n.startswith('cannot build board_test') for n in note): + note.append('park skipped (no ESP-IDF env)') + else: + # --no-build disables builds, not parking (--no-park is that opt-out): + # a board left running a USB-active image is unparked either way + note.append('unparked: board_test not built (--no-build)' + if any(n.startswith('build skipped (--no-build): board_test') for n in note) + else 'unparked: board_test unavailable (build failed/timed out)') + if row['status'] == 'ok': + row['status'] = 'flash-failed' + return + rc, err = call_flasher(getattr(hil_flash, f'flash_{board["flasher"]["name"].lower()}'), + board, str(fw)) + if rc != 0: + note.append(f'park flash failed: {err}') + if row['status'] == 'ok': + row['status'] = 'flash-failed' + return + if kind == 'host': + # no second reset (the park flash's own reset started board_test); POSITIVE + # marker: its hello must appear, and stale bridge-FIFO output alongside it is not + # disqualifying + data = check_host_serial(board, do_reset=False, want_hello=True) + if not (data and b'Hello from TinyUSB' in data): + note.append('park unverified: no board_test output') + if row['status'] == 'ok': + row['status'] = 'flash-failed' + return + if not on_bus_before: + # never enumerated this run: uid-disappearance cannot tell a verified park from a + # silent no-op — say so instead of passing vacuously + note.append('park unverified (device already off bus)') + return + deadline = time.monotonic() + 6 + while time.monotonic() < deadline: + if find_device(board['uid'], None) is None: + return + time.sleep(0.5) + note.append('park unverified: device still enumerated') + if row['status'] == 'ok': + row['status'] = 'flash-failed' + + +def check_board_safe(board: dict, args, allow_recovery: bool, seen: dict) -> dict: + """Isolate one board's exceptions: a crashing worker must not discard every + other board's row, the table, the topology, and the seen-cache write.""" + try: + return check_board(board, args, allow_recovery, seen) + except Exception as e: + name = board.get('name', '?') + say(f'{name:26} INTERNAL ERROR: {e!r}') + return {'name': name, 'probe': '–', 'flash': '–', 'device': '❌ error', + 'note': [repr(e)[:120]], 'status': 'failed'} + + +def controller_summary() -> list[str]: + """USB topology: controller (PCI addr, vendor) -> bus -> root-port subtree device + counts (hubs included, interfaces/root hubs not). Bus numbers renumber every boot; + PCI addresses and root-port numbers are stable.""" + vendor_names = {'0x1022': 'AMD', '0x1912': 'Renesas', '0x8086': 'Intel', '0x1b21': 'ASMedia'} + ctrl = {} + for root in glob.glob('/sys/bus/usb/devices/usb*'): + bus = int(os.path.basename(root)[3:]) + m = re.findall(r'[0-9a-f]{4}:[0-9a-f]{2}:[0-9a-f]{2}\.[0-9a-f]', os.path.realpath(root)) + pci = m[-1] if m else '?' + c = ctrl.setdefault(pci, {'vendor': '?', 'buses': {}}) + subtrees = {} + for d in glob.glob(f'/sys/bus/usb/devices/{bus}-*'): + b = os.path.basename(d) + if ':' in b: + continue + subtrees[b.split('.')[0]] = subtrees.get(b.split('.')[0], 0) + 1 + c['buses'][bus] = subtrees + try: + vid = open(f'/sys/bus/pci/devices/{pci}/vendor').read().strip() + c['vendor'] = vendor_names.get(vid, vid) + except OSError: + pass + + lines = [] + for pci, c in sorted(ctrl.items()): + lines.append(f'{pci} ({c["vendor"]})') + for bus, subtrees in sorted(c['buses'].items()): + detail = ' '.join(f'{k}: {n} dev' for k, n in + sorted(subtrees.items(), key=lambda i: int(i[0].split('-')[1]))) + lines.append(f' bus {bus}: {sum(subtrees.values())} devices' + + (f' {detail}' if detail else '')) + return lines + + +def main() -> None: + # toolchain/flasher CLIs live in the user bin dirs, which non-login shells may lack -- + # the same PATH shim hil_ci.sh applies on the remote side + for d in (Path.home() / 'bin', Path.home() / '.local' / 'bin'): + if d.is_dir() and str(d) not in os.environ.get('PATH', '').split(os.pathsep): + os.environ['PATH'] = f'{d}{os.pathsep}{os.environ.get("PATH", "")}' + + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument('config', nargs='?', help='HIL config json (default: by hostname)') + parser.add_argument('-b', '--board', action='append', default=[], help='only these boards') + parser.add_argument('-B', '--build-dir', default=None, + help='firmware parent dir, searched EXCLUSIVELY when given ' + '(default: examples, plus cmake-build as fallback)') + parser.add_argument('--scan-only', action='store_true', + help='USB presence scan only: no locks, no flashing') + parser.add_argument('--no-build', action='store_true', + help='do not build missing firmware (default: build the light example on the spot)') + parser.add_argument('--no-park', action='store_true', + help='leave the light example running (default: park with board_test)') + # no cross-process flash budget against a concurrent hil_test.py run (its semaphores + # are in-process), so keep this modest + parser.add_argument('-j', '--jobs', type=int, default=4) + parser.add_argument('-v', '--verbose', action='store_true') + args = parser.parse_args() + global _no_build, _jobs, _build_sem + _no_build = args.no_build + _jobs = max(1, args.jobs) + _build_sem = threading.BoundedSemaphore(_jobs) + + host = socket.gethostname() + cfg_name = args.config or CONFIG_BY_HOST.get(host, 'local.json') + cfg_path = Path(cfg_name) + if not cfg_path.exists(): + cfg_path = REPO_ROOT / 'test' / 'hil' / cfg_name + if not cfg_path.exists(): + sys.exit(f'config not found: {cfg_name} (host {host}; dev PCs need test/hil/local.json)') + with cfg_path.open() as f: + config = json.load(f) + + boards = list(config['boards']) # boards-skip (parked hardware) is not scanned by default + if args.board: + boards += config.get('boards-skip', []) # explicitly named parked boards are fair game + unknown = set(args.board) - {b['name'] for b in boards} + if unknown: + sys.exit(f'board(s) not in {cfg_path.name}: {", ".join(sorted(unknown))}') + boards = [b for b in boards if b['name'] in args.board] + + hil_flash.build_dir = args.build_dir or 'examples' + hil_util.verbose = args.verbose + if args.build_dir is None: + # default mode: search both standard layouts (cmake-build/ from tools/build.py and + # ESP-IDF, examples/ from manual builds). An EXPLICIT -B stays exclusive: the caller + # named an artifact tree, so a miss must report rather than flash an older build. + hil_flash.EXTRA_BUILD_DIRS = ['cmake-build', 'examples'] + allow_recovery = not args.scan_only and can_recover() + seen = {} + try: + loaded = json.loads(SEEN_CACHE.read_text()) + if isinstance(loaded, dict): # tolerate a torn/hand-edited cache + seen = {k: v for k, v in loaded.items() if isinstance(v, dict)} + except (OSError, ValueError): + pass + + roots = ' + '.join(dict.fromkeys([hil_flash.build_dir, *hil_flash.EXTRA_BUILD_DIRS])) + say(f'pool check: host {host}, config {cfg_path.name}, {len(boards)} boards, ' + f'{"scan-only" if args.scan_only else f"flash via {{{roots}}}/cmake-build-<board>"}' + f'{"" if allow_recovery or args.scan_only else ", recovery unavailable (no sudo -n / usb_recover.sh)"}') + + if args.verbose: + rows = [check_board_safe(b, args, allow_recovery, seen) for b in boards] + else: + with io.StringIO() as spool, ThreadPoolExecutor(max_workers=args.jobs) as pool: + sys.stdout = spool # silence hil_util.run_cmd's COMMAND FAILED dumps; say() uses __stdout__ + try: + rows = list(pool.map(lambda b: check_board_safe(b, args, allow_recovery, seen), boards)) + finally: + sys.stdout = sys.__stdout__ + + try: + SEEN_CACHE.parent.mkdir(parents=True, exist_ok=True) + tmp = SEEN_CACHE.with_suffix('.json.tmp') + tmp.write_text(json.dumps(seen, indent=1, sort_keys=True) + '\n') + tmp.replace(SEEN_CACHE) # atomic: a killed run can't tear the cache + except OSError: + pass + + status_mark = {'ok': '✅ ok', 'flash-failed': '❌ flash-failed', 'failed': '❌ failed', + 'locked': '🔒 locked'} + headers = ['Board', 'Probe', 'Flash', 'Device', 'Status', 'Note'] + cells = [[r['name'], r['probe'], r['flash'], r['device'], + status_mark.get(r['status'], r['status']), '; '.join(r['note'])] for r in rows] + widths = [max(len(h), *(len(c[i]) for c in cells)) if cells else len(h) + for i, h in enumerate(headers)] + line = lambda vals: '| ' + ' | '.join(v.ljust(w) for v, w in zip(vals, widths)) + ' |' + print() + print(line(headers)) + print('|' + '|'.join('-' * (w + 2) for w in widths) + '|') + for c in cells: + print(line(c)) + + print('\nUSB topology (controller → root-port subtree):') + for line in controller_summary(): + print(f' {line}') + + counts = {'ok': 0, 'flash-failed': 0, 'failed': 0, 'locked': 0} + for r in rows: + counts[r.get('status', 'failed')] += 1 + print(f'\n{counts["ok"]} ok · {counts["flash-failed"]} flash-failed · {counts["failed"]} failed ' + f'· {counts["locked"]} locked · in {time.monotonic() - t0:.0f}s') + if hil_util.sysfs_blind(): + # Without this the table is the worst kind of wrong: once the process latches + # blind, every read answers SYSFS_UNKNOWN, scan_usb() returns {}, and EVERY board + # prints "probe MISSING"/"off bus" -- a clean-looking report declaring the whole + # fleet dead, produced during exactly the incident this tool is run to diagnose, + # and it sends the operator to power-cycle a rig where one device is wedged. + print('WARNING: this scan lost sight of the bus' + f'{hil_util.sysfs_blind_note()}. Rows above that say a probe or board is ' + f'missing may be this tool losing sight of healthy hardware, not absent ' + f'hardware. Find the wedged device (see the usb-kernel-recover skill) and ' + f're-run before acting on the table.') + sys.exit(min(counts['flash-failed'] + counts['failed'], 125)) + + +if __name__ == '__main__': + main() diff --git a/test/hil/helper/hil_select.py b/test/hil/helper/hil_select.py new file mode 100755 index 000000000..f0d4f0b9f --- /dev/null +++ b/test/hil/helper/hil_select.py @@ -0,0 +1,524 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +"""PR-diff -> HIL selection: which rig boards and which tests a change can affect. + +Stdlib-only (runs on bare CI runners; imports hil_util for the example rosters, +never hil_test/pyserial — test_hil_util.BottomLayer enforces the stdlib closure). +Fail-open: any file no rule classifies forces the full matrix. See +docs/superpowers/specs/2026-07-29-hil-pr-scoped-selection-design.md. + +JSON: full, boards (name -> 'all' | [tests]), families (bsp families the diff +touches, including ones with no rig board - build-only consumers such as /pre-pr +sample from these), args (hil_test.py args per config) and args_flasher (the same +args split by each board's flasher, for CI legs that split one rig by flasher). +""" +import argparse +import functools +import glob +import json +import os +import re +import subprocess +import sys + +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # helper/ scripts import via the test/hil root +from helper.hil_util import device_tests, dual_tests, host_test + +ALL_TESTS = {'device': device_tests, 'dual': dual_tests, 'host': host_test} + +# class dir -> config macro suffix exceptions (rule 3); dfu is per-file, handled inline +NET_MACROS = ('ECM_RNDIS', 'NCM') + +_NONCODE_RE = re.compile( + r'^(docs/|\.claude/|.*\.(md|rst)$|LICENSE)') +_FULL_RE = re.compile( + r'^(src/common/|src/osal/|src/tusb\.c$|src/tusb\.h$|src/tusb_option\.h$|' + r'test/hil/|\.github/workflows/build.*\.yml$|\.github/actions/|\.github/scripts/|' + r'tools/build\.py$|tools/get_deps\.py$|tools/cmake/|hw/mcu/|lib/|' + r'hw/bsp/(family_support\.cmake|board_api\.h|board\.c|ansi_escape\.h)$|' + r'examples/build_system/|examples/CMakeLists\.txt$|' + # board_test is HIL infrastructure, not a test: hil_test.py flashes it to park + # every board (variant boundary + end-of-board teardown), so every board depends on it + r'examples/device/board_test/)') + +# --no-renames: with rename detection git reports only a rename's destination, so code +# moved out of an HIL-relevant path would be classified by its new path alone +GIT_DIFF_ARGV = ['git', 'diff', '--no-renames', '--name-only'] + + +def test_role(test: str) -> str: + return test.split('/', 1)[0] # 'device' | 'dual' | 'host' + + +def board_roles(board: dict) -> set: + t = board.get('tests', {}) + roles = set() + if t.get('device'): + roles.add('device') + if t.get('host'): + roles.add('host') + if t.get('dual'): + roles.update(('device', 'host')) + for only in t.get('only', []): + r = test_role(only) + roles.update(('device', 'host') if r == 'dual' else (r,)) + return roles + + +def board_tests(board: dict) -> list: + """Every test this board would run today (mirrors hil_test.test_board's default).""" + t = board.get('tests', {}) + if 'only' in t: + run = list(t['only']) + else: + run = [] + if t.get('device'): + run += device_tests + if t.get('dual'): + run += dual_tests + if t.get('host'): + run += host_test + return [x for x in run if x not in t.get('skip', [])] + + +# cached: called per changed file x roster board, and the tree doesn't change mid-run [email protected]_cache(maxsize=None) +def board_family(board_name: str, repo_root: str): + hits = glob.glob(os.path.join(repo_root, 'hw/bsp/*/boards', board_name)) + return os.path.basename(os.path.dirname(os.path.dirname(hits[0]))) if hits else None + + +# `if (OPTION STREQUAL "1")` guards in family_support.cmake, and the option tokens +# a roster entry passes to the build (NAME=VALUE / -DNAME=VALUE) +_CM_IF_RE = re.compile(r'if\s*\(') +_CM_ELSE_RE = re.compile(r'else(if)?\s*\(') +_CM_ENDIF_RE = re.compile(r'endif\s*\(') +_CM_OPT_RE = re.compile(r'if\s*\(\s*\$?\{?([A-Za-z_]\w*)\}?\s+STREQUAL\s+"?1"?\s*\)') +_CM_PORT_RE = re.compile(r'src/portable/((?:[^/\s]+/)?[^/\s]+)/') +_FALSY = ('', '0', 'off', 'false', 'no') + + [email protected]_cache(maxsize=None) +def port_option_gates(repo_root: str) -> dict: + """port dir -> build options that compile it regardless of the board's family + file, e.g. {'analog/max3421': {'MAX3421_HOST'}} from family_support.cmake.""" + gates = {} + try: + text = open(os.path.join(repo_root, 'hw/bsp/family_support.cmake')).read() + except OSError: + return gates + stack = [] # one entry per open if(): its option, or None + for line in text.splitlines(): + line = line.strip() + if _CM_IF_RE.match(line): + m = _CM_OPT_RE.match(line) + stack.append(m.group(1) if m else None) + elif _CM_ELSE_RE.match(line): + if stack: + stack[-1] = None # the guard doesn't hold in this branch + elif _CM_ENDIF_RE.match(line): + if stack: + stack.pop() + opts = {o for o in stack if o} + m = _CM_PORT_RE.search(line) + if opts and m: + gates.setdefault(m.group(1), set()).update(opts) + return gates + + +_CM_SET_RE = re.compile(r'set\s*\(\s*([A-Za-z_]\w*)\s+([^)\s]+)\s*\)') + + +# cached: called per changed portable file x roster board [email protected]_cache(maxsize=None) +def bsp_board_options(board_name: str, repo_root: str) -> frozenset: + """Build options a board turns on in its own BSP: `set(<OPT> <value>)` in + hw/bsp/<family>/boards/<board>/board.cmake, e.g. MAX3421_HOST on the espressif + and rp2040 max3421 boards. CMake only - HIL CI builds nothing with Make, so a + board.mk-only option (e.g. nrf5340dk's MAX3421_HOST) compiles no port here.""" + fam = board_family(board_name, repo_root) + if not fam: + return frozenset() + path = os.path.join(repo_root, 'hw/bsp', fam, 'boards', board_name, 'board.cmake') + try: + text = open(path).read() + except OSError: + return frozenset() + out = set() + for line in text.splitlines(): + line = line.strip() + if line.startswith('#'): + continue + m = _CM_SET_RE.match(line) + if m and m.group(2).strip('"').lower() not in _FALSY: + out.add(m.group(1)) + return frozenset(out) + + +def board_options(board: dict, repo_root: str) -> set: + """Build options a board has truthy: the roster entry's build.args plus each + variant's defines (NAME=VALUE) and raw CFLAGS (-DNAME=VALUE), plus whatever its + own board.cmake sets (a board can enable a gated port without the roster saying so).""" + toks = list(board.get('build', {}).get('args', [])) + for v in board.get('variant', []): + toks += list(v.get('defines', [])) + toks += v.get('flags', '').split() + out = set(bsp_board_options(board['name'], repo_root)) + for t in toks: + name, _, val = (t[2:] if t.startswith('-D') else t).partition('=') + if name and val.strip().strip('"').lower() not in _FALSY: + out.add(name.strip()) + return out + + [email protected]_cache(maxsize=None) +def port_families(port_dir: str, repo_root: str) -> set: + """Board families that compile this src/portable dir. CMake only: HIL CI builds + every board with CMake, so a port wired up in family.mk alone is compiled for no + HIL board and must not select one. family.cmake lists portable sources directly + for most families; espressif instead references them from a nested component + CMakeLists.txt (hw/bsp/espressif/components/tinyusb_src/CMakeLists.txt).""" + fams = set() + bsp_root = os.path.join(repo_root, 'hw/bsp') + # trailing '/' so a port dir is not a prefix of a sibling: bare 'microchip/pic' + # would otherwise match '.../microchip/pic32mz/...' and inherit its families + needle = port_dir + '/' + for f in glob.glob(os.path.join(bsp_root, '*/family.cmake')) + \ + glob.glob(os.path.join(bsp_root, '*/components/*/CMakeLists.txt')): + try: + if needle in open(f).read(): + fam = os.path.relpath(f, bsp_root).split(os.sep, 1)[0] + fams.add(fam) + except OSError: + pass + return fams + + +_CLS_INC_RE = re.compile(r'#\s*include\s*[<"]class/([^/"<>]+)/([^"<>]+)[">]') + + [email protected]_cache(maxsize=None) +def class_include_edges(repo_root: str) -> dict: + """'<class>/<header>' -> the other class dirs that include it. A class header + pulled in by a second class ships in every firmware enabling that second class: + src/class/midi/midi{,2}_{device,host}.h include class/audio/audio.h, and + net_device.h includes class/cdc/cdc.h. The class rule derives macros from the + directory name alone, so without this edge a change to the included header + selects only its own class's examples - and on a board that skips those (e.g. + metro_m4_express skips audio_test_freertos), nothing at all. + + Derived from the actual #include lines rather than a hand-written table so it + cannot rot when a class picks up or drops a cross-class include.""" + edges = {} + for f in sorted(glob.glob(os.path.join(repo_root, 'src/class/*/*.[ch]'))): + cls = os.path.basename(os.path.dirname(f)) + try: + text = open(f).read() + except OSError: + continue + for inc_cls, inc_hdr in _CLS_INC_RE.findall(text): + if inc_cls != cls: + edges.setdefault(f'{inc_cls}/{inc_hdr}', set()).add(cls) + return edges + + +def class_macros(cls: str, base: str, prefix: str) -> list: + """Config macros that compile a class dir's code, for role prefix TUD/TUH. + `base` refines dfu only (it splits DFU from DFU_RUNTIME per file); pass '' for + a class reached through an include edge, where the widest set is correct.""" + if cls == 'net': + return [f'CFG_{prefix}_{m}' for m in NET_MACROS] + if cls == 'dfu': + if base.startswith('dfu_rt'): + return [f'CFG_{prefix}_DFU_RUNTIME'] + if base.startswith('dfu_device') or base.startswith('dfu_host'): + return [f'CFG_{prefix}_DFU'] + return [f'CFG_{prefix}_DFU', f'CFG_{prefix}_DFU_RUNTIME'] + return [f'CFG_{prefix}_{cls.upper()}'] + + +def _config_enables(cfg_path: str, macros) -> bool: + try: + text = open(cfg_path).read() + except OSError: + return False + return any(re.search(rf'#define\s+{m}\s+\(?\s*0*[1-9]', text) for m in macros) + + +def roster_only_tests(all_boards) -> set: + """Test paths that only appear in a roster board's tests.only list (e.g. + espressif boards), not in the shared device/dual/host_test lists.""" + out = set() + for b in all_boards: + out.update(b.get('tests', {}).get('only', [])) + return out + + +def class_examples(macros, role: str, repo_root: str, extra_tests: set) -> set: + """Tests (from role's + dual lists, plus roster-only-list tests of that role) + whose example config enables any macro.""" + pool = role_tests({role}, extra_tests) + out = set() + for test in pool: + cfg = os.path.join(repo_root, 'examples', test, 'src', 'tusb_config.h') + if _config_enables(cfg, macros): + out.add(test) + return out + + +def role_tests(roles: set, extras: set) -> set: + """Every test for the given role(s): each role's own list + dual tests, + plus roster-only-list tests (extras) matching those roles or 'dual'.""" + pool = set(dual_tests) + for r in roles: + pool |= set(ALL_TESTS[r]) + pool |= {t for t in extras if test_role(t) in roles or test_role(t) == 'dual'} + return pool + + +class _Sel: + """Accumulates contributions. board->set(tests) plus 'all-board' markers.""" + def __init__(self): + self.full = False + self.by_board = {} # name -> set of tests, or 'all' + self.roles = set() # roles touched by any contribution + self.families = set() # bsp families touched (incl. off-rig ones: build-only consumers) + self.reasons = [] + + def add(self, boards, tests, reason): + """tests: 'all' or iterable of test paths.""" + self.reasons.append(reason) + for b in boards: + cur = self.by_board.get(b) + if tests == 'all' or cur == 'all': + self.by_board[b] = 'all' + else: + self.by_board[b] = (cur or set()) | set(tests) + + def force_full(self, reason): + self.full = True + self.reasons.append(reason) + + +def _classify_one(path, repo_root, roster_boards, extras: set, s: _Sel): + base = os.path.basename(path) + if _NONCODE_RE.match(path): + s.reasons.append(f'{path}: non-code, no contribution') + return + if _FULL_RE.match(path): + s.force_full(f'{path}: core/infra -> full matrix') + return + + m = re.match(r'src/portable/((?:[^/]+/)?[^/]+)/', path) + if m: + port = m.group(1) + if re.match(r'(dcd_|.*_device)', base): + roles = {'device'} + elif re.match(r'(hcd_|.*_host)', base): + roles = {'host'} + else: + roles = {'device', 'host'} + fams = port_families(port, repo_root) + if not fams: + # no family references this port: either a new/renamed port dir or a + # family.cmake layout the scan misses - widen instead of contributing nothing + s.force_full(f'{path}: port {port} maps to no board family -> full matrix') + return + s.families.update(fams) + # a board can also pull the port in through a build option (e.g. MAX3421_HOST=1 + # from the roster on metro_m4_express, or from its own board.cmake), which its + # family file never names + gates = port_option_gates(repo_root).get(port, set()) + boards = [b['name'] for b in roster_boards + if (board_family(b['name'], repo_root) in fams or + (gates and board_options(b, repo_root) & gates)) and (board_roles(b) & roles)] + tests = role_tests(roles, extras) + s.roles.update(roles) + why = f'{path}: port {port} -> families {sorted(fams)}' + if gates: + why += f' + option {sorted(gates)}' + s.add(boards, tests, f'{why} -> boards {boards} ({"/".join(sorted(roles))})') + return + + m = re.match(r'src/class/([^/]+)/', path) + if m: + cls = m.group(1) + if re.search(r'_device\.[ch]$', base): + roles = {'device'} + elif re.search(r'_host\.[ch]$', base): + roles = {'host'} + else: + roles = {'device', 'host'} + # this file's own class, plus any class whose headers include it + via = sorted(class_include_edges(repo_root).get(f'{cls}/{base}', ())) + + def macros(prefix): + return (class_macros(cls, base, prefix) + + [m2 for c in via for m2 in class_macros(c, '', prefix)]) + tests = set() + if 'device' in roles: + tests |= class_examples(macros('TUD'), 'device', repo_root, extras) + if 'host' in roles: + tests |= class_examples(macros('TUH'), 'host', repo_root, extras) + boards = [b['name'] for b in roster_boards if board_roles(b) & roles] + s.roles.update(roles) + why = f'{path}: class {cls}' + (f' (+ included by {via})' if via else '') + s.add(boards, tests, f'{why} -> {sorted(tests)} ({"/".join(sorted(roles))})') + return + + m = re.match(r'src/(device|host)/', path) + if m: + role = m.group(1) + boards = [b['name'] for b in roster_boards if role in board_roles(b)] + s.roles.add(role) + s.add(boards, role_tests({role}, extras), f'{path}: core {role} stack -> all {role} tests') + return + + m = re.match(r'hw/bsp/([^/]+)/(?:boards/([^/]+)/)?', path) + if m: + fam, brd = m.group(1), m.group(2) + s.families.add(fam) + if brd: + boards = [b['name'] for b in roster_boards if b['name'] == brd] + why = f'{path}: bsp board {brd}' + else: + boards = [b['name'] for b in roster_boards + if board_family(b['name'], repo_root) == fam] + why = f'{path}: bsp family {fam}' + s.roles.update(('device', 'host')) + s.add(boards, 'all', f'{why} -> boards {boards}') + return + + m = re.match(r'examples/(device|host|dual)/([^/]+)/', path) + if m: + test = f'{m.group(1)}/{m.group(2)}' + known = any(test in pool for pool in ALL_TESTS.values()) or test in extras + if known: + boards = [b['name'] for b in roster_boards] + role = test_role(test) + s.roles.update(('device', 'host') if role == 'dual' else (role,)) + s.add(boards, [test], f'{path}: example -> {test} on all boards') + else: + s.reasons.append(f'{path}: example not in HIL lists, no contribution') + return + + s.force_full(f'{path}: unclassified -> full matrix') + + +def classify(changed_files, repo_root, rosters): + all_boards = [] + seen = set() + for _, boards in rosters: + for b in boards: + if b['name'] not in seen: + seen.add(b['name']) + all_boards.append(b) + + extras = roster_only_tests(all_boards) + s = _Sel() + # no early exit once full: keep classifying so `families` still reports every + # family the diff touches (build-only consumers need it). Nothing after the first + # force_full can change full/boards/args - the full branch below ignores by_board. + for path in changed_files: + _classify_one(path, repo_root, all_boards, extras, s) + + if s.full: + return {'full': True, 'boards': {b['name']: 'all' for b in all_boards}, + 'families': sorted(s.families), 'reasons': s.reasons} + + # role pruning: single-role selections drop the other role's tests and boards + by_name = {b['name']: b for b in all_boards} + out = {} + for name, tests in s.by_board.items(): + allowed = board_tests(by_name[name]) + if tests == 'all': + kept = list(allowed) + else: + kept = [t for t in allowed if t in tests] + if s.roles and s.roles != {'device', 'host'}: + role = next(iter(s.roles)) + kept = [t for t in kept if test_role(t) in (role, 'dual')] + if kept: + out[name] = 'all' if set(kept) == set(allowed) else sorted(kept) + return {'full': False, 'boards': out, 'families': sorted(s.families), + 'reasons': s.reasons} + + +def _board_args(name, chosen) -> list: + parts = [f'-b {name}'] + if chosen != 'all': + parts.append(f'-bt {name}:{",".join(chosen)}') + return parts + + +def selection_args(sel, rosters): + """hil_test.py args per config. Empty means either 'full matrix' or 'nothing + selected' - callers must read sel['full'] to tell them apart.""" + args = {} + for cfg_path, boards in rosters: + parts = [] + if not sel['full']: + for b in boards: + chosen = sel['boards'].get(b['name']) + if chosen is not None: + parts += _board_args(b['name'], chosen) + args[os.path.basename(cfg_path)] = ' '.join(parts) + return args + + +def selection_args_by_flasher(sel, rosters): + """{config: {flasher name: args}}. CI runs one rig as several jobs split by + flasher (esptool vs the rest); each must gate on its own subset, otherwise the + other leg runs a filter matching zero boards and reports a vacuous green.""" + out = {} + for cfg_path, boards in rosters: + per = {} + if not sel['full']: + for b in boards: + chosen = sel['boards'].get(b['name']) + if chosen is None: + continue + per.setdefault(b.get('flasher', {}).get('name', ''), []).extend( + _board_args(b['name'], chosen)) + out[os.path.basename(cfg_path)] = {f: ' '.join(p) for f, p in per.items()} + return out + + +def changed_files_from_git(base, repo_root): + mb = subprocess.run(['git', 'merge-base', 'HEAD', base], cwd=repo_root, + capture_output=True, text=True, check=True).stdout.strip() + diff = subprocess.run(GIT_DIFF_ARGV + [f'{mb}..HEAD'], cwd=repo_root, + capture_output=True, text=True, check=True).stdout + return [l for l in diff.splitlines() if l.strip()] + + +def main(): + ap = argparse.ArgumentParser(description=__doc__) + g = ap.add_mutually_exclusive_group(required=True) + g.add_argument('--base', help='git ref to diff against (merge-base..HEAD)') + g.add_argument('--diff-file', help='newline-separated changed-file list') + ap.add_argument('configs', nargs='+', help='rig roster JSON file(s)') + a = ap.parse_args() + + # test/hil/helper/ -> repo root is FOUR levels up; three left this at <repo>/test + # after the helper/ move and every repo-relative glob silently matched nothing + repo_root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))) + rosters = [] + for c in a.configs: + with open(c) as f: + rosters.append((c, json.load(f)['boards'])) + + files = (open(a.diff_file).read().splitlines() if a.diff_file + else changed_files_from_git(a.base, repo_root)) + files = [f for f in files if f.strip()] + + s = classify(files, repo_root, rosters) + s['args'] = selection_args(s, rosters) + s['args_flasher'] = selection_args_by_flasher(s, rosters) + for r in s['reasons']: + print(f'hil_select: {r}', file=sys.stderr) + print(json.dumps(s)) + + +if __name__ == '__main__': + main() diff --git a/test/hil/helper/hil_util.py b/test/hil/helper/hil_util.py new file mode 100644 index 000000000..54984d20f --- /dev/null +++ b/test/hil/helper/hil_util.py @@ -0,0 +1,585 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +# Bottom layer of the HIL harness: the bounded command runner plus the shared helpers and +# data every other module needs. Stays stdlib-only and imports nothing local -- everything +# else imports this, including the unit tests on GitHub's bare runner; never import them +# from here. Callers set the module global `verbose`. + +from __future__ import annotations + +import glob +import os +import signal +import subprocess +import threading +import sys +from pathlib import Path +from typing import Any + + +# ------------------------------------------------------------- +# HIL example test lists, shared by hil_test.py (runner) and hil_select.py (PR-diff +# selector). Run order is shuffled per board (see test_board); every example carries a +# unique hardcoded idProduct (see its usb_descriptors.c). +# ------------------------------------------------------------- + +# device tests +device_tests = [ + 'device/cdc_dual_ports', + 'device/cdc_msc', + 'device/dfu', + 'device/cdc_msc_throughput', + 'device/audio_test_freertos', + 'device/dfu_runtime', + 'device/cdc_msc_freertos', + 'device/hid_boot_interface', + 'device/msc_dual_lun', + 'device/hid_generic_inout', + 'device/printer_to_cdc', + 'device/midi_test', + 'device/mtp', + 'device/usbtest', # cafe:4010, unique PID; runs the Linux testusb tier-4 battery via usbtest.py + # 'device/net_lwip_webserver', # disabled for PR #3605: USB net iface enum is flaky on the CI HIL host +] + +dual_tests = [ + 'dual/host_info_to_device_cdc', +] + +host_test = [ + 'host/cdc_msc_hid', + 'host/msc_file_explorer', + 'host/msc_file_explorer_freertos', + 'host/device_info', +] + +verbose = False + +def pos_int_env(name: str, default: int) -> int: + # One parsing policy for every HIL_* knob: a bare int() crashes every run at import + # on a malformed value, and 0/negative silently removes the bound the knob enforces. + try: + v = int(os.getenv(name, str(default))) + except ValueError: + print(f'warning: {name} is not an integer; using {default}', + file=sys.stderr, flush=True) + return default + if v <= 0: + print(f'warning: {name}={v} is not usable; using {default}', + file=sys.stderr, flush=True) + return default + return v + + +def pos_float_env(name: str, default: float) -> float: + try: + v = float(os.getenv(name, str(default))) + except ValueError: + print(f'warning: {name} is not a number; using {default}', + file=sys.stderr, flush=True) + return default + # float() accepts 'inf'/'nan': an infinite serial timeout is an unbounded read, the + # very thing these knobs exist to prevent, and nan fails every comparison silently + if not (v > 0 and v < float('inf')): + print(f'warning: {name}={v} is not usable; using {default}', + file=sys.stderr, flush=True) + return default + return v + + +CMD_TIMEOUT = pos_int_env('HIL_CMD_TIMEOUT', 180) + +TINYUSB_ROOT = Path(__file__).resolve().parents[3] # test/hil/helper/ -> repo root + + +def cmd_stdout_text(out: Any) -> str: + if out is None: + return '' + if isinstance(out, bytes): + return out.decode('utf-8', errors='ignore') + return str(out) + + +def _banner_body(out: Any, err: Any) -> str: + # split_stderr callers keep the diagnostic in stderr — a banner of stdout alone + # would be blank exactly when something went wrong + body = cmd_stdout_text(out) + err_text = cmd_stdout_text(err) + if err_text: + body = f'{body}\n{err_text}' if body else err_text + return body + + +# Shared with compact_output's stripper in hil_test: duplicated literals let the two +# layers drift and reintroduce literal marker noise mid-row in the GitHub log. +GROUP_MARK, ENDGROUP_MARK = '::group::', '::endgroup::' + + +def strip_workflow_markers(line: str) -> str: + # run_cmd only ever emits markers at line start; mid-line is not a real case. + return line.removeprefix(GROUP_MARK).removeprefix(ENDGROUP_MARK) + + +def _ci_log_groups() -> bool: + # GitHub folds ::group::/::endgroup:: only at line start of the JOB's real stdout; a + # pool worker's capture is compacted into one row line, where they render literally. + return bool(os.getenv('CI')) and sys.stdout is sys.__stdout__ + + +def _print_banner(title: str, out: Any, err: Any) -> None: + print() + if _ci_log_groups(): + print(f'{GROUP_MARK}{title}') + print(_banner_body(out, err)) + print(ENDGROUP_MARK) + else: + print(title) + print(_banner_body(out, err)) + + +SYSFS_READ_GRACE = 2.0 # bound on one attribute read of a possibly-wedged device +SYSFS_STUCK_MAX = 4 # stranded readers tolerated before read_sysfs goes blind +_sysfs_stuck = 0 # each costs a thread + an fd for the life of the process +_sysfs_stuck_lock = threading.Lock() +_sysfs_blind_logged = False + + +class _SysfsUnknown: + """Sentinel: the read did not answer. NOT "the attribute is absent" -- reading it as + absence turns a healthy board into a firmware regression in the report.""" + __slots__ = () + + def __bool__(self) -> bool: + return False + + def __repr__(self) -> str: + return 'SYSFS_UNKNOWN' + + +SYSFS_UNKNOWN = _SysfsUnknown() + + +def sysfs_blind() -> bool: + """True once this process has stranded SYSFS_STUCK_MAX readers: every later read + answers SYSFS_UNKNOWN, so nothing it reports about a device is a fact any more.""" + return _sysfs_stuck >= SYSFS_STUCK_MAX + + +def sysfs_blind_note() -> str: + """Suffix for a failure message, so a blind worker's verdict never reads as hardware.""" + return (f' (this worker is blind: {SYSFS_STUCK_MAX} sysfs reads stranded on a wedged ' + f'device, so the check could not see the bus)') if sysfs_blind() else '' + + +def read_sysfs(path: str, grace: float = SYSFS_READ_GRACE) -> str | None | _SysfsUnknown: + """Read a sysfs attribute with a WALL-CLOCK bound. + + The value, None when the attribute is genuinely unreadable (OSError), or SYSFS_UNKNOWN + when the read did not answer -- it timed out, or this process is already blind. Callers + MUST keep those apart: absence is a fact, unknown is not. + + usb_string_attr (serial/product/manufacturer) is served under the device lock a wedged + usbfs ioctl holds, so a plain open().read() blocks for as long as the wedge lasts, on + exactly the board an incident is about. The reader sleeps INTERRUPTIBLY (every read + takes usb_lock_device_interruptible, v6.12.96 sysfs.c:124-139 -- uninterruptible is the + ioctl holder, not us), so it dies with a SIGKILLed worker; what it costs meanwhile is a + thread and an fd for this process's life, because on sysfs the open() SUCCEEDS and only + the read blocks. Measured: 20 blocking reads leave 20 live threads. + + Hence the cap: callers rescan (hil_lock's controller_of re-reads every unresolved + device on EVERY permit), and hitting RLIMIT_NOFILE or the thread ceiling raises inside + the worker and loses every board's result -- worse than the hang this prevents. + """ + if sysfs_blind(): + return SYSFS_UNKNOWN + # Known-stranded? Re-reading costs another permanent thread+fd and a blindness credit + # to learn what we already know. Lives HERE, not at the call sites: a call-site memo + # has to be remembered by every new scanner, and twice it was not. + was = _sysfs_stranded.get(path, _STRAND_MISS) + if was is not _STRAND_MISS: + if was is None: + return SYSFS_UNKNOWN # stranded, inode unknown: never re-read it + try: + if os.stat(path).st_ino == was: + return SYSFS_UNKNOWN # same node, still wedged + except OSError: + pass # gone: fall through, the read reports it + _sysfs_stranded.pop(path, None) # replaced or gone -> re-read it + out: dict = {} + + def _read(): + try: + with open(path) as f: + out['v'] = f.read().strip() + except (OSError, ValueError): + pass # no such attribute, or not text: unreadable, and that IS a fact + + t = threading.Thread(target=_read, daemon=True) + t.start() + t.join(grace) + # `out` FIRST, not is_alive() alone: a reader can deposit its value and still be alive + # for a moment afterwards, and counting that as a strand memoises a healthy attribute as + # unreadable and spends one of four blindness credits. bounded_open has always checked + # its box for the same reason. + if t.is_alive() and 'v' not in out: + # Count the PATH once, not once per reader. hil_pool_check runs -j4 by default, + # which equals SYSFS_STUCK_MAX, so four threads hitting ONE wedged device used to + # spend the entire blindness budget between them -- latching blind on the single + # wedge the tool was run to find. The strand is real for each thread, but the + # DEVICE is what the cap is about. + # Under the SAME lock as the counter: check-then-act here is a race, and + # hil_pool_check runs a ThreadPoolExecutor of exactly SYSFS_STUCK_MAX workers in + # ONE process, so four threads on one wedged path could each see `first` before any + # of them recorded it -- spending the whole blindness budget on a single device, + # which is what this memo exists to prevent. note_sysfs_strand takes the lock + # itself, so call it after releasing. + with _sysfs_stuck_lock: + first = path not in _sysfs_stranded + if first: + try: + # stat, never the thread's own open(): stat does not call ->show(), so + # it cannot block on the device lock the reader is stuck behind + _sysfs_stranded[path] = os.stat(path).st_ino + except OSError: + _sysfs_stranded[path] = None # unstattable, but still known-stranded + if first: + note_sysfs_strand() + return SYSFS_UNKNOWN + return out.get('v') + + +def note_sysfs_strand() -> None: + """Record ONE stranded sysfs reader. Shared by read_sysfs and bounded_open so both + account against a single counter -- the report caveat keys off it.""" + global _sysfs_stuck, _sysfs_blind_logged + with _sysfs_stuck_lock: + _sysfs_stuck += 1 + announce = sysfs_blind() and not _sysfs_blind_logged + _sysfs_blind_logged = _sysfs_blind_logged or announce + if announce: + # once per process, on stderr: a worker's stdout is compacted into one report + # row, where this would be lost among the test output + print(f'warning: {SYSFS_STUCK_MAX} sysfs reads stranded on a wedged device; ' + f'this process is now blind and answers SYSFS_UNKNOWN for every ' + f'attribute -- its verdicts about device presence are not evidence', + file=sys.stderr, flush=True) + + +# path -> the inode it had when its read stranded. A stranded attribute stays +# stranded until the DEVICE is replaced, and a re-enumeration destroys the kernfs +# node and makes a new one -- so a changed inode is the all-clear. Keyed by path +# alone it would outlive the wedge: a busport does not change when a board comes +# back on the same port, so the HUNG reflash this branch performs would recover a +# board the harness could then never see again. +_sysfs_stranded: dict = {} +# A stranded path whose inode could not be read is stored as None, so a plain .get() cannot +# tell 'known stranded, inode unknown' from 'never seen' -- and treating the first as the +# second re-reads it, stranding another permanent thread and fd every call. Distinct miss +# sentinel, so None keeps its own meaning. +_STRAND_MISS = object() + + +def usb_scan(vid_pid=None, serial=None, vid=None) -> tuple[list, bool]: + """Enumerated USB devices matching the filters, and whether anything is unknown. + + Returns ([{busport, dir, vid, pid, serial}], unknown). `unknown` True means a bounded + read did not answer, so absence is NOT proven -- the same contract as read_sysfs. + + Three rules, one implementation for every caller: + + * Root hubs excluded (glob `*-*`): no DUT is one, and scans including them measured + seconds slower (observation, no mechanism -- the "autosuspend wake" explanation was + wrong; usb_string_attr reads a cached string, sysfs.c:124-139). + * idVendor/idProduct first: lock-free `sysfs_emit` from udev->descriptor + (sysfs.c:688-705), so they rule out nearly every device for free. + * `serial` last and bounded: it is served under the lock a wedged ioctl holds, and a + path that already stranded is never re-read (each strand costs a thread and an fd + for this process's life). + """ + out = [] + unknown = False + for d in glob.glob('/sys/bus/usb/devices/*-*'): + # Interfaces are '<busport>:<cfg>.<ifnum>' (e.g. 2-4:1.0) -- they CONTAIN the + # colon, they do not end with it, so the original endswith() never fired and every + # scan opened idVendor/idProduct on all of them (measured: 31 of 44 matches). + if ':' in os.path.basename(d): + continue + try: + with open(os.path.join(d, 'idVendor')) as f: + dev_vid = f.read().strip() + with open(os.path.join(d, 'idProduct')) as f: + dev_pid = f.read().strip() + except OSError: + continue # vanished mid-walk, or not a device dir: a fact, not unknown + if vid_pid is not None and (dev_vid, dev_pid) != tuple(vid_pid): + continue # ruled out for free, without touching the locked attribute + if vid is not None and dev_vid != vid: + continue # same, for callers that know the VID but not the PID + sn = read_sysfs(os.path.join(d, 'serial')) + if sn is SYSFS_UNKNOWN: + unknown = True # read_sysfs memoises it; a repeat scan costs nothing + continue + if sn is None: + continue # no serial attribute: a fact + if serial is not None and sn.lower() != serial.lower(): + continue + out.append({'busport': os.path.basename(d), 'dir': d, + 'vid': dev_vid, 'pid': dev_pid, 'serial': sn}) + return out, unknown + + +def bounded_open(path: str, flags: int, timeout: float = SYSFS_READ_GRACE): + """os.open() with a wall-clock bound. + + The fd, None when the open genuinely FAILED (OSError: EBUSY, ENOENT, EACCES), or + SYSFS_UNKNOWN when it did not answer -- the same three-valued contract as read_sysfs, + and for the same reason: folding a fact into an unknown made an ordinary EBUSY read as + a wedged device and sent the operator hunting hardware that is healthy. + + An open CAN block on a wedged device -- not on O_NONBLOCK, which usblp_open never + consults, but on usb_autopm_get_interface(), a runtime-PM resume that does I/O + (v6.12.96 drivers/usb/class/usblp.c). It holds usblp_mutex while it waits, and that + mutex is driver-GLOBAL, so one wedged printer blocks opens of every usblp node. + + Unlike read_sysfs the stranded thread cleans up after itself: if we have given up it + closes the fd it eventually got, so only the thread leaks. Both sides take `handoff` + -- "store or close" and "abandon and drain" are a check-then-act pair that can + interleave into an fd stored after the box was drained, which would leak it into a + node that allows a SINGLE opener (usblp_open returns -EBUSY when usblp->used). + """ + # Same short-circuit as read_sysfs: once blind, another stranded thread buys nothing + # and the cap exists precisely to stop them accumulating. + if sysfs_blind(): + return SYSFS_UNKNOWN + # Known-stranded? Re-opening costs another thread, another fd and another blindness + # credit to learn what we already know -- and the printer test re-opens ONE lp node on + # every retry. Same memo and same inode check as read_sysfs. + was = _sysfs_stranded.get(path, _STRAND_MISS) + if was is not _STRAND_MISS: + if was is None: + return SYSFS_UNKNOWN # stranded, inode unknown: never re-read it + try: + if os.stat(path).st_ino == was: + return SYSFS_UNKNOWN + except OSError: + pass + _sysfs_stranded.pop(path, None) + box: dict = {} + done, abandoned = threading.Event(), threading.Event() + handoff = threading.Lock() + + def _open(): + try: + fd = os.open(path, flags) + except OSError: + done.set() + return + with handoff: + stored = not abandoned.is_set() + if stored: + box['fd'] = fd + if not stored: + try: + os.close(fd) + except OSError: + pass + done.set() + + threading.Thread(target=_open, daemon=True).start() + if not done.wait(timeout): + with handoff: + abandoned.set() + fd = box.pop('fd', None) # completed in the gap between timeout and flag + if fd is not None: + # It DID open, just after our deadline -- the thread finished, so nothing is + # stranded. Report unknown (we already gave up on it) but do not spend a + # blindness credit, and do not call a merely-slow node wedged. + try: + os.close(fd) + except OSError: + pass + return SYSFS_UNKNOWN + # counted like a stranded read_sysfs: the thread and (eventually) its fd are gone + # for the life of the process, and the cap exists to stop that reaching the + # thread/fd ceiling -- an exception there escapes the worker and loses every board. + # Memoised by inode so a retry of the same node does not pay again. + # same lock as read_sysfs, same reason + with _sysfs_stuck_lock: + first = path not in _sysfs_stranded + if first: + try: + _sysfs_stranded[path] = os.stat(path).st_ino + except OSError: + _sysfs_stranded[path] = None + if first: + note_sysfs_strand() + return SYSFS_UNKNOWN + return box.get('fd') + + +def _close_pipes(p: subprocess.Popen) -> None: + """Close OUR ends of an abandoned child's pipes. Never raises.""" + for pipe in (p.stdout, p.stderr, p.stdin): + try: + if pipe is not None: + pipe.close() + except OSError: + pass + + +def run_alongside(argv: list, work, timeout: int) -> subprocess.CompletedProcess: + """Run `argv` alongside `work()`, which runs in THIS thread, then reap it -- bounded. + + The read-while-we-write shape run_cmd cannot express: the caller needs the child + RUNNING while it does something else. Everything else about the contract is run_cmd's + -- own session, killpg, bounded reap, our pipe ends closed, rc 124 on the kill. + + A PROCESS, not a thread: an abandoned thread keeps the fd, and usblp_open returns + -EBUSY while usblp->used (v6.12.96 usblp.c), so every later open in this long-lived + worker would read as a wedged device. A killed process takes its fd with it. + + stdout is captured as BYTES and kept CLEAN -- a caller byte-compares it against the + payload it sent, so a single stderr byte (a PYTHONWARNINGS chirp, a sitecustomize + print, a .pth deprecation from a venv) would read as USB data corruption. stderr gets + its own pipe; communicate() drains both, so the split cannot deadlock. + `work` runs even if the child dies immediately -- the caller's own asserts decide. + """ + p = subprocess.Popen(argv, stdout=subprocess.PIPE, stderr=subprocess.PIPE, + start_new_session=True) + + def _reap() -> subprocess.CompletedProcess: + try: + out, err = p.communicate(timeout=timeout) + return subprocess.CompletedProcess(argv, p.returncode, out, err) + except subprocess.TimeoutExpired: + try: + os.killpg(p.pid, signal.SIGKILL) + except OSError: + p.kill() + try: + out, err = p.communicate(timeout=5) + except subprocess.TimeoutExpired: + # Outlasted SIGKILL: uninterruptible, still holding whatever it opened. + # Abandoned like any other stray -- but as a real child in its own + # session, so the containment sweep FINDS it (child_procs walks the ppid + # tree) and the report names it. That is the whole difference from a + # blocked thread, which no sweep can see and no signal can reach. + out, err = b'', b'' + _close_pipes(p) # our own fds must not leak either + return subprocess.CompletedProcess(argv, 124, out, err) + + try: + work() + except BaseException: + # Reap first so the child never outlives us, then let the caller's error through. + # A `return` inside a `finally` would SWALLOW it -- an assert in `work` would + # vanish and the caller would compare data it never finished sending. + _reap() + raise + return _reap() + + +def run_cmd(cmd: str, cwd: str | None = None, timeout: int | None = None, + binary: bool = False, split_stderr: bool = False, + quiet: bool = False) -> subprocess.CompletedProcess: + if timeout is None: + timeout = CMD_TIMEOUT + # binary: raw bytes (text mode's errors='replace' mangles non-UTF-8 file content). + # split_stderr: keep stderr out of stdout, for callers that parse stdout. quiet: no + # COMMAND FAILED banner, for retry loops that report failures themselves (timeouts + # still print: a killed child is always noteworthy). + popen_kwargs = { + 'cwd': cwd, + 'shell': True, + 'stdout': subprocess.PIPE, + 'stderr': subprocess.PIPE if split_stderr else subprocess.STDOUT, + } + if not binary: + popen_kwargs.update({'text': True, 'encoding': 'utf-8', 'errors': 'replace'}) + if os.name != 'nt': + # C-level setsid, same process-group semantics as preexec_fn=os.setsid but + # safe when called from threads (pool_check runs flashes from a thread pool) + popen_kwargs['start_new_session'] = True + + p = subprocess.Popen(cmd, **popen_kwargs) + try: + out, err = p.communicate(timeout=timeout) + r = subprocess.CompletedProcess(args=cmd, returncode=p.returncode, stdout=out, stderr=err) + except subprocess.TimeoutExpired as ex: + if os.name != 'nt': + try: + os.killpg(p.pid, signal.SIGKILL) + except OSError: + # ProcessLookupError: already gone. PermissionError: an all-root group + # refuses the group kill -- letting either escape would skip the bounded + # reap, the pipe close and the rc-124 return this handler exists for. + pass + else: + p.kill() + try: + out, err = p.communicate(timeout=10) + except subprocess.TimeoutExpired: + # Something in the group outlived SIGKILL: D state (truly unkillable), or + # root-owned because sudo FORKS rather than execs, so the wrapper dies and its + # root child does not. Abandon it and let the report name it; the harness never + # sudo-kills its way out. Our ends of its pipes must not leak, though: a pool + # worker lives for the whole run, so every wedged command would cost it two fds. + out, err = None, None + _close_pipes(p) + # prefer the post-kill buffers (supersets of the exception's), falling back to ex.* + # when the child was unkillable. TimeoutExpired carries BYTES even for a text-mode + # Popen, so the fallbacks must be decoded or a text-mode caller gets bytes exactly + # when the child wedged in D state. + def _typed(v): + if not binary and isinstance(v, bytes): + return v.decode('utf-8', errors='replace') + return v + + timeout_out = _typed(out or ex.stdout) or (b'' if binary else '') + # ...and never None: with split_stderr the SUCCESS path always yields a str/bytes, + # so a caller that does `r.stderr.strip()` works everywhere except the timeout -- + # the one path it was written for. Without split_stderr stderr stays None, as on + # the success path (it was merged into stdout). + timeout_err = _typed(err if err is not None else ex.stderr) + if split_stderr and timeout_err is None: + timeout_err = b'' if binary else '' + _print_banner(f'COMMAND TIMEOUT ({timeout}s): {cmd}', timeout_out, timeout_err) + return subprocess.CompletedProcess(args=cmd, returncode=124, stdout=timeout_out, stderr=timeout_err) + except BaseException: + # BaseException, not Exception (as in CPython's own subprocess.run): + # KeyboardInterrupt is the case that matters, and start_new_session put the child in + # its OWN group, so it never got the terminal's SIGINT -- without this, Ctrl-C + # leaves the flasher or testusb holding the probe and its usbfs node. Kill and + # close, never wait: this path must not add a hang of its own. + if os.name != 'nt': + try: + os.killpg(p.pid, signal.SIGKILL) + except OSError: + pass + else: + p.kill() + _close_pipes(p) + raise + + if r.returncode != 0 and not quiet: + _print_banner(f'COMMAND FAILED: {cmd}', r.stdout, r.stderr) + elif verbose: + print(cmd) + print(cmd_stdout_text(r.stdout)) + return r + + +# get usb serial by id +def get_serial_dev(id, vendor_str, product_str, ifnum): + if vendor_str and product_str: + # known vendor and product + vendor_str = vendor_str.replace(' ', '_') + product_str = product_str.replace(' ', '_') + return f'/dev/serial/by-id/usb-{vendor_str}_{product_str}_{id}-if{ifnum:02d}' + else: + # just use id: mostly for cp210x/ftdi flasher + pattern = f'/dev/serial/by-id/usb-*_{id}-if*' + port_list = glob.glob(pattern) + if len(port_list) == 0: + raise RuntimeError(f'No serial device found for {pattern}') + return port_list[0] diff --git a/test/hil/hfp.json b/test/hil/hfp.json index 735d5a402..2babcaaf3 100644 --- a/test/hil/hfp.json +++ b/test/hil/hfp.json @@ -7,9 +7,8 @@ "device": true, "host": false, "dual": false }, "flasher": { - "name": "jlink", - "uid": "774470029", - "args": "-device STM32L412KB" + "name": "stlink", + "uid": "0673FF575051717867034946" } }, { diff --git a/test/hil/hil_ci.sh b/test/hil/hil_ci.sh index 3ec907979..c7dfa95df 100644 --- a/test/hil/hil_ci.sh +++ b/test/hil/hil_ci.sh @@ -20,6 +20,30 @@ CONFIG=${CONFIG:-$ROOT_DIR/test/hil/tinyusb.json} exit 1 } +# REMOTE_DIR reaches the rig as `rm -rf` input, an scp remote path and an rsync remote +# path -- the remote shell re-splits and expands all three, so no amount of LOCAL quoting +# protects them (and %q would escape the ~ that REMOTE_DIR=~/dir needs). Screen it once. +# The tilde is the whole hazard: the REMOTE shell expands it, so `~/` alone -- one typo +# away from the documented ~/dir override -- means `rm -rf` on that account's HOME. Hence +# `/` or `~/` followed by at least one named component, ending in a name character. +[[ $REMOTE_DIR =~ ^(/|~/)[A-Za-z0-9_.~/-]*[A-Za-z0-9_-]$ && $REMOTE_DIR != *..* + && $REMOTE_DIR != *//* ]] || { + echo "error: REMOTE_DIR must be /path or ~/path of [A-Za-z0-9_.~/-], no '..', no" \ + "trailing slash -- it is an rm -rf target on $REMOTE: $REMOTE_DIR" >&2 + exit 1 +} + +# --build would run tools/build.py ON THE RIG, and this script stages binaries, not the +# build tree -- it is not copied, so the run dies there with a confusing missing-file +# error. Building is the local half of this workflow by design. +for a in "$@"; do + [ "$a" = "--build" ] || continue + echo "error: --build builds on the REMOTE, but this script copies prebuilt binaries" >&2 + echo " (tools/build.py is not staged). Build locally first, then re-run:" >&2 + echo " cd examples && cmake --preset <board> && cmake --build --preset <board>" >&2 + exit 1 +done + # Parse -b BOARD from arguments to know which build to copy BOARD="" ARGS=() @@ -38,21 +62,35 @@ while [[ $# -gt 0 ]]; do esac done -# Setup remote directory. Use `bash -s` + heredoc so REMOTE_DIR (user-overridable) -# is passed as a positional parameter and never reinterpreted by the remote shell. +# Setup remote directory. `bash -s` + heredoc so REMOTE_DIR arrives as a positional +# parameter, keeping the `rm -rf` target out of the command string the heredoc runs. echo "==> Setting up remote $REMOTE:$REMOTE_DIR" ssh "$REMOTE" bash -s -- "$REMOTE_DIR" <<'REMOTE' set -e +# Second gate, on the side that knows what ~ expanded to: only here is $HOME a value +# rather than a guess, and this is the line that actually runs rm -rf. +case "$1" in + ''|/|"$HOME"|"$HOME"/) echo "refusing to rm -rf '$1'" >&2; exit 1 ;; +esac rm -rf -- "$1" -mkdir -p -- "$1/test/hil" "$1/examples" +mkdir -p -- "$1/test/hil/helper" "$1/examples" REMOTE # Copy HIL test script and config echo "==> Copying test scripts" scp -q "$ROOT_DIR/test/hil/hil_test.py" \ + "$ROOT_DIR/test/hil/hil_flash.py" \ + "$ROOT_DIR/test/hil/usbtest.py" \ "$ROOT_DIR/test/hil/pymtp.py" \ + "$ROOT_DIR/test/hil/mtp_test.py" \ "$CONFIG" \ "$REMOTE:$REMOTE_DIR/test/hil/" +scp -q "$ROOT_DIR/test/hil/helper/__init__.py" \ + "$ROOT_DIR/test/hil/helper/hil_util.py" \ + "$ROOT_DIR/test/hil/helper/hil_health.py" \ + "$ROOT_DIR/test/hil/helper/hil_lock.py" \ + "$ROOT_DIR/test/hil/helper/hil_select.py" \ + "$REMOTE:$REMOTE_DIR/test/hil/helper/" # Copy only firmware binaries (elf/bin/hex) plus esptool metadata # (config.env + flash_args needed by the esptool flasher), preserving structure @@ -82,16 +120,25 @@ if [ -n "$BOARD" ]; then add_build_dir "$d" done shopt -u nullglob - while IFS= read -r v; do - add_build_dir "$ROOT_DIR/examples/cmake-build-$v" - done < <(python3 -c ' + # to a file, not a process substitution: `set -e`/pipefail cannot see the exit + # status of the latter, so a malformed roster silently yielded zero variant dirs + VARIANTS_FILE=$(mktemp) + python3 -c ' import json, sys cfg = json.load(open(sys.argv[1])) for b in cfg.get("boards", []): if b["name"] == sys.argv[2]: for v in b.get("variant") or []: print(v["name"]) -' "$CONFIG" "$BOARD") +' "$CONFIG" "$BOARD" > "$VARIANTS_FILE" || { + echo "Error: could not read variants for $BOARD from $CONFIG" + rm -f "$VARIANTS_FILE" + exit 1 + } + while IFS= read -r v; do + add_build_dir "$ROOT_DIR/examples/cmake-build-$v" + done < "$VARIANTS_FILE" + rm -f "$VARIANTS_FILE" if [ ${#BUILD_DIRS[@]} -eq 0 ]; then echo "Error: no build directory found for $BOARD under $ROOT_DIR/examples/" echo "Build first with: cd examples && cmake --preset $BOARD && cmake --build --preset $BOARD" @@ -111,12 +158,24 @@ else done fi -# Run test. Use `bash -s` so REMOTE_DIR + ARGS reach the remote shell as positional -# parameters; quoting and metacharacters in args are preserved. -CONFIG_BASENAME="$(basename "$CONFIG")" +# Run test via `bash -s`, so REMOTE_DIR and the args arrive as positional parameters. +# %q the ARGS -- ssh joins its argv into ONE string that the remote shell re-splits, so +# `-t 'host/cdc msc'` would arrive as two arguments and hil_test.py would see a stray +# word where it expects the config path. REMOTE_DIR is deliberately NOT quoted here: it +# is screened above precisely so it can keep its ~ expansion. +ARGS_Q=() +for a in ${ARGS[@]+"${ARGS[@]}"}; do ARGS_Q+=("$(printf '%q' "$a")"); done +# same re-split, same fix: CONFIG is a user-supplied path and its basename lands in the +# command string too +CONFIG_Q="$(printf '%q' "test/hil/$(basename "$CONFIG")")" echo "==> Running HIL test on $REMOTE" rc=0 -ssh "$REMOTE" bash -s -- "$REMOTE_DIR" "${ARGS[@]}" "test/hil/$CONFIG_BASENAME" <<'REMOTE' || rc=$? +# --retry 1 FIRST, before the user's args: this targets the same shared rig CI uses, and +# the pool guard is a flat constant that does not scale with max_retry -- argparse's +# default of 3 lets a few flaky boards re-pay 510s each until the 3600s guard fires, +# abandoning the pool and holding board flocks against concurrent CI. Placed first, not +# appended, so argparse's last-wins means `hil_ci.sh -r 3` still gets 3. +ssh "$REMOTE" bash -s -- "$REMOTE_DIR" --retry 1 ${ARGS_Q[@]+"${ARGS_Q[@]}"} "$CONFIG_Q" <<'REMOTE' || rc=$? cd -- "$1" shift # Flasher CLIs live in the user bin dirs on ci.lan (esptool/idf in ~/.local/bin, diff --git a/test/hil/hil_flash.py b/test/hil/hil_flash.py new file mode 100755 index 000000000..f4bed45a6 --- /dev/null +++ b/test/hil/hil_flash.py @@ -0,0 +1,332 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +# Firmware flashing for the TinyUSB HIL rig: one flash_*/reset_* pair per flasher type +# (dispatched by config name via getattr) plus find_firmware. The bounded runner run_cmd +# lives in hil_util (never import hil_test here). Callers set the module global +# `build_dir`. `from __future__ import annotations` keeps the Board hints below +# unevaluated: the type is not defined in this module. + +from __future__ import annotations + +import json +import re +import subprocess +from pathlib import Path + +import os +import sys + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) # PYTHONSAFEPATH drops it +from helper import hil_util + +build_dir = 'cmake-build' + +# flasher names (dispatch key, board['flasher']['name'].lower()) whose reset_* is a no-op +RESET_NOOP = {'esptool', 'lm4flash'} + +# extra parents find_firmware ALSO searches after build_dir. Empty by default so +# hil_test's -B stays authoritative: a board missing there must report "Skip (no +# binary)", never silently flash a stale binary from another tree. +EXTRA_BUILD_DIRS: list = [] + +_VID_PID_WARNED: set = set() # one warning per probe, not per command + + +# ------------------------------------------------------------- +# Flashing firmware +# ------------------------------------------------------------- +def flash_jlink(board: Board, firmware: str, timeout=None) -> subprocess.CompletedProcess: + flasher = board['flasher'] + script = ['halt', 'r', f'loadfile {firmware}', 'r', 'go', 'exit'] + f_jlink = Path(f'{board["name"]}_{Path(firmware).name}.jlink') + with f_jlink.open('w') as f: + f.writelines(f'{s}\n' for s in script) + ret = hil_util.run_cmd(f'JLinkExe -USB {flasher["uid"]} {flasher["args"]} -if swd -JTAGConf -1,-1 -speed auto -NoGui 1 -ExitOnError 1 -CommandFile {f_jlink}', + timeout=timeout) + f_jlink.unlink(missing_ok=True) + return ret + + +def reset_jlink(board: Board) -> subprocess.CompletedProcess: + flasher = board['flasher'] + script = ['halt', 'r', 'go', 'exit'] + f_jlink = Path(f'{board["name"]}_reset.jlink') + if not f_jlink.exists(): + with f_jlink.open('w') as f: + f.writelines(f'{s}\n' for s in script) + ret = hil_util.run_cmd(f'JLinkExe -USB {flasher["uid"]} {flasher["args"]} -if swd -JTAGConf -1,-1 -speed auto -NoGui 1 -ExitOnError 1 -CommandFile {f_jlink}') + return ret + + +def flash_stlink(board, firmware, timeout=None): + # --verify catches the partial/corrupt write that exits 0 and sends the test phase + # off to exercise bad firmware. Opt-IN here ("verify": true), unlike flash_openocd's + # opt-out: a default-on read-back silently changes every roster entry that lacks the + # key, including boards on rigs this was never validated against. + flasher = board['flasher'] + verify = ' --verify' if flasher.get('verify', False) else '' + return hil_util.run_cmd(f'STM32_Programmer_CLI --connect port=swd sn={flasher["uid"]} --write {firmware}{verify} --go', + timeout=timeout) + + +def reset_stlink(board): + flasher = board['flasher'] + return hil_util.run_cmd(f'STM32_Programmer_CLI --connect port=swd sn={flasher["uid"]} --rst --go') + + +def _openocd_cmd_base(flasher): + # Optional roster field vid_pid, openocd-verbatim (e.g. "0x1a86 0x8010"), pins probe + # discovery to the probe's IDs so openocd never opens foreign usbfs nodes to read + # strings -- a wedged node makes that open hang unkillably (the 2026-08-10 convoy). + # BEFORE args, because the rescue cfgs run `init` internally and reject (or never see) + # a config command that follows it. + vid_pid = '' + if 'vid_pid' in flasher: + # Validated HERE too, not just in convoy_safe: openocd only warns ("incomplete + # vid_pid configuration directive") and exits 0 on a malformed value, so the pin + # silently does not apply and discovery goes back to opening every usbfs node -- + # the convoy this field exists to stop. The same key name carries a DIFFERENT + # syntax under tests.dev_attached ('1a86_55d4'), so the typo is one copy away. + if valid_vid_pid(flasher['vid_pid']): + vid_pid = f'-c "adapter usb vid_pid {flasher["vid_pid"]}" ' + else: + # stderr + once-per-probe, like the missing-pin branch below: stdout here is + # captured by test_example's redirect_stdout (shown only when the test FAILS) + # and by hil_pool_check's StringIO spool, so on a PASSING run the operator + # would never learn the pin was silently dropped. + uid = flasher.get('uid', '?') + if uid not in _VID_PID_WARNED: + _VID_PID_WARNED.add(uid) + print(f'warning: {uid} has a malformed vid_pid {flasher["vid_pid"]!r} ' + f'(want "0xVVVV 0xPPPP"); probe pin DROPPED, so discovery will open ' + f'foreign usbfs nodes', file=sys.stderr, flush=True) + elif flasher.get('uid') not in _VID_PID_WARNED: + # stderr, once per probe: test_example captures stdout, so a passing run would + # swallow this and the operator would never learn discovery still opens every + # usbfs node + _VID_PID_WARNED.add(flasher.get('uid')) + print(f'warning: openocd flasher {flasher.get("uid", "?")} has no vid_pid pin; ' + f'probe discovery will open every usbfs node (hangs on a wedged one)', + file=sys.stderr, flush=True) + return (f'openocd -c "tcl_port disabled" -c "gdb_port disabled" -c "telnet_port disabled" ' + f'-c "adapter serial {flasher["uid"]}" {vid_pid}{flasher["args"]}') + + +# `verify` is on by default, opted out per board with "verify": false. WCH targets must +# opt out: read-back over the WCH-Link sdi transport returns a repeated word instead of +# memory contents, so verification always mismatches (measured on ch32v103r and ch32v307v, +# 2026-07-30). Do NOT drop verify fleet-wide for them — every other openocd board reads +# back, and without it a partial or corrupt write exits 0 and the tests run bad firmware. +def flash_openocd(board, firmware, timeout=None): + flasher = board['flasher'] + verify = ' verify' if flasher.get('verify', True) else '' + ret = hil_util.run_cmd(f'{_openocd_cmd_base(flasher)} -c "program {firmware}{verify} reset exit"', + timeout=timeout) + return ret + + +def reset_openocd(board, timeout=None): + # timeout: usbtest's post-hang recovery bounds this (RECOVER_RESET_TIMEOUT); an + # unbounded reset there would outlive the caller's outer kill and orphan openocd on + # the probe, which is the stray the recovery exists to avoid. + flasher = board['flasher'] + ret = hil_util.run_cmd(f'{_openocd_cmd_base(flasher)} -c "init; reset run; exit"', + timeout=timeout) + return ret + + +# OpenOCD's messages for "the target's debug port did not answer". The probe is fine when +# these appear ("CMSIS-DAP: Interface ready" is still logged); the chip's debug clock is +# gone, which no probe-driven reset fixes -- the CMSIS-DAP probe has no nRESET line. Which +# message appears depends on DAP topology, not the board, so both are accepted for both +# chips; RESCUE_CFG below picks the rescue. +DAP_WEDGED = ('Failed to connect multidrop', 'Error connecting DP: cannot read IDR') + +# How each RP target reaches its Rescue DP, keyed by the target cfg named in flasher args: +# (cfg substitution, pre args, post args). rp2040.cfg drives the Rescue DP behind a RESCUE +# flag and init/shutdowns itself; rp2350-rescue.cfg never shuts down, so it needs an +# explicit one or it sits in the server loop until CMD_TIMEOUT. +RESCUE_CFG = { + 'target/rp2040.cfg': ('target/rp2040.cfg', '-c "set RESCUE 1" ', ''), + 'target/rp2350.cfg': ('target/rp2350-rescue.cfg', '', ' -c "shutdown"'), +} + + +def rescue_openocd(board, flash_out: str = '', timeout=None) -> bool: + """Power-on-reset a wedged RP2040/RP2350 through its Rescue DP, the one debug port not + gated by the system clock (RP2040 datasheet 2.3.4.2): CDBGPWRUPREQ hard-resets the + chip and the bootrom halts it ready to be flashed. Without it the board needs a + physical replug -- the probe carries no reset line. + + No-op (False) unless this is an openocd RP board AND the flash output shows the wedge, + so a flash that failed for any other reason still just retries. True when a rescue was + attempted; the caller should retry the flash afterwards.""" + flasher = board['flasher'] + if flasher['name'].lower() != 'openocd' or not any(m in flash_out for m in DAP_WEDGED): + return False + for cfg, (rescue_cfg, pre, post) in RESCUE_CFG.items(): + if cfg in flasher['args']: + args = flasher['args'].replace(cfg, rescue_cfg) + return hil_util.run_cmd(f'{_openocd_cmd_base({**flasher, "args": pre + args})}{post}', + timeout=timeout).returncode == 0 + return False + + +# openocd's own syntax: one or more "0xVVVV 0xPPPP" pairs. Validated rather than merely +# tested for truthiness -- `vid_pid` is a hand-edited roster field whose NAME is also used, +# with a different syntax, by tests.dev_attached, and convoy_safe reads a non-empty value +# as PROOF the flasher can deliver a recovery past a poisoned node. A typo there silently +# promised a recovery that openocd would reject at startup. +_VID_PID_RE = re.compile(r'^0x[0-9a-fA-F]{4}(\s+0x[0-9a-fA-F]{4})+$') + + +def valid_vid_pid(value) -> bool: + return isinstance(value, str) and bool(_VID_PID_RE.match(value.strip())) + + +def recover_flasher(board: dict) -> dict: + """The flasher that delivers RECOVERY for this board. + + Optional roster key `flasher_recover`, else the primary. It exists because delivery and + normal flashing have different requirements: a board flashed by jlink/stlink/lm4flash + cannot reach its probe past a poisoned usbfs node, but the same probe driven by openocd + often can (see convoy_safe). Keeping it a separate key rather than a list means the + primary's shape never changes, so nothing that reads board['flasher'] has to care. + """ + return board.get('flasher_recover') or board['flasher'] + + +def convoy_safe(flasher: dict) -> bool: + """Can this flasher DELIVER a recovery while a usbfs node on the rig is poisoned? + + A post-HUNG reflash only helps if the flasher reaches its probe without opening the + wedged node. Two shapes qualify: + + * openocd pinned with the roster's `vid_pid` -- the match is made from the cached + descriptor and the loop `continue`s BEFORE libusb_open, so a foreign node is never + opened. On 2026-08-12 it was the only flasher that still reached its probe. + * esptool -- delivery is `-p <ttyACM>`, a named port; it never enumerates usbfs. + + Everything else enumerates by OPENING nodes, would block in D state on the poisoned + one, survive SIGKILL and become a second stray. JLinkExe cannot be pinned: selection + is serial-only (-USB/-SelectEmuBySN) and reading a serial requires the open (J-Link + Commander V9.66 exposes no VID/PID filter), so those boards can only become + convoy-safe by moving to openocd. + + Verified against openocd 0ce743125 (the rig's build), because the INVERSE is what + bites: cmsis_dap_usb_bulk.c:107 skips on `id_filter && !id_match`, and `id_filter` is + only `vids[0] || pids[0]` -- so without the pin nothing is skipped and every device on + the bus is opened, which the code itself expects to mostly fail. Enumeration cannot + block: libusb reads the `descriptors` sysfs attribute, and descriptors_read (v6.12.101 + drivers/usb/core/sysfs.c) is a memcpy from udev->rawdescriptors under no lock. + + The pin gates the BULK backend, which is the one that runs: `auto` tries usb_bulk -> + hid -> tcp (cmsis_dap.c:62) and stops at the first that opens, so a CMSIS-DAP v2 probe + never reaches the rest. It does NOT cover the HID fallback that a v1 probe or a failed + bulk open takes -- cmsis_dap_usb_hid.c:91 calls hid_enumerate(0x0, 0x0), pin ignored, + and filters afterwards, while hidapi's hidraw backend reads `manufacturer` and + `product` for every HID device it lists (linux/hid.c:744), both usb_string_attr and so + served under the device lock. A wedged DUT running hid_generic_inout, + hid_boot_interface or hid_composite_freertos is a HID device and would stall that walk + -- interruptibly, so it hangs rather than joining the D-state convoy and run_cmd's + timeout ends it, but "never opens a foreign node" is true of the bulk path, not of + every path openocd can take. + """ + name = (flasher.get('name') or '').lower() + if name == 'esptool': + return True + # EXACT, not startswith: rescue_openocd and usbtest's + # getattr(hil_flash, f'flash_{name}') both require the exact name, so an + # 'openocd_wch'-style entry would pass this gate, reserve USBTEST_RECOVERY_BUDGET, + # and then find no recovery path at all -- paying for a path that cannot fire, which + # is the precise cost this gate exists to avoid. + if name != 'openocd': + return False + if valid_vid_pid(flasher.get('vid_pid')): + return True + # openocd over the JLINK driver is safe WITHOUT a pin, and cannot use one: jlink.c + # never reads adapter_usb_get_vids/pids (selection is adapter serial / usb address / + # usb location), but libjaylink's discovery returns early unless idVendor == 0x1366 and + # the PID is in its table, and only THEN calls libusb_open (discovery_usb.c). So it + # never opens a foreign node -- which is exactly what JLinkExe, SEGGER's own tool, + # does do. Verified against openocd 0ce743125 and libjaylink master. + return 'interface/jlink.cfg' in (flasher.get('args') or '') + + +def flash_esptool(board: Board, firmware: str, timeout=None) -> subprocess.CompletedProcess: + flasher = board['flasher'] + port = hil_util.get_serial_dev(flasher["uid"], None, None, 0) + fw_dir = Path(firmware).parent + with (fw_dir / 'config.env').open() as f: + idf_target = json.load(f)['IDF_TARGET'] + with (fw_dir / 'flash_args').open() as f: + flash_args = f.read().strip().replace('\n', ' ') + command = (f'esptool --chip {idf_target} -p {port} {flasher["args"]} ' + f'--before=default_reset --after=hard_reset write_flash {flash_args}') + ret = hil_util.run_cmd(command, cwd=str(fw_dir), timeout=timeout) + return ret + + +def reset_esptool(board): + # NO-OP, and marked as one: esptool's reset would be `--after hard_reset`, which is not + # wired here. Returning rc 0 without resetting is why callers must never read the exit + # code as proof -- recovery_steps skips a primitive carrying `no_op`. + return subprocess.CompletedProcess(args=['dummy'], returncode=0) + + +reset_esptool.no_op = True + + +def flash_lm4flash(board, firmware, timeout=None): + # TI Tiva-C / Stellaris ICDI: lightweight lm4flash, resets and runs after write + flasher = board['flasher'] + ret = hil_util.run_cmd(f'lm4flash -s {flasher["uid"]} {flasher["args"]} {firmware}', + timeout=timeout) + return ret + + +def reset_lm4flash(board): + # lm4flash has no reset-only mode; it resets+runs on flash, so reset is a no-op + return subprocess.CompletedProcess(args=['dummy'], returncode=0) + + +reset_lm4flash.no_op = True + + +# The one place a flasher's firmware extension is decided. A flasher with no entry falls +# back to .elf-or-.bin and can be handed the wrong file — test_hil_select's +# TestRosterFlashersDispatch fails if a roster names one. +FLASHER_SUFFIX = { + 'esptool': '.bin', + 'jlink': '.elf', + 'lm4flash': '.bin', + 'openocd': '.elf', + 'stlink': '.elf', +} + + +def find_firmware(variant: str, example: str, roots: list | None = None, flasher: str | None = None): + """Locate a built example's firmware under <build_dir>/cmake-build-<variant>/<example>/, + then under EXTRA_BUILD_DIRS. `roots` overrides that search list entirely for one call + (e.g. a build just produced by tools/build.py in its fixed cmake-build/ layout) + without widening the global policy. `flasher` is the roster flasher name and selects + which extension counts (FLASHER_SUFFIX), so a build that produced only the other one + is reported missing — a clean "Skip (no binary)" — instead of being handed to the + flasher, which would fail opaquely and burn every retry plus the board lock. + Accepts the single-config layout (firmware directly in the example dir) or Ninja + Multi-Config (a per-config subdir like RelWithDebInfo/). + Returns the full Path INCLUDING extension, or None if not built.""" + base = Path(example).name + suffixes = [FLASHER_SUFFIX.get(flasher.lower())] if flasher else [] + if not suffixes or suffixes == [None]: + suffixes = ['.elf', '.bin'] + for bd in dict.fromkeys(roots if roots is not None else [build_dir, *EXTRA_BUILD_DIRS]): + fw_dir = hil_util.TINYUSB_ROOT / bd / f'cmake-build-{variant}' / example + if not fw_dir.is_dir(): + continue + for cand in [fw_dir / base, fw_dir / 'RelWithDebInfo' / base, + *(p.with_suffix('') for s in suffixes for p in sorted(fw_dir.glob(f'*/{base}{s}')))]: + for s in suffixes: + if cand.with_suffix(s).exists(): + return cand.with_suffix(s) + return None diff --git a/test/hil/hil_test.py b/test/hil/hil_test.py index 4b87154b6..174251343 100755 --- a/test/hil/hil_test.py +++ b/test/hil/hil_test.py @@ -23,11 +23,15 @@ # THE SOFTWARE. # Host setup (required: a missing tool fails its test rather than skipping it): -# - System packages: sudo apt install mtools libmtp9 alsa-utils iperf -# mtools - read_disk_file (device/cdc_msc, device/msc_dual_lun) -# libmtp9 - pymtp ctypes load (device/mtp); Debian 13 uses libmtp9t64 -# alsa-utils - arecord (device/audio_test_freertos) -# iperf - throughput tests (device/net_lwip_*) +# - System packages: sudo apt install mtools libmtp9 libmtp-runtime alsa-utils iperf +# mtools read_disk_file (device/cdc_msc, device/msc_dual_lun) +# libmtp9 pymtp ctypes load (device/mtp); Debian 13 uses libmtp9t64 +# libmtp-runtime mtp-probe and the completed-device /dev/libmtp-* marker +# alsa-utils arecord (device/audio_test_freertos) +# iperf throughput tests (device/net_lwip_*) +# openocd unified openocd from https://github.com/hathach/openocd (branch tinyusb) for wch, rp2040/rp2350, analog max32 +# - device/usbtest: usbtest kernel module + testusb binary (kernel tools/usb/testusb.c) on PATH, +# plus sudo for modprobe / sysfs writes # - Python packages: pip install -r requirements.txt # # udev rules : @@ -36,97 +40,114 @@ import argparse import io +import itertools import os import random import re import select +import signal +import shlex import sys +import tempfile import time -import signal from contextlib import redirect_stdout from pathlib import Path -from typing import Any, TypedDict, NotRequired, cast +from typing import TypedDict, NotRequired, cast import serial import subprocess +import traceback import json import glob -from multiprocessing import Pool, Lock +import multiprocessing from multiprocessing import TimeoutError as MpTimeoutError -import hashlib -import ctypes -from pymtp import MTP + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) # PYTHONSAFEPATH drops it +import hil_flash +from helper import hil_health, hil_lock, hil_util +from helper.hil_util import device_tests, dual_tests, host_test + +# Raw Lock/Semaphore objects in Pool initargs are inheritable only under fork +# (spawn/forkserver pickle them and fail at Pool creation), so pin it against an +# interpreter default change. Windows has no fork: fall back so it still IMPORTS there. + +_mp = multiprocessing.get_context('fork') if os.name != 'nt' else multiprocessing.get_context() +Pool, Lock, Semaphore, Manager = _mp.Pool, _mp.Lock, _mp.Semaphore, _mp.Manager import string -# --- per-board dev-session locks (see test/hil/board_lock.py) ------------ -BOARD_LOCK_DIR = '/tmp/tinyusb-hil-locks' +# Enumeration wait budget: first attempt ENUM_TIMEOUT, retries the shorter +# ENUM_TIMEOUT_RETRY -- a device that will enumerate shows up within seconds, so a failing +# test costs ~3-5x a passing one instead of 10-30x. Set per attempt by test_example(); a +# module global is safe because each pool worker is its own process. +ENUM_TIMEOUT = 8 +ENUM_TIMEOUT_RETRY = 4 +_enum_timeout = ENUM_TIMEOUT -def acquire_board_lock(board_name): - """Take this board's flock for the duration of its flash+test. - Returns an open file handle (keep it referenced; closing releases it), - or None when HIL_NO_BOARD_LOCK=1 or the lock dir is unusable (fail-open: - locking must never break a test run by itself). - Raises RuntimeError only when another session holds the board.""" - import fcntl - if os.environ.get('HIL_NO_BOARD_LOCK') == '1': - return None # user-authorized bypass — see board_lock.py / hil skill - try: - os.makedirs(BOARD_LOCK_DIR, exist_ok=True) - fd = os.open(os.path.join(BOARD_LOCK_DIR, f'{board_name}.lock'), - os.O_RDWR | os.O_CREAT, 0o666) - fh = os.fdopen(fd, 'r+') - except OSError as e: - # odd lock dir (perms, path collision): proceed unlocked, but say so — - # a silent fail-open is indistinguishable from the intentional bypass - print(f'warning: board lock unavailable for {board_name} ({e}); proceeding unlocked', - flush=True) - return None - try: - fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB) - except OSError: - try: - info = fh.read(500).strip() - except (OSError, UnicodeDecodeError): - info = '' - fh.close() - raise RuntimeError(f'board locked: {info or "unknown holder"}') - # announce ourselves so the other side's conflict message is truthful; - # best-effort — the flock itself is already held - try: - fh.truncate(0) - fh.seek(0) - json.dump({'pid': os.getpid(), 'reason': 'hil_test.py', - 'since': time.strftime('%Y-%m-%dT%H:%M:%S%z')}, fh) - fh.flush() - except OSError: - pass - return fh + +def enum_timeout() -> int: + """Enumeration wait budget for the current test attempt.""" + return _enum_timeout -ENUM_TIMEOUT = 15 +def wait_until(predicate, step: float = 1.0, timeout: float | None = None): + """Poll predicate under the per-attempt enum budget. Deadline-based so a slow predicate + body (subprocess, libmtp scan) counts against the budget. An explicit timeout overrides + that budget. Returns the first truthy predicate value, or None on timeout.""" + deadline = time.monotonic() + (enum_timeout() if timeout is None else timeout) + while True: + r = predicate() + if r: + return r + if time.monotonic() >= deadline: + return None + time.sleep(step) STATUS_OK = "\033[32mOK\033[0m" STATUS_FAILED = "\033[31mFailed\033[0m" STATUS_SKIPPED = "\033[33mSkipped\033[0m" -# Plain (non-ANSI) cell symbols for the markdown matrix report (hil_report.md). -# A missing binary is reported as skipped too. +# Plain (non-ANSI) cell symbols for hil_report.md; a missing binary counts as skipped. REPORT_CELL = {'pass': '✅', 'fail': '❌', 'skip': '⚪'} + +class TestFail(AssertionError): + """Fail a test but still surface a metric string in its report cell (e.g. usbtest's '❌ 29/30' + instead of a bare ❌). The cell metric is icon-prefixed so render/tally treat it as a failure.""" + def __init__(self, msg: str, metric: str | None = None, parsed: bool = False): + super().__init__(msg) + self.metric = metric + # parsed=True: a real per-case verdict, so a retry would only re-observe it + # (test_example skips the rest). A failure to RUN the tool stays retryable. + self.parsed = parsed + + verbose = False +# Set when a HUNG usbtest case could not be recovered: the DUT's usbfs node still has a +# D-state holder, so every later flash on that board enumerates into it, blocks, survives +# SIGKILL and becomes another stray. maxtasksperchild=1 gives each board its own worker, +# so this global is board-scoped; test_board resets it anyway. +board_wedged = '' +max_retry = 1 # mirrors argparse's -r default (see main); defined HERE too so + # test_example is callable (and testable) without going through main() +PROFILE = os.environ.get('HIL_PROFILE') == '1' # timestamped logs + permit/flash timing + ctrl-map dump test_only = [] board_test = {} -build_dir = 'cmake-build' skip_flash = False print_lock = None +shuffle_seed = None # per-run seed for the per-board test-order shuffle (HIL_SHUFFLE_SEED to replay) +_current_fw = None # firmware test_example resolved for the RUNNING test (set before each test fn) -def init_worker(lock): - global print_lock +def init_worker(lock, seed, b_mutexes, f_sems, cmap, cmeta, hints_by_uid): + global print_lock, shuffle_seed print_lock = lock + shuffle_seed = seed + hil_lock.init_scheduling(b_mutexes, f_sems, cmap, cmeta, hints_by_uid, log_fn=log_line) def log_line(msg: str) -> None: + if PROFILE: + msg = f'{time.time():.3f} {msg}' out = sys.__stdout__ if sys.__stdout__ is not None else sys.stdout if print_lock is not None: with print_lock: @@ -138,13 +159,21 @@ def log_line(msg: str) -> None: def compact_output(raw: str) -> str: if not raw: return '' - lines = [ln.strip() for ln in raw.replace('\r', '\n').split('\n') if ln.strip()] + # Defense in depth (the emitter already suppresses them, see _ci_log_groups): markers + # piped into this capture land mid-row, where GitHub renders them literally. + lines = [] + for ln in raw.replace('\r', '\n').split('\n'): + ln = hil_util.strip_workflow_markers(ln.strip()).strip() + if ln: + lines.append(ln) return ' | '.join(lines) class FlasherCfg(TypedDict): name: str uid: str - args: str + args: NotRequired[str] # stlink entries carry no args + vid_pid: NotRequired[str] # openocd probe pin, verbatim (e.g. "0x2e8a 0x000c") + verify: NotRequired[bool] # openocd read-back verify opt-out (WCH) class AttachedDevCfg(TypedDict, total=False): @@ -188,67 +217,41 @@ class Board(TypedDict): class HilConfig(TypedDict): boards: list[Board] -CMD_TIMEOUT = int(os.getenv('HIL_CMD_TIMEOUT', '180')) -POOL_TIMEOUT = int(os.getenv('HIL_POOL_TIMEOUT', '3000')) -SERIAL_READ_TIMEOUT = float(os.getenv('HIL_SERIAL_READ_TIMEOUT', '5')) -SERIAL_WRITE_TIMEOUT = float(os.getenv('HIL_SERIAL_WRITE_TIMEOUT', '10')) +# Below the CI job ceilings so THIS guard fires first and still writes a report, well +# above a healthy fleet run (~14 min measured), and deliberately generous: firing early +# abandons boards that were still in flight (30 min fired on 5 of the last 8 HIL jobs), +# while firing late costs minutes on an already-wedged run. The drain keeps whatever had +# already finished either way. +POOL_TIMEOUT = hil_util.pos_int_env('HIL_POOL_TIMEOUT', 3600) -def cmd_stdout_text(out: Any) -> str: - if out is None: - return '' - if isinstance(out, bytes): - return out.decode('utf-8', errors='ignore') - return str(out) +# Headroom on top of a battery's own budget so ONE HUNG recovery (case timeout, SIGKILL +# wait, bounded reflash, settle) can finish. Only spent when cases actually time out. +USBTEST_RECOVERY_BUDGET = hil_util.pos_int_env('HIL_USBTEST_RECOVERY_BUDGET', 250) +# How long usbtest.py may keep starting new cases (--budget). The outer run_cmd timeout is +# always this PLUS the recovery headroom, never a separate literal, or lowering one eats +# the reserve the recovery needs. 0 is refused (usbtest.py reads it as "no limit"); the +# margin over a healthy battery (~200s) keeps contention from becoming BUDGET entries. +USBTEST_BATTERY_BUDGET = hil_util.pos_int_env('HIL_USBTEST_BATTERY_BUDGET', 260) -WCH_RISCV_CONTENT = """ -adapter driver wlinke -adapter speed 6000 -transport select sdi +# The battery checks its budget BEFORE dispatching a case, so it can overshoot by one +# already-started case. Our outer kill must sit ABOVE that or we SIGKILL the battery just +# as it goes to print its JSON, turning ~29 real per-case verdicts into "usbtest did not +# run" and re-paying the whole battery on retry. +# Worst case, from usbtest.py: --timeout 60 (the case) + 5s post-SIGKILL reap + +# dmesg_tail(), which is bounded by HELPER_TIMEOUT=30 and runs on BOTH the FAIL and HUNG +# timeout paths = 95s. 120 leaves a margin; 75 (my first estimate, taken before checking +# dmesg_tail) was 20s SHORT and would have killed the battery mid-print. +USBTEST_OVERSHOOT = 120 +SERIAL_READ_TIMEOUT = hil_util.pos_float_env('HIL_SERIAL_READ_TIMEOUT', 5) +SERIAL_WRITE_TIMEOUT = hil_util.pos_float_env('HIL_SERIAL_WRITE_TIMEOUT', 10) -wlink_set_address 0x00000000 -set _CHIPNAME wch_riscv -sdi newtap $_CHIPNAME cpu -irlen 5 -expected-id 0x00001 - -set _TARGETNAME $_CHIPNAME.cpu - -target create $_TARGETNAME.0 wch_riscv -chain-position $_TARGETNAME -$_TARGETNAME.0 configure -work-area-phys 0x20000000 -work-area-size 10000 -work-area-backup 1 -set _FLASHNAME $_CHIPNAME.flash - -flash bank $_FLASHNAME wch_riscv 0x00000000 0 0 0 $_TARGETNAME.0 - -echo "Ready for Remote Connections" -""" MSC_README_TXT = \ b"This is tinyusb's MassStorage Class demo.\r\n\r\n\ If you find any bugs or get any questions, feel free to file an\r\n\ issue at github.com/hathach/tinyusb" -# ------------------------------------------------------------- -# Path -# ------------------------------------------------------------- -OPENCOD_ADI_PATH = Path.home() / 'app' / 'openocd_adi' -TINYUSB_ROOT = Path(__file__).resolve().parents[2] - -# get usb serial by id -def get_serial_dev(id, vendor_str, product_str, ifnum): - if vendor_str and product_str: - # known vendor and product - vendor_str = vendor_str.replace(' ', '_') - product_str = product_str.replace(' ', '_') - return f'/dev/serial/by-id/usb-{vendor_str}_{product_str}_{id}-if{ifnum:02d}' - else: - # just use id: mostly for cp210x/ftdi flasher - pattern = f'/dev/serial/by-id/usb-*_{id}-if*' - port_list = glob.glob(pattern) - if len(port_list) == 0: - raise RuntimeError(f'No serial device found for {pattern}') - return port_list[0] - - -# get usb disk by id def get_disk_dev(id, vendor_str, lun): return f'/dev/disk/by-id/usb-{vendor_str}_Mass_Storage_{id}-0:{lun}' @@ -271,13 +274,12 @@ def get_alsa_capture_dev(id): def open_serial_dev(port: str): - timeout = ENUM_TIMEOUT + timeout = enum_timeout() ser = None while timeout > 0: if os.path.exists(port): try: - # write_timeout: a wedged device otherwise blocks ser.write() forever, - # hanging the worker until the pool/job timeout kills the whole run + # write_timeout: see serial_write_all ser = serial.Serial(port, baudrate=115200, timeout=SERIAL_READ_TIMEOUT, write_timeout=SERIAL_WRITE_TIMEOUT) break @@ -293,53 +295,71 @@ def open_serial_dev(port: str): def serial_write_all(ser: serial.Serial, data: bytes): - # write_timeout is a total deadline for the whole call (pyserial keeps partial progress - # internally). A timeout means the device stopped draining — treat it as fatal: pyserial - # loses the partial-write count on raise, so retrying would duplicate bytes on the wire. + # write_timeout is a deadline for the whole call. A timeout means the device stopped + # draining, and it is fatal: pyserial loses the partial-write count on raise, so + # retrying would duplicate bytes on the wire. try: ser.write(data) except serial.SerialTimeoutException: raise AssertionError(f'Serial write timeout after {SERIAL_WRITE_TIMEOUT:.1f}s') +LP_OPEN_TIMEOUT = 5 # bound on opening the printer lp node; see test_device_printer_to_cdc +# Runs under hil_util.run_alongside as `python3 -c`. Inline rather than a file so hil_ci.sh's +# staging list does not need another entry to keep the rig working. +LP_READER = ( + 'import os, sys\n' + 'fd = os.open(sys.argv[1], os.O_RDONLY)\n' + # readiness marker: the parent must not send a byte before the node is open, or the + # bytes are lost. A blind sleep raced CPython start-up on a loaded rig. + 'open(sys.argv[3], "w").close()\n' + 'want = int(sys.argv[2])\n' + 'buf = b""\n' + 'while len(buf) < want:\n' + ' chunk = os.read(fd, min(64, want - len(buf)))\n' + ' if not chunk:\n' + ' break\n' + ' buf += chunk\n' + 'sys.stdout.buffer.write(buf)\n' +) +MTYPE_TIMEOUT = 30 # a README-sized read is <1 s; bounds a D-state hang on a wedged device + + def read_disk_file(uid: str, lun: int, fname: str) -> bytes: - # Reads a file from a FAT volume on a block device without mounting it. - # Requires mtools: `apt install mtools` (no pip dependency). + # Reads a file from an unmounted FAT volume; needs mtools. run_cmd everywhere in this + # file rather than subprocess.run/check_output: its post-timeout reap is an unbounded + # communicate() with no killpg (CPython 3.13.5 subprocess.py:558-565 -- kill(), then + # communicate() with NO timeout), which never returns on a device wedged in D state, + # where the kill is queued and never delivered. binary + # keeps the bytes exact, split_stderr keeps mtype warnings out of them. dev = get_disk_dev(uid, 'TinyUSB', lun) - timeout = ENUM_TIMEOUT last_err = None - while timeout > 0: - if os.path.exists(dev): - try: - data = subprocess.check_output( - ['mtype', '-i', dev, f'::/{fname}'], stderr=subprocess.PIPE) - assert data, f'Cannot read file {fname} from {dev}' - return data - except subprocess.CalledProcessError as e: - last_err = e.stderr.decode(errors='replace').strip() - time.sleep(1) - timeout -= 1 - raise AssertionError(f'mtype failed on {dev}: {last_err}' if last_err else f'Storage {dev} not existed') + def try_read(): + nonlocal last_err + if not os.path.exists(dev): + return None + r = hil_util.run_cmd(f"mtype -i {shlex.quote(dev)} ::/{shlex.quote(fname)}", + timeout=MTYPE_TIMEOUT, binary=True, split_stderr=True, quiet=True) + if r.returncode == 0: + if r.stdout: + return r.stdout + # rc 0 with no data is an answer (empty file, zeroed sectors), not "not + # ready" — fail now instead of spinning the budget + raise AssertionError(f'Cannot read file {fname} from {dev}: mtype returned no data') + last_err = (r.stderr or b'').decode(errors='replace').strip() or f'mtype rc {r.returncode}' + return None + data = wait_until(try_read) + if data is None: + raise AssertionError(f'Cannot read file {fname} from {dev}: {last_err}' if last_err + else f'Storage {dev} not existed') + return data -def open_mtp_dev(uid): - mtp = MTP() - timeout = ENUM_TIMEOUT - while timeout > 0: - # unmount gio/gvfs MTP mount which blocks libmtp from accessing the device - subprocess.run(f"gio mount -u mtp://TinyUsb_TinyUsb_Device_{uid}/", - shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) - for raw in mtp.detect_devices(): - mtp.device = mtp.mtp.LIBMTP_Open_Raw_Device(ctypes.byref(raw)) - if mtp.device: - sn = mtp.get_serialnumber().decode('utf-8') - if sn == uid: - return mtp - mtp.disconnect() - time.sleep(1) - timeout -= 1 - return None + +# ~5 KB of transfers plus libmtp setup takes seconds, not minutes; a larger value makes a +# wedged MTP board cost that much on every retry, all charged to the pool guard. +MTP_SESSION_MARGIN = 30 # transfer budget after enumeration; past it the session is killed def get_printer_dev(id: str, vendor_str, product_str, ifnum: int): @@ -348,235 +368,30 @@ def get_printer_dev(id: str, vendor_str, product_str, ifnum: int): product_str = product_str.replace(' ', '_') if product_str else '' for lp in glob.glob('/sys/class/usbmisc/lp*'): try: - sn = open(f'{lp}/device/../serial').read().strip() + # bounded: same device_lock() exposure as the sibling reads (see read_sysfs) + sn = hil_util.read_sysfs(f'{lp}/device/../serial') + # UNKNOWN is not None: the sentinel has no __eq__, so an unanswered read + # would fall through both tests and read as 'not this board' -- the exact + # absence/unknown conflation read_sysfs exists to prevent. + if sn is None or sn is hil_util.SYSFS_UNKNOWN: + continue if sn == id: return f'/dev/usb/{os.path.basename(lp)}' - except (FileNotFoundError, PermissionError, ValueError): + except OSError: # read_sysfs swallows its own OSError/ValueError; glob can race pass return None def open_printer_dev(id: str, vendor_str, product_str, ifnum: int) -> str: """Wait for printer device to enumerate and return its path""" - timeout = ENUM_TIMEOUT - while timeout > 0: + def try_find(): lp_dev = get_printer_dev(id, vendor_str, product_str, ifnum) - if lp_dev and os.path.exists(lp_dev): - return lp_dev - time.sleep(1) - timeout -= 1 - assert False, f'Printer device not found for {id} if{ifnum:02d}' - - -# ------------------------------------------------------------- -# Flashing firmware -# ------------------------------------------------------------- -def run_cmd(cmd: str, cwd: str | None = None, timeout: int = CMD_TIMEOUT) -> subprocess.CompletedProcess: - popen_kwargs = { - 'cwd': cwd, - 'shell': True, - 'stdout': subprocess.PIPE, - 'stderr': subprocess.STDOUT, - 'text': True, - 'encoding': 'utf-8', - 'errors': 'replace', - } - if os.name != 'nt': - popen_kwargs['preexec_fn'] = os.setsid - - p = subprocess.Popen(cmd, **popen_kwargs) - try: - out, _ = p.communicate(timeout=timeout) - r = subprocess.CompletedProcess(args=cmd, returncode=p.returncode, stdout=out) - except subprocess.TimeoutExpired as ex: - if os.name != 'nt': - try: - os.killpg(p.pid, signal.SIGKILL) - except ProcessLookupError: - pass - else: - p.kill() - out, _ = p.communicate() - timeout_out = ex.stdout or out or b'' - title = f'COMMAND TIMEOUT ({timeout}s): {cmd}' - print() - if os.getenv('CI'): - print(f"::group::{title}") - print(cmd_stdout_text(timeout_out)) - print(f"::endgroup::") - else: - print(title) - print(cmd_stdout_text(timeout_out)) - return subprocess.CompletedProcess(args=cmd, returncode=124, stdout=timeout_out) - - if r.returncode != 0: - title = f'COMMAND FAILED: {cmd}' - print() - if os.getenv('CI'): - print(f"::group::{title}") - print(cmd_stdout_text(r.stdout)) - print(f"::endgroup::") - else: - print(title) - print(cmd_stdout_text(r.stdout)) - elif verbose: - print(cmd) - print(cmd_stdout_text(r.stdout)) - return r - - -def flash_jlink(board: Board, firmware: str) -> subprocess.CompletedProcess: - flasher = board['flasher'] - script = ['halt', 'r', f'loadfile {firmware}.elf', 'r', 'go', 'exit'] - f_jlink = Path(f'{board["name"]}_{Path(firmware).name}.jlink') - with f_jlink.open('w') as f: - f.writelines(f'{s}\n' for s in script) - ret = run_cmd(f'JLinkExe -USB {flasher["uid"]} {flasher["args"]} -if swd -JTAGConf -1,-1 -speed auto -NoGui 1 -ExitOnError 1 -CommandFile {f_jlink}') - f_jlink.unlink(missing_ok=True) - return ret - - -def reset_jlink(board: Board) -> subprocess.CompletedProcess: - flasher = board['flasher'] - script = ['halt', 'r', 'go', 'exit'] - f_jlink = Path(f'{board["name"]}_reset.jlink') - if not f_jlink.exists(): - with f_jlink.open('w') as f: - f.writelines(f'{s}\n' for s in script) - ret = run_cmd(f'JLinkExe -USB {flasher["uid"]} {flasher["args"]} -if swd -JTAGConf -1,-1 -speed auto -NoGui 1 -ExitOnError 1 -CommandFile {f_jlink}') - return ret - - -def flash_stlink(board, firmware): - flasher = board['flasher'] - return run_cmd(f'STM32_Programmer_CLI --connect port=swd sn={flasher["uid"]} --write {firmware}.elf --go') - + return lp_dev if lp_dev and os.path.exists(lp_dev) else None -def reset_stlink(board): - flasher = board['flasher'] - return run_cmd(f'STM32_Programmer_CLI --connect port=swd sn={flasher["uid"]} --rst --go') - -def flash_stflash(board, firmware): - flasher = board['flasher'] - ret = run_cmd(f'st-flash --serial {flasher["uid"]} write {firmware}.bin 0x8000000') - return ret - - -def reset_stflash(board): - flasher = board['flasher'] - return subprocess.CompletedProcess(args=['dummy'], returncode=0) - - -def flash_openocd(board, firmware): - flasher = board['flasher'] - ret = run_cmd(f'openocd -c "tcl_port disabled" -c "gdb_port disabled" -c "adapter serial {flasher["uid"]}" ' - f'{flasher["args"]} -c "init; halt; program {firmware}.elf verify; reset; exit"') - return ret - - -def reset_openocd(board): - flasher = board['flasher'] - ret = run_cmd(f'openocd -c "tcl_port disabled" -c "gdb_port disabled" -c "adapter serial {flasher["uid"]}" ' - f'{flasher["args"]} -c "init; reset run; exit"') - return ret - - -def flash_openocd_wch(board, firmware): - flasher = board['flasher'] - f_wch = f"wch-riscv_{board['uid']}.cfg" - if not os.path.exists(f_wch): - with open(f_wch, 'w') as file: - file.write(WCH_RISCV_CONTENT) - - ret = run_cmd(f'openocd_wch -c "adapter serial {flasher["uid"]}" -f {f_wch} ' - f'-c "program {firmware}.elf reset exit"') - return ret - - -def reset_openocd_wch(board): - flasher = board['flasher'] - f_wch = f"wch-riscv_{board['uid']}.cfg" - if not os.path.exists(f_wch): - with open(f_wch, 'w') as file: - file.write(WCH_RISCV_CONTENT) - - ret = run_cmd(f'openocd_wch -c "adapter serial {flasher["uid"]}" -f {f_wch} -c "program reset exit"') - return ret - - -def flash_openocd_adi(board: Board, firmware: str) -> subprocess.CompletedProcess: - flasher = board['flasher'] - openocd = OPENCOD_ADI_PATH / 'src' / 'openocd' - tcl_dir = OPENCOD_ADI_PATH / 'tcl' - ret = run_cmd(f'{openocd} -c "adapter serial {flasher["uid"]}" -s {tcl_dir} ' - f'{flasher["args"]} -c "program {firmware}.elf reset exit"') - return ret - - -def reset_openocd_adi(board: Board) -> subprocess.CompletedProcess: - flasher = board['flasher'] - openocd = OPENCOD_ADI_PATH / 'src' / 'openocd' - tcl_dir = OPENCOD_ADI_PATH / 'tcl' - ret = run_cmd(f'{openocd} -c "adapter serial {flasher["uid"]}" -s {tcl_dir} ' - f'{flasher["args"]} -c "program reset exit"') - return ret - - -def flash_wlink_rs(board, firmware): - flasher = board['flasher'] - # wlink use index for probe selection and lacking usb serial support - ret = run_cmd(f'wlink flash {firmware}.elf') - return ret - - -def reset_wlink_rs(board): - flasher = board['flasher'] - # wlink use index for probe selection and lacking usb serial support - ret = run_cmd(f'wlink reset') - return ret - - -def flash_esptool(board: Board, firmware: str) -> subprocess.CompletedProcess: - flasher = board['flasher'] - port = get_serial_dev(flasher["uid"], None, None, 0) - fw_dir = Path(f'{firmware}.bin').parent - with (fw_dir / 'config.env').open() as f: - idf_target = json.load(f)['IDF_TARGET'] - with (fw_dir / 'flash_args').open() as f: - flash_args = f.read().strip().replace('\n', ' ') - command = (f'esptool --chip {idf_target} -p {port} {flasher["args"]} ' - f'--before=default_reset --after=hard_reset write_flash {flash_args}') - ret = run_cmd(command, cwd=str(fw_dir)) - return ret - - -def reset_esptool(board): - flasher = board['flasher'] - return subprocess.CompletedProcess(args=['dummy'], returncode=0) - - -def flash_uniflash(board, firmware): - flasher = board['flasher'] - ret = run_cmd(f'dslite.sh {flasher["args"]} -f {firmware}.hex') - return ret - - -def reset_uniflash(board): - flasher = board['flasher'] - return subprocess.CompletedProcess(args=['dummy'], returncode=0) - - -def flash_lm4flash(board, firmware): - # TI Tiva-C / Stellaris ICDI: lightweight lm4flash, resets and runs after write - flasher = board['flasher'] - ret = run_cmd(f'lm4flash -s {flasher["uid"]} {flasher["args"]} {firmware}.bin') - return ret - - -def reset_lm4flash(board): - # lm4flash has no reset-only mode; it resets+runs on flash, so reset is a no-op - flasher = board['flasher'] - return subprocess.CompletedProcess(args=['dummy'], returncode=0) + lp_dev = wait_until(try_find) + assert lp_dev, (f'Printer device not found for {id} if{ifnum:02d}' + + hil_util.sysfs_blind_note()) + return lp_dev # ------------------------------------------------------------- @@ -585,18 +400,16 @@ def reset_lm4flash(board): def test_dual_host_info_to_device_cdc(board): uid = board['uid'] declared_devs = [f'{d["vid_pid"]}_{d["serial"]}' for d in board['tests']['dev_attached']] - port = get_serial_dev(uid, 'TinyUSB', "TinyUSB_Device", 0) + port = hil_util.get_serial_dev(uid, 'TinyUSB', "TinyUSB_Device", 0) ser = open_serial_dev(port) ser.timeout = 0.1 - # read until all expected devices are enumerated data = b'' - timeout = ENUM_TIMEOUT + timeout = enum_timeout() while timeout > 0: new_data = ser.read(ser.in_waiting or 1) if new_data: data += new_data - # check if all devices found enum_dev_sn = [] for l in data.decode('utf-8', errors='ignore').splitlines(): vid_pid_sn = re.search(r'ID ([0-9a-fA-F]+):([0-9a-fA-F]+) SN (\w+)', l) @@ -633,22 +446,20 @@ def test_host_device_info(board): flasher = board['flasher'] declared_devs = [f'{d["vid_pid"]}_{d["serial"]}' for d in board['tests']['dev_attached']] - port = get_serial_dev(flasher["uid"], None, None, 0) + port = hil_util.get_serial_dev(flasher["uid"], None, None, 0) ser = open_serial_dev(port) ser.timeout = 0.1 # reset device since we can miss the first line - ret = globals()[f'reset_{flasher["name"].lower()}'](board) + ret = getattr(hil_flash, f'reset_{flasher["name"].lower()}')(board) assert ret.returncode == 0, 'Failed to reset device' - # read until all expected devices are enumerated data = b'' - timeout = ENUM_TIMEOUT + timeout = enum_timeout() while timeout > 0: new_data = ser.read(ser.in_waiting or 1) if new_data: data += new_data - # check if all devices found enum_dev_sn = [] for l in data.decode('utf-8', errors='ignore').splitlines(): vid_pid_sn = re.search(r'ID ([0-9a-fA-F]+):([0-9a-fA-F]+) SN (\w+)', l) @@ -712,17 +523,16 @@ def test_host_cdc_msc_hid(board): if not cdc_devs and not msc_devs: return 'skipped' - port = get_serial_dev(flasher["uid"], None, None, 0) + port = hil_util.get_serial_dev(flasher["uid"], None, None, 0) ser = open_serial_dev(port) ser.timeout = 0.1 # reset device to catch mount messages - ret = globals()[f'reset_{flasher["name"].lower()}'](board) + ret = getattr(hil_flash, f'reset_{flasher["name"].lower()}')(board) assert ret.returncode == 0, 'Failed to reset device' - # Wait for all expected mount messages data = b'' - timeout = ENUM_TIMEOUT + timeout = enum_timeout() wait_cdc = len(cdc_devs) > 0 wait_msc = len(msc_devs) > 0 while timeout > 0: @@ -736,7 +546,6 @@ def test_host_cdc_msc_hid(board): time.sleep(0.1) timeout -= 0.1 - # Lookup serial chip name from vid_pid vid_pid_name = { '0403_6001': 'FTDI', '0403_6010': 'FTDI', '0403_6011': 'FTDI', '0403_6014': 'FTDI', '10c4_ea60': 'CP210x', '10c4_ea70': 'CP210x', @@ -747,7 +556,6 @@ def test_host_cdc_msc_hid(board): lines = data.decode('utf-8', errors='ignore').splitlines() - # Verify and print CDC mount if cdc_devs: assert b'CDC Interface is mounted' in data, 'CDC device not mounted on host' dev = cdc_devs[0] @@ -756,7 +564,6 @@ def test_host_cdc_msc_hid(board): if 'CDC Interface is mounted' in l: print(f'\r\n {chip_name}: {l} ', end='') - # Verify and print MSC mount (inquiry + disk size) if msc_devs: assert b'MassStorage device is mounted' in data, 'MSC device not mounted on host' assert b'Disk Size' in data, 'MSC Disk Size not reported' @@ -776,7 +583,6 @@ def test_host_cdc_msc_hid(board): packet_size = 64 - # Echo test: write random 1-packet_size chunks, wait for echo before sending next echo_len = 1024 echo_data = rand_ascii(echo_len) ser.reset_input_buffer() @@ -784,7 +590,6 @@ def test_host_cdc_msc_hid(board): while offset < echo_len: chunk_size = min(random.randint(1, packet_size), echo_len - offset) serial_write_all(ser, echo_data[offset:offset + chunk_size]) - # wait until this chunk is echoed back echo = b'' t_end = time.monotonic() + 1.0 while time.monotonic() < t_end and len(echo) < chunk_size: @@ -805,17 +610,16 @@ def test_host_msc_file_explorer(board): if not msc_devs: return 'skipped' - port = get_serial_dev(flasher["uid"], None, None, 0) + port = hil_util.get_serial_dev(flasher["uid"], None, None, 0) ser = open_serial_dev(port) ser.timeout = 0.1 # reset device to catch mount messages - ret = globals()[f'reset_{flasher["name"].lower()}'](board) + ret = getattr(hil_flash, f'reset_{flasher["name"].lower()}')(board) assert ret.returncode == 0, 'Failed to reset device' - # Wait for MSC mount (Disk Size message) data = b'' - timeout = ENUM_TIMEOUT + timeout = enum_timeout() while timeout > 0: new_data = ser.read(ser.in_waiting or 1) if new_data: @@ -850,14 +654,12 @@ def test_host_msc_file_explorer(board): if MSC_README_TXT.decode() in resp_text: print('README.TXT matched ', end='') - # MSC throughput test: send dd command to read sectors time.sleep(0.5) ser.reset_input_buffer() for ch in 'dd 1024\r': serial_write_all(ser, ch.encode()) time.sleep(0.002) - # Read dd output until prompt resp = b'' t = 30.0 while t > 0: @@ -874,12 +676,13 @@ def test_host_msc_file_explorer(board): for line in resp_text.splitlines(): if 'KB/s' in line: print(f'{line.strip()} ', end='') - m = re.search(r'([\d.]+\s*[KMG]B/s)', line) # MSC read speed for the report cell + m = re.search(r'([\d.]+)\s*([KMG]B/s)', line) # MSC read speed for the report cell if m: - speed = 'rd ' + m.group(1).replace(' ', '') + speed = f'{m.group(1)} {m.group(2)}' break ser.close() + assert speed is not None, 'MSC read produced no speed report (dd stalled or failed)' return speed @@ -891,15 +694,14 @@ def test_host_msc_file_explorer_freertos(board): # Tests: device # ------------------------------------------------------------- def test_device_board_test(board): - # Dummy test pass def test_device_cdc_dual_ports(board): uid = board['uid'] port = [ - get_serial_dev(uid, 'TinyUSB', "TinyUSB_Device", 0), - get_serial_dev(uid, 'TinyUSB', "TinyUSB_Device", 2) + hil_util.get_serial_dev(uid, 'TinyUSB', "TinyUSB_Device", 0), + hil_util.get_serial_dev(uid, 'TinyUSB', "TinyUSB_Device", 2) ] ser = [open_serial_dev(p) for p in port] @@ -938,7 +740,7 @@ def test_device_cdc_dual_ports(board): def test_device_cdc_msc(board): uid = board['uid'] # CDC Echo test - port = get_serial_dev(uid, 'TinyUSB', "TinyUSB_Device", 0) + port = hil_util.get_serial_dev(uid, 'TinyUSB', "TinyUSB_Device", 0) ser = open_serial_dev(port) def rand_ascii(length): @@ -967,6 +769,20 @@ def test_device_cdc_msc_freertos(board): test_device_cdc_msc(board) +def link_is_fs(speed) -> bool: + """Payload scaling from a `speed` attribute. Anything not positively read as high speed + counts as FS -- including None and SYSFS_UNKNOWN: the FS payload merely tests an HS + board less, while the HS payload hard-fails a healthy FS board.""" + return speed not in ('480', '5000', '10000') + + +def dd_timeout(mib: float) -> int: + """Bound one dd by what was ASKED for: 2.5 s/MiB is the slowest rate this test has + measured (FS CDC, ~420 kB/s), over a 30 s floor. A flat bound fails a healthy board as + soon as the payload grows or the leaf-hub uplink is shared.""" + return int(30 + 2.5 * mib) + + def test_device_cdc_msc_throughput(board): uid = board['uid'] @@ -977,59 +793,64 @@ def test_device_cdc_msc_throughput(board): return f'{float(m.group(1)):.1f} {m.group(2)}ps' return '?' - # Wait for MSC disk enumeration dev = get_disk_dev(uid, 'TinyUSB', 0) - timeout = ENUM_TIMEOUT + timeout = enum_timeout() while timeout > 0: if os.path.exists(dev): break time.sleep(0.1); timeout -= 0.1 assert timeout > 0, f'Disk {dev} not found' - # Wait for CDC tty enumeration - tty = get_serial_dev(uid, 'TinyUSB', 'Throughput', 0) - timeout = ENUM_TIMEOUT + tty = hil_util.get_serial_dev(uid, 'TinyUSB', 'Throughput', 0) + timeout = enum_timeout() while timeout > 0: if os.path.exists(tty): break time.sleep(0.1); timeout -= 0.1 assert timeout > 0, f'CDC tty {tty} not found' - # Detect speed (12 Mbps FS / 480 Mbps HS) for payload scaling - is_fs = False - for f in glob.glob('/sys/bus/usb/devices/*/serial'): - try: - if open(f).read().strip() == uid: - is_fs = (open(os.path.join(os.path.dirname(f), 'speed')).read().strip() == '12') - break - except (OSError, ValueError): - pass + # Detect speed (12 Mbps FS / 480 Mbps HS) for payload scaling; a device we never find + # keeps the FS payload (see link_is_fs) + # usb_scan, not a private glob: it skips root hubs and remembers paths that already + # stranded, so one wedged peer cannot spend this worker's blindness budget four reads + # at a time. + is_fs = True + speed_known = False + devs, _ = hil_util.usb_scan(vid='cafe', serial=uid) + if devs: + speed = hil_util.read_sysfs(os.path.join(devs[0]['dir'], 'speed')) + is_fs = link_is_fs(speed) + speed_known = speed not in (None, hil_util.SYSFS_UNKNOWN) # Put tty in raw mode so dd sees pure binary throughput. - rs = run_cmd(f'timeout 30 stty -F {tty} raw -echo') - assert rs.returncode == 0, f'stty failed: {cmd_stdout_text(rs.stdout)}' + rs = hil_util.run_cmd(f'timeout 30 stty -F {tty} raw -echo') + assert rs.returncode == 0, f'stty failed: {hil_util.cmd_stdout_text(rs.stdout)}' # Payload aim: ~5 s per direction at FS (~830 kB/s), much less at HS. msc_count = 2 if is_fs else 16 # bs=1M cdc_count = 16 if is_fs else 128 # bs=64K tmp_file = f'/tmp/cdc_msc_tp_{uid}.bin' + t_cdc, t_msc = dd_timeout(cdc_count / 16), dd_timeout(msc_count) - rw = run_cmd(f'timeout 30 dd if=/dev/zero of={tty} bs=64K count={cdc_count} 2>&1') - assert rw.returncode == 0, f'CDC dd write failed: {cmd_stdout_text(rw.stdout)}' - cdc_w = parse_speed(cmd_stdout_text(rw.stdout)) + rw = hil_util.run_cmd(f'timeout {t_cdc} dd if=/dev/zero of={tty} bs=64K count={cdc_count} 2>&1') + assert rw.returncode == 0, f'CDC dd write failed: {hil_util.cmd_stdout_text(rw.stdout)}' + cdc_w = parse_speed(hil_util.cmd_stdout_text(rw.stdout)) - rr = run_cmd(f'timeout 30 dd if={tty} of=/dev/null bs=64K count={cdc_count} iflag=fullblock 2>&1') - assert rr.returncode == 0, f'CDC dd read failed: {cmd_stdout_text(rr.stdout)}' - cdc_r = parse_speed(cmd_stdout_text(rr.stdout)) + rr = hil_util.run_cmd(f'timeout {t_cdc} dd if={tty} of=/dev/null bs=64K count={cdc_count} iflag=fullblock 2>&1') + assert rr.returncode == 0, f'CDC dd read failed: {hil_util.cmd_stdout_text(rr.stdout)}' + cdc_r = parse_speed(hil_util.cmd_stdout_text(rr.stdout)) - rmr = run_cmd(f'dd if={dev} of={tmp_file} bs=1M count={msc_count} iflag=direct 2>&1') - assert rmr.returncode == 0, f'MSC dd read failed: {cmd_stdout_text(rmr.stdout)}' - msc_r = parse_speed(cmd_stdout_text(rmr.stdout)) + # inner bound, like the CDC pair above: run_cmd's SIGKILL is merely QUEUED against a + # dd blocked in the block layer on a half-dead device, so without one the call rides + # CMD_TIMEOUT and is abandoned holding the disk and usbfs nodes. + rmr = hil_util.run_cmd(f'timeout {t_msc} dd if={dev} of={tmp_file} bs=1M count={msc_count} iflag=direct 2>&1') + assert rmr.returncode == 0, f'MSC dd read failed: {hil_util.cmd_stdout_text(rmr.stdout)}' + msc_r = parse_speed(hil_util.cmd_stdout_text(rmr.stdout)) - rmw = run_cmd(f'dd if={tmp_file} of={dev} bs=1M count={msc_count} oflag=direct 2>&1') - assert rmw.returncode == 0, f'MSC dd write failed: {cmd_stdout_text(rmw.stdout)}' - msc_w = parse_speed(cmd_stdout_text(rmw.stdout)) + rmw = hil_util.run_cmd(f'timeout {t_msc} dd if={tmp_file} of={dev} bs=1M count={msc_count} oflag=direct 2>&1') + assert rmw.returncode == 0, f'MSC dd write failed: {hil_util.cmd_stdout_text(rmw.stdout)}' + msc_w = parse_speed(hil_util.cmd_stdout_text(rmw.stdout)) try: os.remove(tmp_file) @@ -1037,40 +858,66 @@ def test_device_cdc_msc_throughput(board): pass print(f' CDC read {cdc_r} write {cdc_w}, MSC read {msc_r} write {msc_w} ', end='') - # compact read/write speed for the report cell, e.g. "✅ CDC 652k/422k MSC 1.1M/783k" - short = lambda s: (s.split()[0].rstrip('0').rstrip('.') + s.split()[-1][0]) if ' ' in s else s - return f'{REPORT_CELL["pass"]} CDC {short(cdc_r)}/{short(cdc_w)} MSC {short(msc_r)}/{short(msc_w)}' + + # report cell, e.g. "✅ C 652/422k M 1.1M/783k" (C=CDC, M=MSC; shared unit shown once) + def short(s): + return (s.split()[0].rstrip('0').rstrip('.') + s.split()[-1][0]) if ' ' in s else s + + def pair(r, w): + r, w = short(r), short(w) + if r[-1:] == w[-1:] and r[-1:].isalpha(): + r = r[:-1] + return f'{r}/{w}' + + # 'FS?' when the speed could not be read: the numbers below were produced against the FS + # payload, so an HS board reads as suspiciously slow. Say so rather than publish a green + # cell whose scale is a guess. + scale = '' if speed_known else ' FS?' + return f'{REPORT_CELL["pass"]} C {pair(cdc_r, cdc_w)} M {pair(msc_r, msc_w)}{scale}' def test_device_dfu(board): uid = board['uid'] + vid_pid = 'cafe:400b' - # Wait device enum - timeout = ENUM_TIMEOUT - while timeout > 0: - ret = run_cmd(f'dfu-util -l') - stdout = cmd_stdout_text(ret.stdout) - if f'serial="{uid}"' in stdout and 'Found DFU: [cafe:4000]' in stdout: + # Deadline-based: dfu-util takes ~1 s per call, which a countdown would not charge + # against the budget. -d pins enumeration to THIS example's ids: a bare `-l` opens every + # DFU-capable node, and one wedged node blocks that open in D state. The pair is doubled + # because dfu-util matches run-time and DFU-mode devices against SEPARATE id pairs + # (parse_vendprod: an omitted DFU-mode pair matches ANY DFU-mode device). The deadline + # is only tested BETWEEN calls, so the per-call bound is what caps a blocked open. + deadline = time.monotonic() + enum_timeout() + found = False + while time.monotonic() < deadline: + ret = hil_util.run_cmd(f'dfu-util -d {vid_pid},{vid_pid} -l', timeout=15) + stdout = hil_util.cmd_stdout_text(ret.stdout) + if f'serial="{uid}"' in stdout and f'Found DFU: [{vid_pid}]' in stdout: + found = True break time.sleep(1) - timeout = timeout - 1 - assert timeout > 0, 'Device not available' + assert found, 'Device not available' f_dfu0 = f'dfu0_{uid}' f_dfu1 = f'dfu1_{uid}' - # Test upload try: os.remove(f_dfu0) os.remove(f_dfu1) except OSError: pass - ret = run_cmd(f'dfu-util -S {uid} -a 0 -U {f_dfu0}') + # -d as well as -S: dfu-util matches the SERIAL only after libusb_open() (dfu_util.c + # probes the descriptor for iSerialNumber), so -S alone still opens every DFU-capable + # node. The id filter runs BEFORE the open; -S then picks our board (see the poll). + # Each partition is one short string, so a healthy upload is ~1 s; the bound is there + # for a node that stops answering mid-transfer. + ret = hil_util.run_cmd(f'dfu-util -d {vid_pid},{vid_pid} -S {uid} -a 0 -U {f_dfu0}', + timeout=30) assert ret.returncode == 0, 'Upload failed' - ret = run_cmd(f'dfu-util -S {uid} -a 1 -U {f_dfu1}') + ret = hil_util.run_cmd(f'dfu-util -d {vid_pid},{vid_pid} -S {uid} -a 1 -U {f_dfu1}', + timeout=30) assert ret.returncode == 0, 'Upload failed' with open(f_dfu0) as f: @@ -1085,17 +932,19 @@ def test_device_dfu(board): def test_device_dfu_runtime(board): uid = board['uid'] - # Wait device enum - timeout = ENUM_TIMEOUT - while timeout > 0: - ret = run_cmd(f'dfu-util -l') - stdout = cmd_stdout_text(ret.stdout) - if f'serial="{uid}"' in stdout and 'Found Runtime: [cafe:4000]' in stdout: + vid_pid = 'cafe:400c' + # enumeration pinned to this example's ids, same per-call bound (see test_device_dfu) + deadline = time.monotonic() + enum_timeout() + found = False + while time.monotonic() < deadline: + ret = hil_util.run_cmd(f'dfu-util -d {vid_pid},{vid_pid} -l', timeout=15) + stdout = hil_util.cmd_stdout_text(ret.stdout) + if f'serial="{uid}"' in stdout and f'Found Runtime: [{vid_pid}]' in stdout: + found = True break time.sleep(1) - timeout = timeout - 1 - assert timeout > 0, 'Device not available' + assert found, 'Device not available' def test_device_hid_boot_interface(board): @@ -1103,8 +952,7 @@ def test_device_hid_boot_interface(board): kbd = get_hid_dev(uid, 'TinyUSB', 'TinyUSB_Device', 'event-kbd') mouse1 = get_hid_dev(uid, 'TinyUSB', 'TinyUSB_Device', 'if01-event-mouse') mouse2 = get_hid_dev(uid, 'TinyUSB', 'TinyUSB_Device', 'if01-mouse') - # Wait device enum - timeout = ENUM_TIMEOUT + timeout = enum_timeout() while timeout > 0: if os.path.exists(kbd) and os.path.exists(mouse1) and os.path.exists(mouse2): break @@ -1120,12 +968,9 @@ def test_device_hid_composite_freertos(id): def test_device_printer_to_cdc(board): - import threading - uid = board['uid'] - # Wait for CDC port and printer device - cdc_port = get_serial_dev(uid, 'TinyUSB', "TinyUSB_Device", 0) + cdc_port = hil_util.get_serial_dev(uid, 'TinyUSB', "TinyUSB_Device", 0) ser = open_serial_dev(cdc_port) lp_dev = open_printer_dev(uid, 'TinyUSB', 'TinyUSB_Device', 2) @@ -1145,7 +990,6 @@ def test_device_printer_to_cdc(board): sizes = [32, 64, 128, 256, 512, random.randint(2000, 5000)] - # flush any stale data ser.reset_input_buffer() # Test 1: Printer -> CDC with multiple sizes, write in random 1-64 byte chunks @@ -1155,7 +999,17 @@ def test_device_printer_to_cdc(board): ser.reset_input_buffer() rd = b'' offset = 0 - lp_fd = os.open(lp_dev, os.O_WRONLY | os.O_NONBLOCK) + # bounded: O_NONBLOCK does NOT save us -- usblp_open() takes the device mutex + # first -- and this open runs on the worker itself, with no thread to abandon + lp_fd = hil_util.bounded_open(lp_dev, os.O_WRONLY | os.O_NONBLOCK, 5) + # Three-valued on purpose: an OSError here is a FACT about the node (EBUSY from + # usblp's single-opener rule, ENOENT from a re-enumeration race, EACCES from a + # udev gap) and must not be reported as a wedge -- that sends the operator to + # usb-kernel-recover for hardware that is fine. + assert lp_fd is not hil_util.SYSFS_UNKNOWN, ( + f'printer: opening {lp_dev} for write blocked (device wedged)' + f'{hil_util.sysfs_blind_note()}') + assert lp_fd is not None, f'printer: {lp_dev} could not be opened for write' try: while offset < size: chunk_size = min(random.randint(1, 64), size - offset) @@ -1179,132 +1033,92 @@ def test_device_printer_to_cdc(board): assert rd == test_data, (f'Printer->CDC wrong data ({size} bytes):\n' f' expected: {test_data[:64]}\n received: {rd[:64]}') - # Test 2: CDC -> Printer with multiple sizes, write in random 1-64 byte chunks - # Use a thread to read from printer since /dev/usb/lp read blocks + # Test 2: CDC -> Printer with multiple sizes, write in random 1-64 byte chunks. + # The lp read runs in a PROCESS, not a thread: /dev/usb/lp* blocks on read, usblp + # allows a SINGLE opener, and a blocked thread cannot be abandoned without keeping + # that fd -- which poisoned the node for every later test this worker ran. A killed + # process takes its fd with it. ser.reset_input_buffer() time.sleep(0.5) for size in sizes: test_data = rand_ascii(size) - rd_result = [b'', None] # [data, error] - reader_ready = threading.Event() - - def lp_reader(): - try: - rd = b'' - fd = os.open(lp_dev, os.O_RDONLY) - reader_ready.set() - try: - while len(rd) < size: - chunk = os.read(fd, min(64, size - len(rd))) - if not chunk: - break - rd += chunk - finally: - os.close(fd) - rd_result[0] = rd - except Exception as e: - rd_result[1] = e - reader_ready.set() - reader = threading.Thread(target=lp_reader, daemon=True) - reader.start() - # wait for reader to open lp device before writing - reader_ready.wait(timeout=5) - time.sleep(0.1) + ready = Path(tempfile.gettempdir()) / f'hil-lp-ready-{os.getpid()}-{size}' + ready.unlink(missing_ok=True) - # Write to CDC in small chunks with flush to avoid overflowing device FIFO - offset = 0 - while offset < size: - chunk_size = min(random.randint(1, 64), size - offset) - serial_write_all(ser, test_data[offset:offset + chunk_size]) - time.sleep(0.01) - offset += chunk_size + def write_cdc(): + # WAIT for the reader to have the node open. The child has to fork, exec and + # boot a CPython interpreter; on a loaded rig that routinely exceeds the 0.3s + # this used to sleep, and every byte sent early is lost -- surfacing as a + # spurious data mismatch rather than a timeout. + deadline = time.monotonic() + LP_OPEN_TIMEOUT + 5 + while not ready.exists(): + if time.monotonic() > deadline: + return # reader never opened; the rc/compare below reports it + time.sleep(0.02) + offset = 0 + while offset < size: + chunk_size = min(random.randint(1, 64), size - offset) + serial_write_all(ser, test_data[offset:offset + chunk_size]) + time.sleep(0.01) + offset += chunk_size - reader.join(timeout=10) - assert not reader.is_alive(), f'CDC->Printer timeout ({size} bytes)' - assert rd_result[1] is None, f'CDC->Printer read error: {rd_result[1]}' - assert rd_result[0] == test_data, (f'CDC->Printer wrong data ({size} bytes):\n' - f' expected: {test_data[:64]}\n received: {rd_result[0][:64]}') + try: + r = hil_util.run_alongside( + [sys.executable, '-c', LP_READER, lp_dev, str(size), str(ready)], + write_cdc, LP_OPEN_TIMEOUT + 12) + finally: + ready.unlink(missing_ok=True) + # stderr, not stdout: run_alongside keeps the payload stream clean, so a traceback + # from the reader now arrives on its own pipe + assert r.returncode == 0, (f'CDC->Printer reader failed ({size} bytes, rc ' + f'{r.returncode}): {hil_util.cmd_stdout_text(r.stderr)[:200]}') + assert r.stdout == test_data, (f'CDC->Printer wrong data ({size} bytes):\n' + f' expected: {test_data[:64]}\n received: {r.stdout[:64]}') time.sleep(0.2) ser.close() def test_device_mtp(board): + # The whole session lives in mtp_test.py under run_cmd: libmtp calls are synchronous + # ctypes that block unkillably (D state) on a wedged device, so a disposable process is + # the only thing the harness can walk away from. uid = board['uid'] - - # --- BEFORE: mute C-level stderr for libmtp vid/pid warnings --- - fd = sys.stderr.fileno() - _saved = os.dup(fd) - _null = os.open(os.devnull, os.O_WRONLY) - os.dup2(_null, fd) - - mtp = open_mtp_dev(uid) - - # --- AFTER: restore stderr --- - os.dup2(_saved, fd) - os.close(_null) - os.close(_saved) - - if mtp is None or mtp.device is None: - assert False, 'MTP device not found' - - try: - assert b"TinyUSB" == mtp.get_manufacturer(), 'MTP wrong manufacturer' - assert b"MTP Example" == mtp.get_modelname(), 'MTP wrong model' - assert b'1.0' == mtp.get_deviceversion(), 'MTP wrong version' - assert b'TinyUSB MTP' == mtp.get_devicename(), 'MTP wrong device name' - - # read and compare readme.txt and logo.png - f1_expect = b'TinyUSB MTP Filesystem example' - f2_md5_expect = '40ef23fc2891018d41a05d4a0d5f822f' # md5sum of logo.png - f1 = uid.encode("utf-8") + b'_file1' - f2 = uid.encode("utf-8") + b'_file2' - f3 = uid.encode("utf-8") + b'_file3' - mtp.get_file_to_file(1, f1) - with open(f1, 'rb') as file: - f1_data = file.read() - os.remove(f1) - assert f1_data == f1_expect, 'MTP file1 wrong data' - mtp.get_file_to_file(2, f2) - with open(f2, 'rb') as file: - f2_data = file.read() - os.remove(f2) - assert f2_md5_expect == hashlib.md5(f2_data).hexdigest(), 'MTP file2 wrong data' - # test send file - with open(f3, "wb") as file: - f3_data = os.urandom(random.randint(1024, 3*1024)) - file.write(f3_data) - file.close() - fid = mtp.send_file_from_file(f3, b'file3') - f3_readback = f3 + b'_readback' - mtp.get_file_to_file(fid, f3_readback) - with open(f3_readback, 'rb') as f: - f3_rb_data = f.read() - os.remove(f3_readback) - assert f3_rb_data == f3_data, 'MTP file3 wrong data' - os.remove(f3) - mtp.delete_object(fid) - finally: - mtp.disconnect() + script = Path(__file__).resolve().parent / 'mtp_test.py' + # 2x, as master's in-process open_mtp_dev used: libmtp-runtime publishes + # /dev/libmtp-* only after its SYNCHRONOUS mtp-probe finishes, seconds on a freshly + # flashed FS board, and the gio unmount eats part of what is left before the first + # probe. Extracting the session into a subprocess halved this by accident (8s/4s), + # which fails healthy hardware on the retry. + t = 2 * enum_timeout() + r = hil_util.run_cmd( + f'{shlex.quote(sys.executable)} {shlex.quote(str(script))} --uid {shlex.quote(uid)} --timeout {t}', + timeout=t + MTP_SESSION_MARGIN) + if r.returncode == 124: + # "abandoned", not "killed": a session blocked in a usbfs ioctl (D state) never + # receives the SIGKILL -- it lingers until its device path clears, by design + raise AssertionError(f'MTP session wedged (abandoned after {t + MTP_SESSION_MARGIN}s; ' + f'the session process may linger unkillable in D state)') + assert r.returncode == 0, f'MTP session failed (rc {r.returncode}):\n{r.stdout}' def test_device_net_lwip_webserver(board): # MAC hard-coded in examples/device/net_lwip_webserver/src/main.c; Linux names the - # USB network interface enx<MAC_lowercase_no_colons>. Device IP is 192.168.7.1 and - # the example runs an iperf2 TCP server on port 5001 (INCLUDE_IPERF). + # iface enx<MAC_lowercase_no_colons>. Device IP 192.168.7.1, iperf2 TCP server on 5001 + # (INCLUDE_IPERF). import socket mac_no_colons = '0202846a9600' iface = 'enx' + mac_no_colons device_ip = '192.168.7.1' iperf_port = 5001 - # Wait for the host to get an IPv4 address in the device's subnet (DHCP served by the device). - # USB enum + DHCP serve can take longer on the CI HIL hardware than on local — give it 30s. + # Wait for an IPv4 address in the device's subnet (it serves DHCP); 30s because USB + # enum + DHCP serve is slower on the CI HIL hardware than locally. iface_timeout = 30 - deadline = time.time() + iface_timeout + deadline = time.monotonic() + iface_timeout host_ip = None - while time.time() < deadline: + while time.monotonic() < deadline: ret = subprocess.run(['ip', '-o', '-4', 'addr', 'show', iface], capture_output=True, text=True, timeout=2) m = re.search(r'inet (192\.168\.7\.\d+)/', ret.stdout) if ret.returncode == 0 else None @@ -1314,11 +1128,10 @@ def test_device_net_lwip_webserver(board): time.sleep(0.5) assert host_ip, f'USB net iface {iface} did not come up with 192.168.7.x within {iface_timeout}s' - # Poll the iperf TCP port until the device is accepting. The net stack comes up a bit - # after DHCP completes; iperf server binding isn't instantaneous after reflash. - deadline = time.time() + ENUM_TIMEOUT + # Poll until the device accepts: the net stack and the iperf bind come up after DHCP. + deadline = time.monotonic() + enum_timeout() last_err = None - while time.time() < deadline: + while time.monotonic() < deadline: try: with socket.create_connection((device_ip, iperf_port), timeout=1): last_err = None @@ -1326,14 +1139,14 @@ def test_device_net_lwip_webserver(board): except OSError as e: last_err = e time.sleep(0.3) - assert last_err is None, f'iperf TCP {device_ip}:{iperf_port} not accepting within {ENUM_TIMEOUT}s: {last_err}' + assert last_err is None, f'iperf TCP {device_ip}:{iperf_port} not accepting within {enum_timeout()}s: {last_err}' - # Throughput: 5-second iperf2 TCP test, CSV output for stable parsing. - # iperf2 CSV final summary line: timestamp,src_ip,src_port,dst_ip,dst_port,id,interval,bytes,bps - ret = subprocess.run(['iperf', '-c', device_ip, '-t', '5', '-y', 'C'], - capture_output=True, text=True, timeout=30) - stderr = ret.stderr.strip() - stdout = ret.stdout.strip() + # 5-second iperf2 TCP test; -y C for stable parsing (final summary line is + # timestamp,src_ip,src_port,dst_ip,dst_port,id,interval,bytes,bps). + ret = hil_util.run_cmd(f'iperf -c {device_ip} -t 5 -y C', + timeout=30, split_stderr=True, quiet=True) + stderr = (ret.stderr or '').strip() + stdout = (ret.stdout or '').strip() assert ret.returncode == 0, f'iperf rc={ret.returncode}: stderr={stderr!r} stdout={stdout!r}' lines = [l for l in stdout.splitlines() if l] assert lines, f'iperf produced no output (rc={ret.returncode}, stderr={stderr!r})' @@ -1344,19 +1157,16 @@ def test_device_net_lwip_webserver(board): mbps = bps / 1e6 print(f' iperf {mbps:5.1f} Mbps', end='') - # Reject implausibly low throughput - a working USB-net link should clear this easily. assert mbps >= 1.0, f'iperf throughput too low: {mbps:.2f} Mbps' def test_device_msc_dual_lun(board): uid = board['uid'] - # Read README from LUN 0 data0 = read_disk_file(uid, 0, 'README0.TXT') readme0 = b"LUN0: " + MSC_README_TXT assert data0 == readme0, f'MSC LUN0 wrong data in README0.TXT\n expected: {readme0}\n received: {data0}' - # Read README from LUN 1 data1 = read_disk_file(uid, 1, 'README1.TXT') readme1 = b"LUN1: " + MSC_README_TXT assert data1 == readme1, f'MSC LUN1 wrong data in README1.TXT\n expected: {readme1}\n received: {data1}' @@ -1365,8 +1175,7 @@ def test_device_msc_dual_lun(board): def test_device_midi_test(board): uid = board['uid'] - # Find MIDI device via /dev/snd/by-id using board UID - timeout = ENUM_TIMEOUT + timeout = enum_timeout() midi_port = None while timeout > 0: pattern = f'/dev/snd/by-id/usb-*_{uid}-*' @@ -1383,31 +1192,40 @@ def test_device_midi_test(board): timeout -= 1 assert midi_port is not None, f'MIDI device not found for {uid}' - # Read MIDI messages and verify note on/off import select - with open(midi_port, 'rb') as f: - notes = [] + midi_fd = os.open(midi_port, os.O_RDONLY | os.O_NONBLOCK) + try: + data = bytearray() # Read for up to 3 seconds to capture a few notes (286ms interval) - end_time = time.time() + 3 - while time.time() < end_time: - ready, _, _ = select.select([f], [], [], 0.5) - if ready: - data = f.read(64) - if data: - # Parse MIDI bytes: note_on = 0x90, note_off = 0x80 - i = 0 - while i + 2 < len(data): - status = data[i] - if (status & 0xF0) == 0x90: # Note On - notes.append(data[i + 1]) - i += 3 - elif (status & 0xF0) == 0x80: # Note Off - i += 3 - else: - i += 1 + end_time = time.monotonic() + 3 + while (remaining := end_time - time.monotonic()) > 0: + ready, _, _ = select.select([midi_fd], [], [], min(0.5, remaining)) + if not ready: + continue + try: + chunk = os.read(midi_fd, 64) + except BlockingIOError: + continue + if not chunk: + break + data.extend(chunk) + finally: + os.close(midi_fd) + + notes = [] + # Parse MIDI bytes: note_on = 0x90, note_off = 0x80 + i = 0 + while i + 2 < len(data): + status = data[i] + if (status & 0xF0) == 0x90: # Note On + notes.append(data[i + 1]) + i += 3 + elif (status & 0xF0) == 0x80: # Note Off + i += 3 + else: + i += 1 assert len(notes) >= 2, f'Expected at least 2 MIDI notes, got {len(notes)}' - # Verify notes are from the expected sequence note_sequence = [ 74, 78, 81, 86, 90, 93, 98, 102, 57, 61, 66, 69, 73, 78, 81, 85, 88, 92, 97, 100, 97, 92, 88, 85, 81, 78, 74, 69, 66, 62, 57, 62, @@ -1425,7 +1243,7 @@ def test_device_audio_test_freertos(board): return 'skipped' pcm = None - timeout = ENUM_TIMEOUT + timeout = enum_timeout() while timeout > 0: pcm = get_alsa_capture_dev(uid) if pcm: @@ -1448,8 +1266,11 @@ def test_device_audio_test_freertos(board): raw_path, ] - ret = subprocess.run(cmd, capture_output=True, text=True, timeout=20) - assert ret.returncode == 0, f'arecord failed: {ret.stderr.strip() or ret.stdout.strip()}' + # run_cmd: ALSA capture from a wedged device blocks in D state (see read_disk_file) + ret = hil_util.run_cmd(' '.join(shlex.quote(c) for c in cmd), + timeout=20, split_stderr=True, quiet=True) + assert ret.returncode == 0, \ + f'arecord failed: {(ret.stderr or "").strip() or (ret.stdout or "").strip()}' try: with open(raw_path, 'rb') as f: @@ -1467,32 +1288,23 @@ def test_device_audio_test_freertos(board): samples = [int.from_bytes(raw[i:i + 2], 'little', signed=False) for i in range(0, len(raw), 2)] assert sample_count > 1024, f'Not enough samples captured: {sample_count}' - # The firmware sends a continuous uint16 ramp. Using ALSA hw: capture bypasses - # PulseAudio processing, so most adjacent samples should differ by exactly 1. - total_diffs = sample_count - 1 - one_step = 0 - near_step = 0 - for i in range(total_diffs): - d = (samples[i + 1] - samples[i]) & 0xFFFF - if d == 1: - one_step += 1 - if d in (0, 1, 2, 47, 48, 49): - near_step += 1 + # The producer is already running while ALSA activates streaming, so the + # initial overwritable software FIFO (at most 224 samples) can transition + # between ramp generations. After that startup window, require an exact ramp. + startup_samples = 256 + for i in range(startup_samples, sample_count - 1): + expected = (samples[i] + 1) & 0xFFFF + assert samples[i + 1] == expected, ( + f'Audio mismatch at sample {i + 1}: expected {expected}, got {samples[i + 1]}') - one_ratio = one_step / total_diffs - near_ratio = near_step / total_diffs - assert one_ratio >= 0.85, f'Unexpected audio pattern (strict ratio={one_ratio:.3f})' - assert near_ratio >= 0.98, f'Unexpected audio pattern (relaxed ratio={near_ratio:.3f})' - - print(f' ALSA {pcm} strict={one_ratio:.3f} relaxed={near_ratio:.3f}', end='') + print(f' ALSA {pcm}', end='') def test_device_hid_generic_inout(board): uid = board['uid'] import hid # cython-hidapi (pip: hidapi, apt: python3-hid) - # Find HID device by UID (VID=0xCafe) - timeout = ENUM_TIMEOUT + timeout = enum_timeout() dev = None while timeout > 0: for d in hid.enumerate(0xCafe): @@ -1508,7 +1320,6 @@ def test_device_hid_generic_inout(board): h = hid.device() h.open(dev['vendor_id'], dev['product_id'], uid) try: - # Echo test: send random data and verify echo for size in [8, 32, 63]: # Report ID (0) + payload, padded to 64 bytes payload = bytes([random.randint(1, 255) for _ in range(size)]) @@ -1524,60 +1335,186 @@ def test_device_hid_generic_inout(board): h.close() -# ------------------------------------------------------------- -# Main -# ------------------------------------------------------------- -# device tests -# note don't test 2 examples with cdc or 2 msc next to each other -device_tests = [ - # Order matters: cdc_msc and cdc_msc_throughput share the same VID:PID (cafe:4003), so keep a - # differently-PID'd example (dfu, cafe:4000) between them. Boards whose CPU-reset does not drop - # D+ (e.g. WCH CH58x via openocd) only re-enumerate when the PID changes; back-to-back same-PID - # firmware would otherwise leave the host on the previous example's cached descriptors. - 'device/cdc_dual_ports', - 'device/cdc_msc', - 'device/dfu', - 'device/cdc_msc_throughput', - 'device/audio_test_freertos', - 'device/dfu_runtime', - 'device/cdc_msc_freertos', - 'device/hid_boot_interface', - 'device/msc_dual_lun', - 'device/hid_generic_inout', - 'device/printer_to_cdc', - 'device/midi_test', - 'device/mtp', - # 'device/net_lwip_webserver', # disabled for PR #3605: USB net iface enum is flaky on the CI HIL host -] +def test_device_usbtest(board): + global board_wedged + # Runs test/hil/usbtest.py against the cafe:4010 device; the pass count goes in the + # report cell ("✅ 30/30", or "❌ 29/30" on a partial). + uid = board['uid'] + + def usbtest_enumerated(): + """True, False, or None when a bounded read did not answer -- absence unproven.""" + # vid_pid FIRST: right after flashing, the previous example's enumeration (same + # serial, different PID) can linger and would fail usbtest.py's lookup -- and + # filtering on the two lock-free descriptor fields rules out every other device + # on the bus before the one read that can block. usb_scan memoises paths that + # already stranded, so one wedged peer cannot spend the blindness budget here. + devs, unknown = hil_util.usb_scan(vid_pid=('cafe', '4010'), serial=uid) + if devs: + return True + return None if unknown else False -dual_tests = [ - 'dual/host_info_to_device_cdc', -] + end = time.monotonic() + enum_timeout() + seen = usbtest_enumerated() + while time.monotonic() < end and seen is not True: + time.sleep(0.2) + seen = usbtest_enumerated() + # fail before usbtest_permit: an absent device would otherwise queue on the battery + # mutex for minutes behind real batteries just to have usbtest.py report "no device" + if seen is not True: + # 0/30 rather than a bare cell: the battery never ran (30 = standard case count) + raise TestFail( + f'no cafe:4010 device with serial {uid}' if seen is False else + f'cannot tell whether cafe:4010 {uid} is present: the bounded sysfs reads did ' + f'not answer{hil_util.sysfs_blind_note()}', + metric=f'{REPORT_CELL["fail"]} 0/30') + # settle: right after flashing the enumeration can bounce once (and on dual-port parts + # the other port's stale node — same serial and PID — lingers), and testusb run into + # that gap sees the device drop mid-case + time.sleep(3) -host_test = [ - 'host/cdc_msc_hid', - 'host/msc_file_explorer', - 'host/msc_file_explorer_freertos', - 'host/device_info', -] + # --keep-binding is required for concurrent batteries: usbtest.py's cleanup unbinds + # EVERY usbtest-bound interface, killing a peer battery under USBTEST_PARALLEL > 1, and + # that unbind path has also wedged a host xHCI (usb_hcd_alloc_bandwidth) here. Harmless + # to leave: the next example enumerates under a different PID. + script = Path(__file__).resolve().parent / 'usbtest.py' + # --budget makes the battery a real bound: repeated case timeouts (a FAIL, not a HUNG, + # so the battery keeps going) can otherwise spend the whole outer timeout inside the + # case loop, leaving the recovery below nothing. + cmd = (f'{shlex.quote(sys.executable)} {shlex.quote(str(script))} ' + f'--serial {shlex.quote(uid)} --json --keep-binding ' + f'--timeout 60 --budget {USBTEST_BATTERY_BUDGET}') + # Post-hang recovery reflashes the DUT through its own probe, NEVER a root-port cycle + # (one board reached instead of every fixture under the port; see usb-kernel-recover). + # _current_fw is the artifact test_example flashed for THIS test: re-deriving it from + # board['name'] reflashes the wrong build on variant-only boards. --outer-timeout lets + # usbtest skip a reflash it cannot finish before our run_cmd kill, which would orphan + # the flasher (own session) on the probe. Never under --skip-flash -- and say so: a + # HUNG case then holds the DUT's usbfs lock for the rest of the run, and a probe reset + # is no substitute (the DWC2 pullup survives a core halt). + # ...and only when this flasher can DELIVER that reflash past a poisoned node + # (hil_flash.convoy_safe). Otherwise the flags cost twice: the delivery adds a SECOND + # stray, and the board reserves recovery budget for a path that cannot fire. + # The RECOVERY flasher, which may be the roster's optional `flasher_recover` rather + # than the primary -- a jlink/stlink board can name an openocd entry that reaches the + # same probe convoy-safely without changing how the board is normally flashed. + _rec_flasher = hil_flash.recover_flasher(board) + recovery = bool(_current_fw and not skip_flash and hil_flash.convoy_safe(_rec_flasher)) + # ONE bound, computed here and used for BOTH the child's --outer-timeout and our own + # run_cmd kill below. Three separate expressions disagreed: --skip-flash appended no + # --outer-timeout at all (usbtest reads 0 as "no limit"), and the no-recovery branch + # narrowed only the CHILD's view while run_cmd still waited the full reserve -- so a + # board that cannot recover held a pool worker AND its battery permit idle for + # USBTEST_RECOVERY_BUDGET it had no way to spend, under a usbtest width of 2. + outer = USBTEST_BATTERY_BUDGET + (USBTEST_RECOVERY_BUDGET if recovery + else USBTEST_OVERSHOOT) + if _current_fw and skip_flash: + print('note: --skip-flash disables usbtest hang recovery; a HUNG case will leave ' + 'the device wedged until it is reflashed', flush=True) + elif _current_fw and not recovery: + print(f'note: {_rec_flasher["name"]} cannot deliver a reflash past a poisoned ' + f'usbfs node, so usbtest hang recovery is disabled for {board["name"]}; a ' + f'HUNG case will leave it wedged for the rest of the run', flush=True) + if recovery: + # ship the RECOVERY flasher as `flasher`: usbtest.py, recovery_steps and + # convoy_safe all read board['flasher'], so substituting here keeps the entire + # child side unaware that a second roster entry exists + rb = json.dumps({'name': board['name'], 'flasher': _rec_flasher}) + cmd += f' --recover-board {shlex.quote(rb)} --recover-fw {shlex.quote(_current_fw)}' + cmd += f' --outer-timeout {outer}' + # The reserve above USBTEST_BATTERY_BUDGET exists because the battery can overrun by + # one already-started case, and a hang there needs room for the recovery (whose reflash + # is bounded by usbtest.RECOVER_FLASH_TIMEOUT, not HIL_CMD_TIMEOUT). Without it run_cmd + # SIGKILLs usbtest.py mid-recovery, losing the JSON and the diagnosis. + with hil_lock.usbtest_permit(uid): + # split_stderr: the battery's final JSON is parsed from stdout, and stderr is the + # only detail left when the outer timeout kills the battery before it prints + r = hil_util.run_cmd(cmd, timeout=outer, split_stderr=True) + out = hil_util.cmd_stdout_text(r.stdout) + brace = out.find('{') + try: + # brace < 0 would slice from the END ('...rc 0' -> '0' -> int 0, whose subscript + # raises TypeError outside the tuple below and loses the diagnosis) + if brace < 0: + raise ValueError('no JSON object on stdout') + data = json.loads(out[brace:]) + passed, failed = int(data['passed']), int(data['failed']) + except (ValueError, KeyError, TypeError, json.JSONDecodeError): + # compact BOTH, never `or`: a battery SIGKILLed mid-print leaves a truthy JSON + # fragment on stdout, so an `or` drops the stderr that explains the failure + parts = [compact_output(hil_util.cmd_stdout_text(r.stderr)), compact_output(out)] + detail = ' | '.join(p for p in parts if p) + # Retryable even on rc 124 (run_cmd's outer kill), though the retry re-pays the + # whole budget: 124 only says the timer expired, which a healthy battery can hit + # under load, and test_example REFLASHES before each attempt. Where usbtest's + # in-band recovery is off (--skip-flash, a flasher failing convoy_safe, a terminal + # wedge) that reflash is the only thing left to unpoison the DUT for the boards + # that share its controller. + # No JSON to read the verdict from, so fall back to the text: a battery SIGKILLed + # mid-hang still says HUNG on stdout, and this raise happens BEFORE the latch below + # -- which is why the outer-timeout case, the likeliest real wedge, never latched. + if 'HUNG' in out: + board_wedged = (f'{board["name"]}: usbtest reported a hang and was killed ' + f'before it could report a verdict') + raise TestFail(f'usbtest did not run: {detail}', + metric=f'{REPORT_CELL["fail"]} 0/30') + # A HUNG case that recovery could not clear leaves a D-state holder on this board's + # usbfs node. Latch it: the remaining examples would each flash THROUGH that node, + # block, survive SIGKILL and add another stray -- turning one wedge into one stray per + # remaining example, which is the convoy this branch exists to contain. + # The battery's OWN verdict first: `recovery` only says the flags were passed, not that + # the reflash worked, so a convoy-safe board whose recovery failed used to come back + # unlatched and flash every remaining example through the poisoned node. + if data.get('wedged') or (not recovery and 'HUNG' in out): + # _rec_flasher, NOT board['flasher']: recovery was decided against recover_flasher() + # at the top of this function, and the two diverge as soon as a roster carries the + # optional `flasher_recover` key -- naming the wrong one sends the operator to the + # wrong probe. The wording stays on what usbtest actually reported ("still wedged"), + # because unrecovered_hang is also set by the ambiguous/inconclusive aborts, where + # nothing hung and the old text was false on both clauses. + board_wedged = (f'{board["name"]}: usbtest reports the device still wedged ' + + (f'after a recovery reflash via {_rec_flasher["name"]}' if recovery + else f'and {_rec_flasher["name"]} cannot deliver a recovery reflash')) -def find_firmware(variant: str, example: str): - """Locate a built example's firmware base path (no extension) under - cmake-build-<variant>/<example>/. Accepts the single-config layout (firmware - directly in the example dir) or Ninja Multi-Config (a per-config subdir like - RelWithDebInfo/). Returns the base Path, or None if not built.""" - fw_dir = TINYUSB_ROOT / build_dir / f'cmake-build-{variant}' / example - base = Path(example).name - if fw_dir.is_dir(): - for cand in [fw_dir / base, fw_dir / 'RelWithDebInfo' / base, - *(p.with_suffix('') for p in sorted(fw_dir.glob(f'*/{base}.elf')))]: - if cand.with_suffix('.elf').exists() or cand.with_suffix('.bin').exists(): - return cand - return None + # notrun counts toward the denominator but is NOT a failure: listing cases that never + # ran as failures sends a maintainer bisecting one of them. + notrun = int(data.get('notrun', 0)) + total = passed + failed + notrun + if board_wedged and failed == 0 and notrun == 0: + # Every case passed and the device STILL wedged -- usbtest's inconclusive/ambiguous + # abort fires after the last case, so nothing back-fills a BUDGET entry. Reporting + # the pass would exit 0 with a D-state holder on the rig and the board absent from + # the re-run spec. parsed=True: a retry re-pays the whole battery to re-observe a + # wedge, and flashes through the poisoned node to do it. + raise TestFail(f'usbtest {passed}/{total} but the device wedged ({board_wedged})', + metric=f'{REPORT_CELL["fail"]} {passed}/{total}', parsed=True) + if failed == 0 and notrun == 0 and total > 0: + return f'{REPORT_CELL["pass"]} {passed}/{total}' + bad = [c.get('num') for c in data.get('cases', []) + if c.get('status') not in ('PASS', 'BUDGET')] + why = f'usbtest {passed}/{total}' + if bad: + why += f' (cases failed: {bad})' + if notrun: + # the reason is per BUDGET entry: a hang or a device drop also aborts the battery, + # and blaming the budget points the maintainer at the wrong thing + reasons = {c.get('detail', '') for c in data.get('cases', []) + if c.get('status') == 'BUDGET'} + reason = (reasons.pop().replace('not run: ', '') if len(reasons) == 1 + else 'the battery stopped early') + why += f'; {notrun} case(s) never ran ({reason}), so this says nothing about them' + # parsed ONLY when every case ran: an aborted battery (budget expiry, kernel hang, bus + # drop) leaves BUDGET entries, and those are exactly what a reflash retry can fix. + raise TestFail(why, metric=f'{REPORT_CELL["fail"]} {passed}/{total}', + parsed=(notrun == 0)) -def test_example(board: Board, variant: str, example: str) -> tuple[int, str]: +# ------------------------------------------------------------- +# Main +# ------------------------------------------------------------- + + +def test_example(board: Board, variant: str, example: str) -> tuple[int, str, str | None]: """ Test example firmware :param board: board dict @@ -1594,26 +1531,69 @@ def test_example(board: Board, variant: str, example: str) -> tuple[int, str]: test_name = f'{variant:40} {example:30} ...' - fw_name = find_firmware(variant, example) + # --skip-flash runs whatever is already on the board, so any build counts as present; + # filtering by flasher there would skip the test over an extension it never uses. + fw_name = hil_flash.find_firmware(variant, example, + flasher=None if skip_flash else board['flasher']['name']) if fw_name is None: log_line(f'{test_name} Skip (no binary)') return 0, 'skip', None + # usbtest's hang recovery reflashes the exact artifact under test; re-deriving it from + # board['name'] breaks on variant-only boards + global _current_fw + _current_fw = str(fw_name) if verbose: - log_line(f'Flashing {fw_name}.elf') + log_line(f'Firmware {fw_name}') - # flash firmware (unless --skip-flash), then run the test. Both may fail randomly, - # retry a few times. + global _enum_timeout start_s = time.time() flash_ok = True last_err = '' last_detail = '' + wedge_break = False for i in range(max_retry): + if board_wedged and i: + # The latch is set MID-attempt (a HUNG usbtest whose flasher cannot recover), + # so test_board's check between tests is too late for THIS test's own retries: + # every further attempt re-flashes into the D-state-held node, blocks, survives + # SIGKILL and leaves another stray. The wedge is not something a retry can fix. + log_line(f'{test_name} not retrying: {board_wedged}') + # COUNT it. Breaking out here skips the i == max_retry - 1 branch that would + # have incremented err_count, so the board rendered a red cell, contributed 0 + # to the exit status and was omitted from the re-run spec -- a rig left with a + # D-state holder published under sys.exit(0). Latent at CI's --retry 1, live + # for every local run and for the workflows that pass no -r. + wedge_break = True + break + _enum_timeout = ENUM_TIMEOUT if i == 0 else ENUM_TIMEOUT_RETRY attempt_out = io.StringIO() with redirect_stdout(attempt_out): if not skip_flash: - ret = globals()[f'flash_{board["flasher"]["name"].lower()}'](board, str(fw_name)) - flash_ok = (ret.returncode == 0) + with hil_lock.flash_permit(board['uid']): + t_flash = time.monotonic() + try: + ret = getattr(hil_flash, + f'flash_{board["flasher"]["name"].lower()}')(board, str(fw_name)) + except Exception as e: + # A flasher that RAISES (esptool's get_serial_dev when the adapter + # drops off the bus, a missing config.env, an unwritable CWD) would + # propagate out of the worker and abort the whole drain, costing + # every board still in flight. + print(f'flash raised: {type(e).__name__}: {e}', flush=True) + ret = subprocess.CompletedProcess(args='flash', returncode=1, + stdout=f'{type(e).__name__}: {e}') + if PROFILE: + log_line(f'[prof] {variant} {example} flash attempt {i + 1}: ' + f'{time.monotonic() - t_flash:.1f}s rc={ret.returncode}') + flash_ok = (ret.returncode == 0) + # A wedged RP2040/RP2350 DAP answers nothing and the probe has no + # reset line, so the retry fails identically; POR it via the Rescue DP + # first (no-op otherwise). NOT gated on a remaining attempt: CI HIL jobs + # run --retry 1, and this leaves the DAP POR'd for the jobs that follow. + if not flash_ok and \ + hil_flash.rescue_openocd(board, hil_util.cmd_stdout_text(ret.stdout)): + log_line(f'{variant} {example}: DAP wedged, rescued via Rescue DP') if flash_ok: try: tret = globals()[f'test_{example.replace("/", "_")}'](board) @@ -1624,7 +1604,6 @@ def test_example(board: Board, variant: str, example: str) -> tuple[int, str]: else: status = STATUS_OK result_status = 'pass' - # a test may return a string to show in its report cell (e.g. speed) metric = tret if isinstance(tret, str) else None msg = f'{test_name} {status}' if last_detail: @@ -1635,8 +1614,21 @@ def test_example(board: Board, variant: str, example: str) -> tuple[int, str]: except Exception as e: last_err = str(e) last_detail = compact_output(attempt_out.getvalue()) + if getattr(e, 'parsed', False): + # a PARSED per-case result (usbtest's "29/30"): retrying re-pays + # the whole battery, inside the fleet's usbtest permit, to + # re-observe a number the JSON already reported. Only that case. + err_count += 1 + metric = getattr(e, 'metric', None) + msg = f'{test_name} {STATUS_FAILED}: {e}' + if last_detail: + msg += f' {last_detail}' + msg += f' in {time.time() - start_s:.1f}s' + log_line(msg) + break if i == max_retry - 1: err_count += 1 + metric = getattr(e, 'metric', None) msg = f'{test_name} {STATUS_FAILED}: {e}' if last_detail: msg += f' {last_detail}' @@ -1668,13 +1660,21 @@ def test_example(board: Board, variant: str, example: str) -> tuple[int, str]: msg += f' in {time.time() - start_s:.1f}s' log_line(msg) + if wedge_break and not err_count: + # ONE error for the test, never two: a board that also failed to flash has already + # been counted just above. Without this the test returns 0 -- red cell, clean exit + # status, absent from the re-run spec. + err_count += 1 return err_count, result_status, metric def build_board(board: Board) -> tuple[str, int]: """Build firmware for this board via tools/build.py. Honors board config's variant list and build.args defines. - Output goes to cmake-build/cmake-build-<variant>/ (tools/build.py layout).""" + Output goes to cmake-build/cmake-build-<variant>/ (tools/build.py layout). + + Unbounded on purpose: --build is a local convenience (no CI workflow passes it), so + the developer watching the build is the timeout.""" name = board['name'] bcfg = cast(BuildCfg, board.get('build', {})) extra_defs = bcfg.get('args', []) @@ -1682,7 +1682,7 @@ def build_board(board: Board) -> tuple[str, int]: failed = 0 for v in variants: - cmd = [sys.executable, str(TINYUSB_ROOT / 'tools' / 'build.py'), '-b', name] + cmd = [sys.executable, str(hil_util.TINYUSB_ROOT / 'tools' / 'build.py'), '-b', name] for d in extra_defs: cmd += ['-D', d] if v['name'] != name: @@ -1694,33 +1694,51 @@ def build_board(board: Board) -> tuple[str, int]: if verbose: cmd.append('-v') print(f' + {" ".join(cmd)}') - r = subprocess.run(cmd, cwd=TINYUSB_ROOT) - if r.returncode != 0: + # stdio is inherited so the build STREAMS: a silent buffer is + # indistinguishable from a stall. + proc = subprocess.Popen(cmd, cwd=hil_util.TINYUSB_ROOT, start_new_session=True) + try: + rc = proc.wait() + except KeyboardInterrupt: + # start_new_session means the build never saw the terminal's SIGINT + try: + os.killpg(proc.pid, signal.SIGKILL) + except OSError: + proc.kill() + raise + if rc != 0: failed += 1 return name, failed -def test_board(board: Board) -> tuple[str, int, list[str], list]: +# pseudo-test column for a variant boundary the park-flash could not clear (see below) +BOUNDARY_CELL = 'same-PID boundary' + + +def test_board(board: Board) -> tuple[str, int, list[str], list, float]: + swept = False name = board['name'] flasher = board['flasher'] + global board_wedged + board_wedged = '' try: - _lock_fh = acquire_board_lock(name) + _lock_fh = hil_lock.acquire_board_lock(name) except RuntimeError as e: log_line(f'{name:25} {STATUS_FAILED}: {e}') - # visible report row so the ❌ matches the exit code; failed-tests stays - # empty so a re-run repeats the whole board (no bogus -bt test filter) - return name, 1, [], [(name, {'board-locked': 'fail'})] + # visible report row so the ❌ matches the exit code; failed-tests stays empty so a + # re-run repeats the whole board (no bogus -bt filter) + return name, 1, [], [(name, {'board-locked': 'fail'}, None)], 0.0 + # after the lock: flock wait behind a concurrent run is not board cost + t_board = time.monotonic() try: - # default to all tests test_list = [] if name in board_test: test_list = board_test[name] elif len(test_only) > 0: - # Explicit -t: filter against the board's capabilities so a device-only - # board doesn't try to run host/dual tests (the test functions need a - # `dev_attached` entry in the board config that won't exist). + # Explicit -t: filter against the board's capabilities, or a device-only board + # runs host/dual tests whose `dev_attached` config entry does not exist. board_tests = board.get('tests', {}) if 'only' in board_tests: allowed = set(board_tests['only']) @@ -1749,33 +1767,132 @@ def test_board(board: Board) -> tuple[str, int, list[str], list]: err_count = 0 failed_tests = [] - rows = [] # list of (row_label, {example: status}) — one row per build variant + board_wide_fail = False # re-run the whole board, not a subset of its tests + rows = [] # list of (row_label, {example: status}, duration) — one per build variant + # a -t/-bt filtered run times only a subset; report no duration so an accumulate + # re-run keeps the previous full-run value + partial = bool(test_only) or name in board_test variants = board.get('variant') or [{'name': name, 'flags': ''}] + prev_last = None # last test of the previous variant: the variant boundary is an adjacency too for v in variants: vname = v['name'] + # Shuffle each (board, variant)'s run order: spreads batteries and flash churn + # across the timeline instead of convoying, and surfaces order-dependent bugs. + # Seeded for replay (HIL_SHUFFLE_SEED). Unique per-example PIDs re-enumerate + # between examples; only the variant boundary can repeat one. + run_list = list(test_list) + if shuffle_seed is not None and len(run_list) > 1: + random.Random(f'{shuffle_seed}:{name}:{vname}').shuffle(run_list) + if run_list[0] == prev_last: + run_list[0], run_list[-1] = run_list[-1], run_list[0] cells = {} - for test in test_list: + if run_list and run_list[0] == prev_last and not skip_flash: + # Same example (same PID) still repeats across the boundary (a one-test + # -bt run has nothing to swap with). Park on board_test first: it disables + # the board's USB, so the next flash must re-enumerate to be seen. + t_park = time.monotonic() + # _should_park, same as the teardown park: this is attempt 0, so + # test_example's retry guard does not stop it flashing into a poisoned node + park_ec, park_status, _ = ( + test_example(board, vname, 'device/board_test') if _should_park(skip_flash) + else (0, 'skip', None)) + if park_ec or park_status == 'skip': + # Boundary not cleared: the previous variant may still be enumerated + # under the same PID, so this variant's tests could pass against ITS + # firmware. Skip them and record the boundary as the failure, so the + # report matches the exit code instead of rendering all-green. + # A 'skip' here has two very different causes: no board_test build, or + # _should_park refusing to flash a WEDGED board. Reporting the latter as + # a missing binary sends the operator hunting a build that exists. + wedge_skip = park_status == 'skip' and bool(board_wedged) + why = ('the board is wedged' if wedge_skip else + 'no board_test binary' if park_status == 'skip' else + 'park flash failed') + log_line(f'{vname:40} {"same-PID boundary":30} {STATUS_FAILED}: not cleared ({why}); ' + f'skipping {len(run_list)} test(s) on this variant') + # the wedge already charged its own error through test_device_usbtest; + # charging again would double-count one incident in the exit code + if not wedge_skip: + err_count += 1 + cells[BOUNDARY_CELL] = 'fail' + # blaming run_list[0] would re-run an innocent test that then passes, + # leaving the boundary unretested; re-run the whole board instead + board_wide_fail = True + # leave prev_last alone: the board still holds the previous variant's + # firmware, so the next variant must attempt the park again + run_list = [] + t_board += time.monotonic() - t_park # park is teardown, not board cost + if run_list: + prev_last = run_list[-1] + t_variant = time.monotonic() + for test in run_list: + if board_wedged: + # Do NOT flash through a poisoned node: each attempt enumerates into + # it, blocks uninterruptibly and leaves another stray behind. Report + # the skip so the cell is not mistaken for a pass. + cells[test] = f'{REPORT_CELL["skip"]} board wedged' + # ...and re-run the WHOLE board, like the boundary-failure path above: + # these tests never executed, so naming them individually in the .failed + # spec is not enough -- an --accumulate re-run that fixes only the wedged + # test would merge a green cell over it and leave these skips standing + # from the earlier attempt, forever, under a green job. + board_wide_fail = True + continue ec, status, metric = test_example(board, vname, test) err_count += ec cells[test] = metric if metric else status if ec > 0: failed_tests.append(test) - rows.append((vname, cells)) + if board_wedged: + log_line(f'{vname:40} SKIPPING the rest of this board: {board_wedged}; ' + f'flashing through the poisoned node would add a stray per test') + dur = f'{time.monotonic() - t_variant:.0f}s' if run_list and not partial else None + rows.append((vname, cells, dur)) - # flash board_test last to disable board's usb (skipped when --skip-flash is set); - # this is teardown/park, not a test — not recorded in the report - if not skip_flash: + # excludes the teardown park-flash below; a partial (filtered) run reports 0.0 so + # it never overwrites a cached full-run duration + t_total = 0.0 if partial else time.monotonic() - t_board + + # park: flash board_test last to disable the board's usb; teardown, not a test, + # so it is not recorded in the report. + # + # NOT on a wedged board: the latch has just skipped every remaining test precisely + # because flashing through a D-state-held node blocks, survives SIGKILL and leaves + # a stray -- and this park is a flash like any other. test_example's own guard does + # not stop it (that one only suppresses RETRIES, and this is attempt 0), so the + # containment path would add the very stray it exists to prevent. + if _should_park(skip_flash): test_example(board, variants[0]['name'], 'device/board_test') - return name, err_count, sorted(set(failed_tests)), rows + # Sweep HERE, not in main()'s finally: maxtasksperchild=1 retires this process as + # soon as it returns, reparenting anything it spawned to init and off the pool's + # ppid tree, so the main-side sweep walks fresh idle workers and finds nothing. + # Measured: 4 tasks, zero overlap, sweep 0, all 4 strays alive. + stray = hil_health.kill_own_children() + swept = True + + # LAST fields: whether this worker ran out of bounded-read budget, and what it could + # not kill. Only the worker can answer either -- the blindness latch is + # process-global and this is a separate process -- and the result tuple already + # crosses back, so no Manager round-trip. + return (name, err_count, [] if board_wide_fail else sorted(set(failed_tests)), + rows, t_total, hil_util.sysfs_blind(), stray) finally: + # A raise skips the sweep above, and maxtasksperchild=1 retires this process + # immediately afterwards -- reparenting its flasher to init and erasing the ppid + # link, so main's sweep cannot see it either. The count cannot reach the report on + # this path (there is no result tuple), but the KILL still frees the probe. + if not swept: + try: + hil_health.kill_own_children() + except Exception as se: # noqa: BLE001 - never mask the original failure + print(f'warning: stray sweep failed: {type(se).__name__}: {se}', flush=True) if _lock_fh: try: - # clear our pid record before dropping the flock: this worker - # process lives on (pool reuse), so a stale record would make - # board_lock.py's pid-liveness checks report a freed board as - # still locked for the rest of the run + # clear our pid record before dropping the flock: this worker process + # lives on (pool reuse), so a stale record would make hil_lock's + # pid-liveness checks report a freed board as locked for the rest of the run _lock_fh.truncate(0) except OSError: pass @@ -1784,22 +1901,43 @@ def test_board(board: Board) -> tuple[str, int, list[str], list]: REPORT_MD = 'hil_report.md' REPORT_JSON = 'hil_report.json' +# controller hints from previous runs: uid -> {'name', 'pci', 'duration'}. Only 'pci' is +# consumed (dispatch order and first-flash budgeting, never battery serialization). PCI +# addresses are boot-stable, so the cache survives reboots and goes stale on re-cabling. +CONTROLLER_CACHE = Path.home() / '.cache' / 'tinyusb-hil' / 'controller_cache.json' + + +def schedule_boards(boards: list, pci_of_uid: dict) -> list: + """Dispatch order: round-robin across host controllers so every controller's + serialized usbtest battery chain is fed from t=0 instead of one card's boards + convoying at the head of the queue. Boards without a controller hint form their + own bucket; config order is kept within a bucket.""" + buckets = {} + for b in boards: + buckets.setdefault(pci_of_uid.get(b['uid'], '?'), []).append(b) + return [b for grp in itertools.zip_longest(*buckets.values()) for b in grp if b is not None] def render_matrix(rows_all: list) -> str: - """Render rows (list of (row_label, {example: status})) as an aligned markdown - matrix: columns = tests (bare names) centered, boards left-aligned.""" - canonical = device_tests + dual_tests + host_test + """Render rows (list of (row_label, {example: status}, duration)) as an aligned + markdown matrix: columns = tests (bare names) centered, boards left-aligned, + per-row duration as the trailing column.""" seen = set() - for _, cells in rows_all: + for _, cells, _ in rows_all: seen.update(cells) if not seen: return 'No tests were run.' - # columns: canonical order first, then any extras (e.g. from -t) alphabetically - columns = [t for t in canonical if t in seen] - columns += [t for t in sorted(seen) if t not in canonical] - headers = [c.rsplit('/', 1)[-1] for c in columns] # bare example name + # metric-bearing columns pinned first, the rest alphabetical: stable regardless of the + # shuffled execution order + pinned = ['usbtest', 'cdc_msc_throughput', 'msc_file_explorer', 'msc_file_explorer_freertos'] + + def col_key(t): + name = t.rsplit('/', 1)[-1] + return (pinned.index(name) if name in pinned else len(pinned), name, t) + + columns = sorted(seen, key=col_key) + headers = [c.rsplit('/', 1)[-1] for c in columns] + ['duration'] # bare example names def cell(cells, col): v = cells.get(col) @@ -1807,10 +1945,12 @@ def render_matrix(rows_all: list) -> str: return '' return REPORT_CELL.get(v, v) # status symbol, or a metric string (e.g. speed) verbatim + rows_vals = [(lbl, [cell(cells, c) for c in columns] + [dur or '']) + for lbl, cells, dur in rows_all] board_hdr = 'Board' - board_w = max([len(board_hdr)] + [len(lbl) for lbl, _ in rows_all]) - col_w = [max([len(h)] + [len(cell(cells, c)) for _, cells in rows_all]) - for h, c in zip(headers, columns)] + board_w = max([len(board_hdr)] + [len(lbl) for lbl, _ in rows_vals]) + col_w = [max([len(h)] + [len(vals[i]) for _, vals in rows_vals]) + for i, h in enumerate(headers)] def line(label, values): padded = [label.ljust(board_w)] + [v.center(w) for v, w in zip(values, col_w)] @@ -1818,56 +1958,280 @@ def render_matrix(rows_all: list) -> str: header = line(board_hdr, headers) sep = '| ' + '-' * board_w + ' | ' + ' | '.join(':' + '-' * (w - 2) + ':' for w in col_w) + ' |' - body = [line(lbl, [cell(cells, c) for c in columns]) for lbl, cells in rows_all] + body = [line(lbl, vals) for lbl, vals in rows_vals] - # tally run cells (blank/not-run cells are absent from the dicts); a metric string counts as pass - failed = sum(v == 'fail' for _, cells in rows_all for v in cells.values()) - skipped = sum(v == 'skip' for _, cells in rows_all for v in cells.values()) - passed = sum(v not in ('fail', 'skip') for _, cells in rows_all for v in cells.values()) + # tally run cells (not-run cells are absent from the dicts). A cell is a bare status or + # a metric string carrying its own icon ("❌ 29/30"), so classify by the leading icon. + def cell_kind(v): + if v == 'fail' or (isinstance(v, str) and v.startswith(REPORT_CELL['fail'])): + return 'fail' + if v == 'skip' or (isinstance(v, str) and v.startswith(REPORT_CELL['skip'])): + return 'skip' + return 'pass' + kinds = [cell_kind(v) for _, cells, _ in rows_all for v in cells.values()] + failed = kinds.count('fail') + skipped = kinds.count('skip') + passed = kinds.count('pass') summary = (f'**{REPORT_CELL["pass"]} {passed} passed · {REPORT_CELL["fail"]} {failed} failed · ' f'{REPORT_CELL["skip"]} {skipped} skipped · blank not run**') return summary + '\n\n' + '\n'.join([header, sep] + body) -def accumulate_report(mret: list, report_dir: Path, fresh: bool) -> str: +def _write_failed_spec(failed_fname: Path, report_dir: Path, mret: list) -> None: + """Re-run spec: only the failed boards (-b), each restricted to its own failed tests + (-bt); a board with failures but no test list re-runs entirely. + + Shared with the pool-guard path, which feeds it the boards that never reported. That + path used to leave this unwritten -- and a fresh run has already unlinked it -- so + build.yml's "Get re-run spec" step found nothing and the GitHub re-run repeated the + whole fleet to find the one board that wedged.""" + parts = ['--accumulate'] + for name, err, fts, *_ in mret: + if err > 0: + parts.append(f'-b {name}') + if fts: + parts.append(f'-bt {name}:{",".join(fts)}') + if len(parts) > 1: # build-only failures have no boards to re-run + report_dir.mkdir(parents=True, exist_ok=True) + with failed_fname.open('w') as f: + f.write(' '.join(parts)) + else: + failed_fname.unlink(missing_ok=True) + + +class PoolDrainTimeout(MpTimeoutError): + """Guard expiry, carrying the rows that DID finish. + + They ride on the exception because the raise is the containment path: losing them here + is what map_async did, and what the drain exists to stop. + """ + + def __init__(self, finished: list): + super().__init__() + self.finished = finished + + +def drain_pool(it, boards: list, deadline: float, out: list | None = None) -> list: + """Collect imap_unordered results against ONE deadline. Returns the finished rows. + + Raises PoolDrainTimeout (carrying those same rows) when the deadline passes with boards + still in flight -- the caller keeps them, names only what is missing, and writes a + re-run spec covering just those. + + A function, not an inline loop, so the tests can call THIS instead of a copy of it: the + loop's previous test built its own ThreadPool and its own drain and asserted on those, + so deleting the real one outright kept the suite green. + """ + # `out` is the CALLER's list: a worker that raises something other than a timeout + # (get_serial_dev on a dropped adapter, a Manager EOFError) propagates bare, and a + # local accumulator would take every finished board with it -- the exact loss the + # drain replaced map_async to prevent. + mret: list = out if out is not None else [] + for _ in boards: + left = deadline - time.monotonic() + if left <= 0: + raise PoolDrainTimeout(mret) + try: + mret.append(it.next(timeout=left)) + except MpTimeoutError: + raise PoolDrainTimeout(mret) from None + return mret + + +def _should_park(skip_flash: bool) -> bool: + """Flash the teardown park (device/board_test, to switch the DUT's USB off)? + + Not on a wedged board. The latch has just skipped every remaining test precisely + because flashing through a D-state-held node blocks, survives SIGKILL and leaves a + stray -- and the park is a flash like any other. test_example's own guard does not stop + it either: that one only suppresses RETRIES, and the park is always attempt 0. So the + containment path would end by adding the very stray it exists to prevent. + """ + return not skip_flash and not board_wedged + + +def _stray_note(mret: list) -> str: + """Name the strays the workers could not kill, for the report banner. + + Summed from the result tuples rather than computed in main()'s finally: that finally + runs AFTER accumulate_report on both abort paths, so a banner appended there was + written to a variable nobody read again. + """ + dirty = [(r[0], r[6]) for r in mret if len(r) > 6 and r[6]] + if not dirty: + return '' + total = sum(n for _, n in dirty) + return (f'> **Rig dirty.** {total} process(es) survived SIGKILL and still hold a probe ' + f'or usbfs node into the next job: ' + f'{", ".join(f"{b} ({n})" for b, n in dirty)}.\n') + + +def _blind_note(mret: list) -> str: + """Name the boards whose worker went blind, for the report banner. + + A blind worker answers SYSFS_UNKNOWN for every attribute, so its "device not found" is + "could not tell". That already reaches the log and the per-cell failure text, but the + TABLE is what gets quoted -- and a red cell there is read as a broken board. Seen live + (run 31794359407): four workers blind, several cells red because of it, and a report + that said nothing. + + Per-board, not global: maxtasksperchild=1 gives every board a fresh worker, so a board + that ran on a healthy one is not smeared by a neighbour's wedge. Rows synthesised by + the timeout path are 5 fields wide and have nothing to report. + """ + blind = [r[0] for r in mret if len(r) > 5 and r[5]] + if not blind: + return '' + return (f'> **Not all verdicts are evidence.** {len(blind)} board(s) ran on a worker ' + f'that went blind on sysfs -- too many bounded reads stranded on a wedged ' + f'device -- so "not found" from them means "could not tell": ' + f'{", ".join(blind)}. See the usb-kernel-recover skill.\n') + + +def accumulate_report(mret: list, report_dir: Path, fresh: bool, scope: str = '', + banner: str = '') -> str: """Merge this run's results into hil_report.json in report_dir, then (re)write - the markdown matrix to hil_report.md. `fresh` (a full run, no --skip-board/-bt) + the markdown matrix to hil_report.md. `fresh` (a first run, no --accumulate) starts a new report; otherwise a re-run accumulates so boards/tests that - already passed are preserved while re-run cells are updated. Returns the md.""" - acc = {} # ordered {row_label: {example: status}} + already passed are preserved while re-run cells are updated. `scope` names the + board filter, if any, so a scoped table is not mistaken for a full one. + Returns the md.""" + acc = {} # ordered {row_label: [cells dict, duration str|None]} + prior_banner = '' jpath = report_dir / REPORT_JSON if not fresh and jpath.is_file(): try: - for entry in json.loads(jpath.read_text()).get('rows', []): - acc[entry['board']] = dict(entry['cells']) + saved = json.loads(jpath.read_text()) + # CI keys the report dir by run id, so the sidecar is from an earlier attempt + for entry in saved.get('rows', []): + acc[entry['board']] = [dict(entry['cells']), entry.get('duration')] + # ... and so is the caveat those cells were collected under. A rerun on a rig + # that has since recovered contributes no banner, and the .failed spec reruns + # only FAILURES -- so the earlier attempt's passes are never re-earned and + # would be published as clean results of a rig that was not. + prior_banner = saved.get('banner', '') except (ValueError, KeyError, TypeError): pass # corrupt/old sidecar: start fresh - # merge this run: current cells override prior for boards/tests that ran - for name, _, _, rows in mret: - if rows and not any('board-locked' in cells for _, cells in rows): - # board ran for real this time: clear a stale lock-failure cell - # (its row is keyed by board name; test rows may be variant names) + # current cells override prior for boards/tests that ran; a filtered run reports + # duration None, keeping the previous full-run value + for name, _, _, rows, *_ in mret: + if rows and not any('board-locked' in cells for _, cells, _ in rows): + # board ran for real: clear a stale lock-failure cell (its row is keyed by + # board name; test rows may be variant names) stale = acc.get(name) if stale is not None: - stale.pop('board-locked', None) - if not stale: - # variant-keyed boards never repopulate the board-name row — - # drop it or it renders as a blank ghost row + stale[0].pop('board-locked', None) + if not stale[0]: + # variant-keyed boards never repopulate the board-name row, so drop it + # or it renders as a blank ghost row del acc[name] - for row_label, cells in rows: - acc.setdefault(row_label, {}).update(cells) + for row_label, cells, dur in rows: + row = acc.setdefault(row_label, [{}, None]) + # the boundary cell is only ever written on failure, so a re-run of this + # variant that cleared the boundary must drop the previous attempt's ❌ + if BOUNDARY_CELL not in cells: + row[0].pop(BOUNDARY_CELL, None) + row[0].update(cells) + if dur is not None: + row[1] = dur report_dir.mkdir(parents=True, exist_ok=True) - jpath.write_text(json.dumps({'rows': [{'board': k, 'cells': v} for k, v in acc.items()]}, - indent=2) + '\n') + # by LINE, deduped: attempts repeat the same caveat far more often than they add a new + # one, and three copies of the D-state note reads as three incidents + seen, merged = set(), [] + for line in (prior_banner + banner).splitlines(): + if line.strip() and line not in seen: + seen.add(line) + merged.append(line) + banner = '\n'.join(merged) + '\n' if merged else '' + jpath.write_text(json.dumps({'rows': [{'board': k, 'cells': c, 'duration': d} + for k, (c, d) in acc.items()], + 'banner': banner}, indent=2) + '\n') - md = render_matrix(list(acc.items())) + md = render_matrix([(k, c, d) for k, (c, d) in acc.items()]) + if scope: + # a scoped run's small table is otherwise indistinguishable from a full one, and + # it replaces the previous full table in the sticky PR comment + md = f'_Scoped run: {scope}. Boards/tests not listed were not run._\n\n' + md + # LAST, so it is outermost: a rig-health caveat outranks the table AND the scope note, + # and the top of the report is where hil/SKILL.md tells the agent to look for it. + if banner: + md = banner + '\n' + md (report_dir / REPORT_MD).write_text(md + '\n', encoding='utf-8') return md +# containment paths print through hil_health._p: stdout may already be a dead pipe (a +# dropped ssh session), and a BrokenPipeError there would skip os._exit +_p = hil_health._p + + +def _abandon_exit(pool, mgr, abandoned: bool, err_count: int, + report: Path | None = None) -> None: + """Free the runner when the pool could not be shut down. Returns only if not abandoned. + + Must run even while an exception is propagating: multiprocessing's atexit handler + SIGTERMs its daemon workers (ignored in uninterruptible sleep) and then join()s them + with NO timeout, so an abandoned pool plus any raise between the pool's finally and + here hangs the interpreter until the job ceiling kills it. Reproduced: rc=124 at 25s + with SIGTERM-ignoring workers standing in for D state.""" + if not abandoned: + return + try: + if sys.exc_info()[0] is not None: + # os._exit below discards the traceback, and this is often the only place the + # real failure would ever be printed + traceback.print_exc() + except OSError: + pass + # Word this on evidence: shutdown_pool also returns False when terminate() RAISES, and + # a live worker after terminate() is what distinguishes a wedge from a harness bug. + # Count WORKERS only -- _pool_procs appends the Manager, our own healthy child, so + # including it made n >= 1 always and the harness-error branch unreachable. It is killed + # separately: os._exit skips its finalizer, and orphaned it holds the runner's stdout. + n = hil_health.kill_pool_children(pool) + hil_health.kill_pool_children(None, mgr) + if n: + _p(f'HIL worker pool would not terminate ({n} worker(s) still live, ' + f'uninterruptible); SIGKILLed them and abandoned the rest to free the ' + f'runner. Boards held by any leaked worker stay locked until the host is ' + f'power-cycled.', flush=True) + else: + _p('HIL worker pool shutdown failed but left no live worker behind, so this is ' + 'a harness error rather than a wedged rig -- see the Pool.terminate() ' + 'warning above. Exiting early anyway to free the runner; no board should ' + 'stay locked.', flush=True) + # A report already written by accumulate_report says nothing about the abandon, and a + # green table under a red job is how an agent ends up pasting it as this run's result. + # Prepend the caveat; best-effort, never at the cost of exiting. + if report is not None: + try: + if report.exists(): + # utf-8 explicitly (the cells are ✅/❌/⚪) and catch ValueError too: a torn + # report or a LANG=C locale raises UnicodeDecodeError -- NOT an OSError -- + # straight past os._exit, stranding the runner. + body = report.read_text(encoding='utf-8', errors='replace') + # Only when no banner is there yet, searched anywhere in the head rather + # than at char 0: write_timeout_report's banner must stay FIRST (its table + # is a PREVIOUS attempt's) and it puts the rig-health quote above itself. + if '**HIL run ab' not in body[:2000]: + report.write_text( + '**HIL run abandoned: the worker pool would not shut down.** The ' + 'table below was collected before the abandon; treat board ' + 'results as unverified.\n\n' + body, encoding='utf-8') + except (OSError, ValueError): + pass + try: + sys.stdout.flush() + except OSError: + pass + # Clamped: os._exit takes a status byte, so err_count == 256 would truncate to 0 and + # report a failing, abandoned run as green. + os._exit(min(err_count, 125) if err_count else 1) + + def main() -> None: """ Hardware test on specified boards @@ -1875,7 +2239,6 @@ def main() -> None: global verbose global test_only global board_test - global build_dir global max_retry global skip_flash @@ -1884,28 +2247,42 @@ def main() -> None: parser = argparse.ArgumentParser() parser.add_argument('config_file', help='Configuration JSON file') parser.add_argument('-b', '--board', action='append', default=[], help='Boards to test, all if not specified') - parser.add_argument('-s', '--skip-board', action='append', default=[], help='Skip boards from test') + parser.add_argument('--flasher', action='append', default=[], + help='Only boards using these flashers, e.g. esptool ' + '(for splitting one config across CI jobs)') + parser.add_argument('--exclude-flasher', action='append', default=[], + help='Exclude boards using these flashers') + parser.add_argument('-a', '--accumulate', action='store_true', + help='Merge results into the existing report instead of starting fresh ' + '(re-runs; the .failed file starts with this)') parser.add_argument('-sf', '--skip-flash', action='store_true', help='Run tests without flashing firmware (use whatever is already on the board)') parser.add_argument('-t', '--test-only', action='append', default=[], help='Tests to run, all if not specified') parser.add_argument('-bt', '--board-test', action='append', default=[], help='Per-board test list as BOARD:test1,test2 (overrides -t for that board); repeat for multiple boards') parser.add_argument('-B', '--build-dir', default='cmake-build', help='Build folder name (default: cmake-build)') parser.add_argument('--build', action='store_true', help='Build firmware for selected boards with cmake before running tests') - parser.add_argument('-r', '--retry', type=int, default=3, help='Retry count for failed tests (default: 3)') + # default 1, not 3: the pool guard is a FLAT 3600s that does not scale with max_retry, + # and one usbtest test at default 3 can burn 1530s of it (510s outer x3) for a single + # board. Every CI caller already pins --retry 1; the bare invocations in the hil skill + # and hil-validate.js run against the same one-slot rig and used to inherit 3. + parser.add_argument('-r', '--retry', type=int, default=1, help='Retry count for failed tests (default: 1)') parser.add_argument('-v', '--verbose', action='store_true', help='Verbose output') args = parser.parse_args() + if args.retry < 1: + # 0 would make every test loop body never run: all-red cells, exit 0 + parser.error('--retry must be >= 1') config_file = Path(args.config_file) boards = args.board - skip_boards = args.skip_board verbose = args.verbose + hil_util.verbose = args.verbose test_only = args.test_only for entry in args.board_test: bname, _, tnames = entry.partition(':') if not bname or not tnames: parser.error(f'invalid --board-test value: {entry!r} (expected BOARD:test1,test2)') board_test[bname] = [t for t in tnames.split(',') if t] - build_dir = args.build_dir + hil_flash.build_dir = args.build_dir max_retry = args.retry skip_flash = args.skip_flash @@ -1916,7 +2293,7 @@ def main() -> None: config = cast(HilConfig, json.load(f)) if len(boards) == 0: - config_boards = [e for e in config['boards'] if e['name'] not in skip_boards] + config_boards = list(config['boards']) else: unknown = [b for b in boards if b not in {e['name'] for e in config['boards']}] if unknown: @@ -1924,11 +2301,39 @@ def main() -> None: print(f'ERROR: board(s) not in {config_file.name}: {", ".join(unknown)}') sys.exit(1) config_boards = [e for e in config['boards'] if e['name'] in boards] + config_boards = [e for e in config_boards if e['flasher']['name'] not in args.exclude_flasher + and (not args.flasher or e['flasher']['name'] in args.flasher)] + if not config_boards: + # same reason the unknown -b board exits 1: 'No tests were run.' with rc 0 reads as + # a green HIL leg, so a roster edit emptying a leg's filter stops testing silently + msg = (f'No boards left after the flasher filter (--flasher ' + f'{args.flasher or "-"}, --exclude-flasher {args.exclude_flasher or "-"})') + print(msg, flush=True) + # loud AND leaving evidence: exiting with no report at all lets the PR comment + # keep the previous push's stale table under a red job + try: + rd = Path(os.environ.get('HIL_REPORT_DIR', '.')) + rd.mkdir(parents=True, exist_ok=True) + (rd / REPORT_MD).write_text(f'**HIL run selected no boards.** {msg}\n', + encoding='utf-8') + except OSError: + pass + sys.exit(1) + + + # Before the build: the probe needs nothing from it, and the annotation is more useful + # early than after a multi-board cmake build has been paid for. + # One line, not a probe: a D-state pid at start-up is a hint for whoever reads a red + # cell, never a reason to refuse the run. hil_pool_check does diagnosis. + note = hil_health.d_state_note() + if note: + log_line(f'rig note: {note}') + health_banner = f'> **Rig note.** {note}. Not a fault on its own -- a healthy testusb sits in D state for most of every case.\n' if note else '' build_err = 0 if args.build: - if build_dir != 'cmake-build': - print(f'warning: --build writes into cmake-build/, but -B is {build_dir!r}; ' + if hil_flash.build_dir != 'cmake-build': + print(f'warning: --build writes into cmake-build/, but -B is {hil_flash.build_dir!r}; ' f'tests will not find the freshly built firmware') print('-' * 30) print(f'Build phase: {len(config_boards)} board(s)') @@ -1940,51 +2345,264 @@ def main() -> None: print(f'Build phase done: {build_err} failed') print('-' * 30) - # HIL report sidecar (hil_report.json/.md). A full run starts fresh; a re-run - # (--skip-board / -bt, i.e. the .skip file) accumulates so already-passed - # boards/tests are preserved. Clear any prior report up front on a fresh run so - # a crash mid-run can't leave stale results to be merged by a retry or posted. + # The report sidecar and the .failed re-run spec live in report_dir (CI keys it by run + # id: persistent across attempts, private to one run). A full run starts fresh; a re-run + # (--accumulate, which .failed always starts with) merges so already-passed boards + # survive. -bt alone is not a re-run marker. report_dir = Path(os.environ.get('HIL_REPORT_DIR', '.')) - fresh = not (args.skip_board or args.board_test) + failed_fname = report_dir / (config_file.name + '.failed') + fresh = not args.accumulate + # The unlink is DEFERRED to inside the pool try/except below: wiping here leaves + # Manager() and Pool() running with the old report gone and no report-writing path + # armed, so an EAGAIN/ENOMEM on fork gives CI an EMPTY report dir with no reason. + + seed = os.getenv('HIL_SHUFFLE_SEED') or str(int(time.time())) + log_line(f'test-order shuffle seed: {seed} (HIL_SHUFFLE_SEED={seed} to replay); ' + f'flash/usbtest parallel per controller: {hil_lock.FLASH_PARALLEL}/{hil_lock.USBTEST_PARALLEL}; ' + f'enum timeout first/retry: {ENUM_TIMEOUT}/{ENUM_TIMEOUT_RETRY}s; ' + # all three are env-tunable, so a run that dies on the guard is otherwise + # unattributable from the log alone + f'pool guard: {POOL_TIMEOUT}s') + + hints = {} + try: + with CONTROLLER_CACHE.open() as f: + loaded = json.load(f) + # tolerate a hand-edited/torn cache: keep only the expected uid -> dict shape + if isinstance(loaded, dict): + hints = {k: v for k, v in loaded.items() if isinstance(v, dict)} + except (OSError, ValueError): + pass + hints_by_uid = {uid: h['pci'] for uid, h in hints.items() if h.get('pci')} + config_boards = schedule_boards(config_boards, hints_by_uid) + log_line('dispatch order: ' + ', '.join(b['name'] for b in config_boards)) + + # Bound BEFORE the try so the finally can name them whatever failed: Pool() forks, and + # the EAGAIN/ENOMEM the wipe comment below worries about is most likely to come from + # that fork -- after a convoy, where every stranded read holds a thread and an fd. Left + # outside, an OSError there escaped with mgr LIVE and `pool` unbound, so no report was + # written and the interpreter unwound into multiprocessing's unbounded atexit join. + pool = mgr = cmap = None + # Defined before the pool so _abandon_exit always has a value: a raise before + # `err_count = build_err + ...` would turn the containment path into a NameError. + err_count = build_err + # Fail CLOSED: only a shutdown_pool() that actually returned True clears this, and the + # assignment sits at the END of the inner finally, so anything raising before it + # (kill_worker_children, a BrokenPipeError from its print) leaves _abandon_exit armed. + pool_abandoned = True + # BEFORE Manager()/Pool(), not inside the try: hil_ci.sh reuses a persistent REMOTE_DIR + # and scp's the report back unconditionally, so if a fork failure (OSError/EAGAIN right + # after a convoy -- the case this whole block guards) skipped the wipe, the finally's + # _abandon_exit would prepend "HIL run abandoned" to the PREVIOUS run's table and + # publish last night's board results as this run's. Nothing is live yet here, so an + # OSError from the wipe itself just exits with its traceback -- it cannot strand the + # interpreter in multiprocessing's unbounded atexit join, which is what deferring it + # was protecting against. if fresh: report_dir.mkdir(parents=True, exist_ok=True) for f in (REPORT_JSON, REPORT_MD): (report_dir / f).unlink(missing_ok=True) + failed_fname.unlink(missing_ok=True) + try: + mgr = Manager() + cmap = mgr.dict() + initargs = (Lock(), seed, + hil_lock.make_permit_sems(Semaphore, hil_lock.USBTEST_PARALLEL), + hil_lock.make_permit_sems(Semaphore, hil_lock.FLASH_PARALLEL), + cmap, Lock(), hints_by_uid) + # maxtasksperchild=1: the sysfs blindness latch is process-global and permanent + # (no decrement anywhere -- see hil_util.SYSFS_STUCK_MAX), so a worker that goes + # blind on ONE wedged board would report 0/30 and "probe missing" for the 2-3 + # healthy boards it picked up afterwards. A fresh worker per board confines the + # damage to the board that caused it; the extra fork is noise against a + # flash+test cycle. + pool = Pool(processes=os.cpu_count() or 1, initializer=init_worker, + initargs=initargs, maxtasksperchild=1) + # OUTER: encloses the pool block too, not just the reporting below. An exception + # escaping async_ret.get() (a worker exception, a Ctrl-C) runs the pool finally and + # then propagates straight out of main(); with _abandon_exit in a sibling try it + # was never reached. + try: + # imap_unordered, NOT map_async: map_async is all-or-nothing, so a guard expiry + # threw away every board that had already finished -- up to a worker-width of + # completed rig time -- and left the re-run spec unwritten, so CI re-tested all + # ~26 boards to find the one that wedged. Draining as results arrive keeps what + # finished and names only what was still in flight. + it = pool.imap_unordered(test_board, config_boards) + mret = [] + deadline = time.monotonic() + POOL_TIMEOUT + try: + mret = drain_pool(it, config_boards, deadline, out=mret) + except MpTimeoutError as te: + mret = te.finished + stuck = [b['name'] for b in config_boards + if b['name'] not in {r[0] for r in mret}] + # The re-run spec FIRST and before the raise: a fresh run already unlinked + # it, so leaving it unwritten is what made the GitHub re-run repeat the + # whole fleet. Only the boards that never reported go in it. + _write_failed_spec(failed_fname, report_dir, + [(n, 1, [], None, 0) for n in stuck] + + [r for r in mret if r[1] > 0]) + # Then the report, with the rows that DID finish, before anything that can + # block. Then RAISE into the ONE containment path: the inner finally runs + # the ordered sweep (kill_worker_children BEFORE terminate, or a reaped + # worker's flasher reparents out of reach), the outer one os._exit's. + banner = (f'**HIL run abandoned: worker pool timed out after ' + f'{POOL_TIMEOUT}s.** {len(mret)} board(s) below finished and ' + f'are this run\'s; {len(stuck)} never reported and are NOT in ' + f'the table: {", ".join(stuck)}. Re-run covers those.\n') + try: + accumulate_report(mret, report_dir, fresh, '', + health_banner + _blind_note(mret) + + _stray_note(mret) + banner) + except Exception as rerr: # noqa: BLE001 - the raise below must still happen + # FALL BACK, do not just warn: accumulate_report can raise on an + # unwritable/root-owned report dir or a torn JSON, and _abandon_exit + # only PREPENDS to a report that exists. Without this the artifact + # upload finds nothing (if-no-files-found: ignore) and the sticky PR + # comment keeps the previous push's green table under a red job. + print(f'warning: partial report failed: {type(rerr).__name__}: {rerr}; ' + f'falling back to the board list', flush=True) + try: + hil_health.write_timeout_report( + report_dir, [b for b in config_boards + if b['name'] in stuck], POOL_TIMEOUT, REPORT_MD, + prefix=health_banner) + except Exception as re2: # noqa: BLE001 + print(f'warning: fallback report failed too: ' + f'{type(re2).__name__}: {re2}', flush=True) + _p(f'HIL worker pool timed out after {POOL_TIMEOUT}s; sweeping and ' + f'shutting it down (abandoning it if a worker is unkillable)', + flush=True) + raise RuntimeError(f'HIL worker pool timed out after {POOL_TIMEOUT}s') + except Exception as e: + # A worker RAISED -- e.g. a flasher adapter dropping off the bus makes + # get_serial_dev raise in the worker's flash section, which no per-test + # handler guards. Same treatment as the timeout path: the drain means + # `mret` already holds every board that finished, so keep those rows and + # name only the ones still in flight. (Under map_async they were all lost, + # which is what the old banner here claimed.) + done = {r[0] for r in mret} + stuck = [b['name'] for b in config_boards if b['name'] not in done] + _write_failed_spec(failed_fname, report_dir, + [(n, 1, [], None, 0) for n in stuck] + + [r for r in mret if r[1] > 0]) + banner = (f'**HIL run aborted: a worker raised {type(e).__name__}: {e}.** ' + f'{len(mret)} board(s) below finished and are this run\'s; ' + f'{len(stuck)} did not report: {", ".join(stuck)}.\n') + try: + accumulate_report(mret, report_dir, fresh, '', + health_banner + _blind_note(mret) + + _stray_note(mret) + banner) + except Exception as re2: # noqa: BLE001 - the raise below must still happen + print(f'warning: partial report failed: {type(re2).__name__}: {re2}', + flush=True) + raise - with Pool(processes=os.cpu_count() or 1, initializer=init_worker, initargs=(Lock(),)) as pool: - async_ret = pool.map_async(test_board, config_boards) + err_count = build_err + sum(e[1] for e in mret) + _write_failed_spec(failed_fname, report_dir, mret) + finally: + # Not `with Pool(...)`: its __exit__ joins the workers unbounded, hanging on + # any worker in uninterruptible sleep. shutdown_pool bounds the same terminate() + # by a grace period, so the pool is NOT cleanly closed/joined when it returns + # False. Record the outcome but never exit here: the report below is the only + # record of a run that otherwise passed. + # + # Same ordering as the timeout path: what the workers spawned must be + # snapshotted and killed while its parent is alive, or terminate() reparents it + # out of reach. + # + # Both calls must stay guarded: a raise here skips accumulate_report(), so a run + # whose boards ALL passed publishes an empty report dir -- and both can raise + # for reasons unrelated to the results. pool_abandoned stays fail-CLOSED, so + # _abandon_exit still arms. + try: + # Still worth running for the TIMEOUT path, where the workers are + # genuinely stuck mid-task and their children are still reachable through + # the pool's ppid tree. On the normal path every worker has already swept + # its own (kill_own_children) and retired, so this finds nothing. + # + # No banner from here: this finally runs AFTER accumulate_report on both + # abort paths, so anything appended to health_banner now is written to a + # variable nobody reads again. The report gets its count from the result + # tuples instead, via _stray_note. + hil_health.kill_worker_children(pool, mgr) + except Exception as e: + print(f'warning: worker-child sweep failed: {type(e).__name__}: {e}', + flush=True) + try: + pool_abandoned = not hil_health.shutdown_pool(pool) + except Exception as e: + print(f'warning: pool shutdown failed: {type(e).__name__}: {e}', flush=True) + + # refresh controller hints: pci resolved this run, plus durations from full runs + # only (a filtered run would understate the board's real cost) try: - mret = async_ret.get(timeout=POOL_TIMEOUT) - except MpTimeoutError: - pool.terminate() - pool.join() - raise RuntimeError(f'HIL worker pool timed out after {POOL_TIMEOUT}s') + if PROFILE: + # debug snapshot of the run's live uid->PCI / PCI->slot resolutions + report_dir.mkdir(parents=True, exist_ok=True) + with (report_dir / 'hil_profile_ctrl.json').open('w') as f: + json.dump(dict(cmap), f, indent=1, sort_keys=True) + uid_of = {b['name']: b['uid'] for b in config['boards']} + for name, _, _, _, dur, *_ in mret: + uid = uid_of.get(name) + if uid is None: + continue + h = dict(hints.get(uid) or {}) + h['name'] = name # informational: cache is keyed by uid + h['pci'] = cmap.get(f'uid:{uid}') or h.get('pci') + if dur > 0: # test_board reports 0.0 for filtered (partial) runs + h['duration'] = round(dur, 1) + hints[uid] = h + # merge-on-write: another HIL job (e.g. the esp split) may have finished since + # our startup read, so overlay only this run's boards and replace atomically + merged = {} + try: + with CONTROLLER_CACHE.open() as f: + cur = json.load(f) + if isinstance(cur, dict): + merged = {k: v for k, v in cur.items() if isinstance(v, dict)} + except (OSError, ValueError): + pass + merged.update({uid_of[n]: hints[uid_of[n]] for n, *_ in mret if n in uid_of}) + CONTROLLER_CACHE.parent.mkdir(parents=True, exist_ok=True) + tmp = CONTROLLER_CACHE.with_suffix('.json.tmp') + with tmp.open('w') as f: + json.dump(merged, f, indent=1, sort_keys=True) + tmp.replace(CONTROLLER_CACHE) + except Exception as e: + # Deliberately broad, and it must stay that way: this best-effort refresh makes + # Manager proxy RPCs that raise EOFError / BrokenPipeError / RemoteError when + # the Manager child has died, none of them OSErrors -- an OSError-only guard let + # those skip accumulate_report(). Nothing here is worth the report. + print(f'warning: cannot persist controller hints to {CONTROLLER_CACHE}: ' + f'{type(e).__name__}: {e}') - err_count = build_err + sum(e[1] for e in mret) - # generate skip list for next re-run if failed: skip boards that fully passed, - # and emit -bt BOARD:t1,t2 so each failed board only re-runs its own failed tests. - skip_fname = config_file.with_suffix(config_file.suffix + '.skip') - if err_count > 0: - skip_boards += [name for name, err, _, _ in mret if err == 0] - parts = [f'--skip-board {i}' for i in skip_boards] - parts += [f'-bt {name}:{",".join(fts)}' for name, err, fts, _ in mret if err > 0 and fts] - with skip_fname.open('w') as f: - f.write(' '.join(parts)) - elif skip_fname.exists(): - skip_fname.unlink() - # board x test result matrix -> hil_report.md (accumulates across re-runs) + stdout - report = accumulate_report(mret, report_dir, fresh) - print() - print(report) - print(f'\nReport written to {(report_dir / REPORT_MD).resolve()}') + # board x test result matrix -> hil_report.md (accumulates across re-runs) + stdout. + # -b/-bt means a filtered run (PR selection or a re-run spec): say so, or the report + # looks exactly like a full run that happened to be small + scoped = sorted(set(args.board) | set(board_test)) + scope = f'{len(scoped)} board(s) — {", ".join(scoped)}' if scoped else '' + report = accumulate_report(mret, report_dir, fresh, scope, + health_banner + _blind_note(mret) + + _stray_note(mret)) + print() + print(report) + print(f'\nReport written to {(report_dir / REPORT_MD).resolve()}') - duration = time.time() - duration - print() - print("-" * 30) - print(f'Total failed: {err_count} in {duration:.1f}s') - print("-" * 30) - sys.exit(err_count) + duration = time.time() - duration + print() + print("-" * 30) + print(f'Total failed: {err_count} in {duration:.1f}s') + print("-" * 30) + finally: + # In the finally, not after: any raise above (accumulate_report sits outside the + # OSError handler) would skip the abandon path and unwind into multiprocessing's + # unbounded atexit join, hanging the runner. + _abandon_exit(pool, mgr, pool_abandoned, err_count, report_dir / REPORT_MD) + # Same clamp: exit status is a byte either way, so 256 failures would report green. + sys.exit(min(err_count, 125)) if __name__ == '__main__': diff --git a/test/hil/mtp_test.py b/test/hil/mtp_test.py new file mode 100644 index 000000000..92d54bdbe --- /dev/null +++ b/test/hil/mtp_test.py @@ -0,0 +1,246 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +# One MTP test session for one board, in a disposable process. Every libmtp call is +# synchronous ctypes in our own address space and blocks in a usbfs ioctl in D state on +# a wedged device, where not even SIGKILL is delivered — so the session must be +# something the harness can abandon: hil_test.test_device_mtp runs it under +# hil_util.run_cmd (killpg + bounded reap, rc 124 on timeout). Imports stay stdlib + +# pymtp: nothing here may pull in the harness. +# +# Exit 0 on a fully passing session; 1 with the failure on stdout/stderr otherwise. +import argparse +import ctypes +import glob +import hashlib +import os +import signal +import subprocess +import sys +import threading +import time + +sys.path.append(os.path.dirname(os.path.abspath(__file__))) # PYTHONSAFEPATH drops it +# -- APPEND so PYTHONPATH still wins (the tests steer a fake pymtp that way) + +from pathlib import Path +from pymtp import LIBMTP_DeviceEntry, LIBMTP_RawDevice, MTP + +FILE1_EXPECT = b'TinyUSB MTP Filesystem example' +FILE2_MD5_EXPECT = '40ef23fc2891018d41a05d4a0d5f822f' # md5sum of logo.png + + +# Real paths by default; the offline tests point these at a fixture tree, the same way +# they steer the pymtp fake through FAKE_PYMTP_*. +# The one test seam: '' in production, a tmpdir in the offline tests, which mirror the +# real layout beneath it. This runs as a SUBPROCESS (a libmtp call blocked in a usbfs +# ioctl hangs its thread forever, so the session must be somewhere killable), and neither +# monkeypatching nor import shadowing crosses that boundary -- unlike the fake pymtp, +# which the tests inject through PYTHONPATH alone. +_ROOT = os.environ.get('HIL_MTP_FAKE_ROOT', '') +_MARKER_GLOB = f'{_ROOT}/dev/libmtp-*' +_SYS_USB = Path(f'{_ROOT}/sys/bus/usb/devices') +_USB_DEV = Path(f'{_ROOT}/dev/bus/usb') + + +def _bounded_read(path, grace: float = 2.0): + """Read a sysfs attribute with a wall-clock bound, or return None. + + `serial` is served under the device lock a wedged usbfs ioctl holds, and EVERY MTP DUT + is cafe:4017 -- so the vid/pid filter below cannot rule out a wedged NEIGHBOUR, and an + unbounded read of its serial would burn this session's whole budget and report a + healthy board as wedged. Stdlib only by design (this file never imports the harness), + so this is a small local twin of hil_util.read_sysfs. + """ + out = {} + + def _read(): + try: + out['v'] = path.read_text().strip() + except OSError: + pass + + t = threading.Thread(target=_read, daemon=True) + t.start() + t.join(grace) + return out.get('v') + + +def _ready_marker(uid: str): + """(busnum, devnum) of the udev-ready MTP device with this serial, or None. + + /dev/libmtp-<sysname> is published by libmtp-runtime AFTER its synchronous mtp-probe + accepts the device, so this set is both small and ready -- unlike a sysfs-wide scan, + which races re-enumerations from other boards' jobs. Requires the libmtp-runtime + package. + """ + for marker_name in glob.glob(_MARKER_GLOB): + marker = Path(marker_name) + try: + dev = _SYS_USB / marker.name[len('libmtp-'):] + # vid/pid first: lock-free descriptor fields, so they rule out every other + # device before the `serial` read, which the kernel serves under the device + # lock a wedged usbfs ioctl would hold + if ((dev / 'idVendor').read_text().strip() != 'cafe' + or (dev / 'idProduct').read_text().strip() != '4017'): + continue + # bounded: this one CAN block, and a wedged neighbour shares the vid/pid above + serial = _bounded_read(dev / 'serial') + if serial is None or serial.lower() != uid.lower(): + continue + busnum = int((dev / 'busnum').read_text()) + devnum = int((dev / 'devnum').read_text()) + node = _USB_DEV / f'{busnum:03d}' / f'{devnum:03d}' + if marker.resolve(strict=True) != node or not os.access(node, os.R_OK | os.W_OK): + continue + return busnum, devnum + except (OSError, ValueError): + # a marker can vanish while another board flashes: not our device's problem + continue + return None + + +def _gvfs_unmount(uid: str, deadline: float) -> None: + """Drop any gvfs claim on this device, immediately before opening it. + + Called only once the udev marker exists. gvfs claims an MTP device AFTER udev + probing, so before the marker there is nothing to unmount: an earlier call is a + guaranteed no-op that still forks a process, and it leaves the gap between the + unmount and the open unprotected -- the hang this exists to prevent. Per-iteration + calls also forked one gio per second of the enumeration budget. + """ + # Popen, not run(timeout=): run's post-timeout reap is an unbounded wait(), and a gio + # blocked in D state on a wedged usbfs node does not die on SIGKILL, so run(timeout=2) + # can hang for good. Bounded by at most HALF of what is LEFT of our own budget, never + # a fixed sub-bound: the parent gives us --timeout 8 (4 on a retry), so anything larger + # collapsed the poll loop to one attempt and made a slow gio look like a wedged session. + gio_bound = max(0.5, min(3.0, (deadline - time.monotonic()) / 2)) + try: + # argv, not shell=True: uid comes from a hand-edited roster and is board firmware + # output, so a space or $(...) would unmount the wrong URI (leaving the gvfs mount + # held) or run as us. + gio = subprocess.Popen(['gio', 'mount', '-u', + f'mtp://TinyUsb_TinyUsb_Device_{uid}/'], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, start_new_session=True) + except OSError: + # glib2.0-bin absent (ci.lan has no gio at all): nothing holds a gvfs mount + # either, so go straight on to the open. + return + try: + gio.wait(timeout=gio_bound) + except subprocess.TimeoutExpired: + try: + os.killpg(gio.pid, signal.SIGKILL) + except OSError: + gio.kill() + try: + gio.wait(timeout=2) # reap it: an abandoned gio leaves a zombie + except subprocess.TimeoutExpired: + pass + print('gio unmount timed out; continuing', file=sys.stderr) + + +def open_mtp_dev(uid: str, timeout: float): + mtp = MTP() + deadline = time.monotonic() + timeout + while True: + try: + # pymtp raises USB_LAYER/PTP_LAYER/GENERAL/AlreadyConnected on a board still + # settling right after a flash; an unguarded raise would skip the rest of the + # enumeration budget (and the disconnect) instead of retrying. + # + # Never detect_devices(): that PROBES every MTP device on the rig, so a board + # still initialising in a parallel job answers our scan (the race #3790 fixed). + # libmtp-runtime publishes /dev/libmtp-<sysname> only after its own mtp-probe + # has accepted a device, so start from that small, ready-only set and open OUR + # device directly by bus/dev address. + target = _ready_marker(uid) + if target: + # ready first, THEN unmount, then open -- see _gvfs_unmount + _gvfs_unmount(uid, deadline) + busnum, devnum = target + # TinyUSB needs no libmtp quirks, so the raw entry can be built here + entry = LIBMTP_DeviceEntry(None, 0xcafe, None, 0x4017, 0) + raw = LIBMTP_RawDevice(entry, busnum, devnum) + mtp.device = mtp.mtp.LIBMTP_Open_Raw_Device(ctypes.byref(raw)) + if mtp.device: + serial = mtp.get_serialnumber() + if (serial.decode('utf-8') if serial else '').lower() == uid.lower(): + return mtp + mtp.disconnect() + except Exception as e: + print(f'mtp poll: {type(e).__name__}: {e}', file=sys.stderr) + # only when a device was actually opened: pymtp's `self.device == None` + # guard does NOT catch a ctypes NULL pointer (falsy, but != None), so + # disconnecting blindly calls LIBMTP_Release_Device(NULL) + if getattr(mtp, 'device', None): + try: + mtp.disconnect() + except Exception: + pass + mtp.device = None + if time.monotonic() >= deadline: + return None + time.sleep(1) + + +def run_session(uid: str, timeout: float) -> int: + mtp = open_mtp_dev(uid, timeout) + if mtp is None or mtp.device is None: + print('MTP device not found') + return 1 + + try: + assert b"TinyUSB" == mtp.get_manufacturer(), 'MTP wrong manufacturer' + assert b"MTP Example" == mtp.get_modelname(), 'MTP wrong model' + assert b'1.0' == mtp.get_deviceversion(), 'MTP wrong version' + assert b'TinyUSB MTP' == mtp.get_devicename(), 'MTP wrong device name' + + f1 = uid.encode("utf-8") + b'_file1' + f2 = uid.encode("utf-8") + b'_file2' + f3 = uid.encode("utf-8") + b'_file3' + mtp.get_file_to_file(1, f1) + with open(f1, 'rb') as file: + f1_data = file.read() + os.remove(f1) + assert f1_data == FILE1_EXPECT, 'MTP file1 wrong data' + mtp.get_file_to_file(2, f2) + with open(f2, 'rb') as file: + f2_data = file.read() + os.remove(f2) + assert FILE2_MD5_EXPECT == hashlib.md5(f2_data).hexdigest(), 'MTP file2 wrong data' + with open(f3, "wb") as file: + # 1524-byte payload + 12-byte MTP header = 3 full 512-byte buffers, so this + # exercises delivery of the final OUT payload before its ZLP. Deliberate and + # FIXED: a random size hits that boundary in ~0.2% of runs, which is not a test + # of it. Deterministic content so a mismatch is reproducible. + f3_data = bytes((i % 251) + 1 for i in range(1524)) + file.write(f3_data) + file.close() + fid = mtp.send_file_from_file(f3, b'file3') + f3_readback = f3 + b'_readback' + mtp.get_file_to_file(fid, f3_readback) + with open(f3_readback, 'rb') as f: + f3_rb_data = f.read() + os.remove(f3_readback) + assert f3_rb_data == f3_data, 'MTP file3 wrong data' + os.remove(f3) + mtp.delete_object(fid) + except AssertionError as e: + print(e) + return 1 + finally: + mtp.disconnect() + return 0 + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument('--uid', required=True, help='board_get_unique_id serial to match') + parser.add_argument('--timeout', type=float, default=30, help='enumeration wait budget (s)') + args = parser.parse_args() + return run_session(args.uid, args.timeout) + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/test/hil/pymtp.py b/test/hil/pymtp.py index 8b694df94..fc0c66104 100644 --- a/test/hil/pymtp.py +++ b/test/hil/pymtp.py @@ -420,6 +420,8 @@ _libmtp.LIBMTP_Get_Playlist.restype = ctypes.POINTER(LIBMTP_Playlist) _libmtp.LIBMTP_Get_Folder_List.restype = ctypes.POINTER(LIBMTP_Folder) _libmtp.LIBMTP_Find_Folder.restype = ctypes.POINTER(LIBMTP_Folder) _libmtp.LIBMTP_Get_Errorstack.restype = ctypes.POINTER(LIBMTP_Error) +_libmtp.LIBMTP_Dump_Errorstack.argtypes = [ctypes.POINTER(LIBMTP_MTPDevice)] +_libmtp.LIBMTP_Dump_Errorstack.restype = None _libmtp.LIBMTP_Open_Raw_Device.restype = ctypes.POINTER(LIBMTP_MTPDevice) _libmtp.LIBMTP_Open_Raw_Device.argtypes = [ctypes.POINTER(LIBMTP_RawDevice)] @@ -451,16 +453,14 @@ class MTP: def debug_stack(self): """ - Checks if __DEBUG__ is set, if so, prints and clears the - errorstack. + Checks if __DEBUG__ is set, and if so prints the error stack. @rtype: None @return: None """ - if __DEBUG__: - self.mtp.LIBMTP_Dump_Errorstack() - #self.mtp.LIBMTP_Clear_Errorstack() + if __DEBUG__ and self.device: + self.mtp.LIBMTP_Dump_Errorstack(self.device) def detect_devices(self): """ diff --git a/test/hil/requirements.txt b/test/hil/requirements.txt index ef1cf575b..abfb93783 100644 --- a/test/hil/requirements.txt +++ b/test/hil/requirements.txt @@ -1,7 +1,8 @@ # System packages (install separately): -# sudo apt install mtools libmtp9 alsa-utils iperf +# sudo apt install mtools libmtp9 libmtp-runtime alsa-utils iperf # mtools - read_disk_file (device/cdc_msc, device/msc_dual_lun) # libmtp9 - pymtp ctypes load (device/mtp); Debian 13 uses libmtp9t64 +# libmtp-runtime - mtp-probe and the completed-device /dev/libmtp-* marker # alsa-utils - arecord (device/audio_test_freertos) # iperf - throughput tests (device/net_lwip_*) hidapi diff --git a/test/hil/test/stubs/pymtp.py b/test/hil/test/stubs/pymtp.py new file mode 100644 index 000000000..2720321f6 --- /dev/null +++ b/test/hil/test/stubs/pymtp.py @@ -0,0 +1,125 @@ +# SPDX-License-Identifier: MIT +# Fake pymtp for the hil unit tests — stands in both for the import (GitHub's bare +# pre-commit runner has no libmtp/pymtp) and for a scripted MTP device. Behavior is +# driven by env vars so subprocesses (mtp_test.py under run_cmd) can be steered: +# FAKE_PYMTP_MODE absent (default) | ok | hang +# FAKE_PYMTP_UID serial number the fake device reports +# FAKE_PYMTP_FILE1 text served as file id 1 (README.TXT) +# FAKE_PYMTP_LOGO path to the logo bytes served as file id 2 +# File contents come from env, not constants: the test extracts them from the example's +# own sources, so this stub cannot drift out of sync with the firmware. +# 'hang' blocks forever inside detect_devices — the in-process libmtp equivalent of a +# D-state usbfs ioctl on a wedged device. +import ctypes +import os +import time + + +class NotConnected(Exception): + pass + + +class LIBMTP_DeviceEntry(ctypes.Structure): + """Real pymtp exposes this; mtp_test builds one to open a KNOWN device instead of + probing every MTP device on the bus.""" + _fields_ = [('vendor', ctypes.c_char_p), ('vendor_id', ctypes.c_uint16), + ('product', ctypes.c_char_p), ('product_id', ctypes.c_uint16), + ('device_flags', ctypes.c_uint32)] + + +class LIBMTP_RawDevice(ctypes.Structure): + _fields_ = [('device_entry', LIBMTP_DeviceEntry), ('bus_location', ctypes.c_uint32), + ('devnum', ctypes.c_uint8)] + + +class _LibShim: + @staticmethod + def LIBMTP_Open_Raw_Device(_ref): + # mtp_test no longer calls detect_devices() (it probed every MTP device on the + # rig), so the scripted modes have to act here -- this is the only libmtp entry + # point the marker-based open goes through. + mode = os.environ.get('FAKE_PYMTP_MODE', 'absent') + if mode == 'hang': + time.sleep(10000) + if mode == 'error': + raise RuntimeError('CommandFailed: LIBMTP_ERROR_USB_LAYER') + if mode == 'error_then_ok': + flag = os.environ.get('FAKE_PYMTP_ERRED_MARKER', '/tmp/.fake_pymtp_erred') + if not os.path.exists(flag): + open(flag, 'w').close() + raise RuntimeError('CommandFailed: LIBMTP_ERROR_PTP_LAYER') + if mode == 'absent': + return ctypes.POINTER(ctypes.c_int)() # NULL: nothing to open + # the real one has restype POINTER(LIBMTP_MTPDevice): a failed open returns a + # NULL pointer, which is FALSY but compares unequal to None -- the distinction + # mtp_test's `if mtp.device:` guards depend on + if os.environ.get('FAKE_PYMTP_OPEN') == 'null': + return ctypes.POINTER(ctypes.c_int)() + return 1 + + +class MTP: + def __init__(self): + self.mtp = _LibShim() + self.device = None + self._sent = {} + self._next_id = 3 + + def detect_devices(self): + mode = os.environ.get('FAKE_PYMTP_MODE', 'absent') + if mode == 'hang': + time.sleep(10000) + if mode == 'error': + # real pymtp raises for USB_LAYER/PTP_LAYER/GENERAL/AlreadyConnected; + # the first poll after a flash routinely hits one + raise RuntimeError('CommandFailed: LIBMTP_ERROR_USB_LAYER') + if mode == 'error_then_ok': + if not getattr(self, '_erred', False): + self._erred = True + raise RuntimeError('CommandFailed: LIBMTP_ERROR_USB_LAYER') + return [ctypes.c_int(1)] + if mode != 'ok': + return [] + return [ctypes.c_int(1)] + + def get_serialnumber(self): + return os.environ.get('FAKE_PYMTP_UID', '').encode() + + def get_manufacturer(self): + return b'TinyUSB' + + def get_modelname(self): + return b'MTP Example' + + def get_deviceversion(self): + return b'1.0' + + def get_devicename(self): + return b'TinyUSB MTP' + + def get_file_to_file(self, fid, path): + if fid == 1: + data = os.environ['FAKE_PYMTP_FILE1'].encode() + elif fid == 2: + with open(os.environ['FAKE_PYMTP_LOGO'], 'rb') as f: + data = f.read() + else: + data = self._sent[fid] + with open(path, 'wb') as f: + f.write(data) + + def send_file_from_file(self, path, _name): + with open(path, 'rb') as f: + self._sent[self._next_id] = f.read() + self._next_id += 1 + return self._next_id - 1 + + def delete_object(self, fid): + del self._sent[fid] + + def disconnect(self): + # vendored pymtp raises when nothing is connected; a stub that silently accepts + # it hides a LIBMTP_Release_Device(NULL) call on real hardware + if self.device is None: + raise NotConnected('no device connected') + self.device = None diff --git a/test/hil/test/test_hil_bounded.py b/test/hil/test/test_hil_bounded.py new file mode 100644 index 000000000..908a142d5 --- /dev/null +++ b/test/hil/test/test_hil_bounded.py @@ -0,0 +1,1701 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +# Unit tests proving hil_test's storage and MTP helpers cannot hang the worker: a +# wedged device blocks the call in D state forever (child process or in-process ioctl), +# so these paths go through a bounded runner. Fakes stand in for the wedge (a real one +# cannot be manufactured on demand): a PATH-injected `mtype` script and a +# PYTHONPATH-injected `pymtp` module, each with a mode that blocks forever. +# Scope: mtype, the gio unmount, the libmtp session, the arecord/iperf reaps, and the +# printer read (a process now, via run_alongside, so a killed reader takes its fd with +# it -- usblp allows ONE opener, and a blocked thread kept the node for the worker's life). +# Known residue (unbounded, backstopped only by the pool guard): hid open/write and +# midi's read(64). +# +# hil_test imports pyserial, which GitHub's bare pre-commit runner does not have — so +# an inert serial module is stubbed into sys.modules BEFORE the import (nothing here +# exercises serial paths). MTP traffic never touches hil_test: it all goes through the +# mtp_test.py subprocess, which gets the fake pymtp via PYTHONPATH. +# Run directly: +# python3 test/hil/test/test_hil_bounded.py +import os +import stat +import sys +import threading +from multiprocessing import TimeoutError as MpTimeoutError +import time +import types +import unittest +from pathlib import Path +from tempfile import TemporaryDirectory + +TEST_DIR = os.path.dirname(os.path.abspath(__file__)) +# the modules under test live in the parent dir (test/hil), not here +sys.path.insert(0, os.path.dirname(TEST_DIR)) + +serial_stub = types.ModuleType('serial') +serial_stub.Serial = type('Serial', (), {}) +serial_stub.SerialException = type('SerialException', (Exception,), {}) +serial_stub.SerialTimeoutException = type('SerialTimeoutException', (Exception,), {}) +sys.modules.setdefault('serial', serial_stub) +import hil_flash +import hil_test + + +def write_script(path: Path, body: str) -> None: + path.write_text('#!/bin/sh\n' + body + '\n') + path.chmod(path.stat().st_mode | stat.S_IEXEC) + + +def run_bounded(fn, timeout: float): + """Run fn in a daemon thread; return (finished, exception). A still-running thread is + the hang under test — leave it to die with the interpreter.""" + exc = [] + + def wrapper(): + try: + fn() + except BaseException as e: # noqa: BLE001 - tests inspect the exception + exc.append(e) + + t = threading.Thread(target=wrapper, daemon=True) + t.start() + t.join(timeout) + return not t.is_alive(), exc[0] if exc else None + + [email protected](os.name == 'nt', 'POSIX shell fakes') +class ReadDiskFile(unittest.TestCase): + def setUp(self): + self.tmp = TemporaryDirectory() + tmp = Path(self.tmp.name) + self.addCleanup(self.tmp.cleanup) + # fake block device node: get_disk_dev is patched to this existing path + self.dev = tmp / 'fakedev' + self.dev.write_bytes(b'') + # addCleanup, not tearDown: tearDown does NOT run when setUp raises, and a leaked + # PATH entry points at a temp bin dir this class already deleted. + for name in ('get_disk_dev', '_enum_timeout', 'MTYPE_TIMEOUT'): + self.addCleanup(setattr, hil_test, name, getattr(hil_test, name)) + hil_test.get_disk_dev = lambda uid, vendor, lun: str(self.dev) + hil_test._enum_timeout = 2 + self.bin = tmp / 'bin' + self.bin.mkdir() + self.addCleanup(os.environ.__setitem__, 'PATH', os.environ['PATH']) + os.environ['PATH'] = f'{self.bin}:{os.environ["PATH"]}' + self.pidfile = tmp / 'mtype.pid' + self.addCleanup(self._reap_mtype) + + def _reap_mtype(self): + if self.pidfile.exists(): # reap a leaked hang-mode mtype + try: + os.kill(int(self.pidfile.read_text()), 9) + except (OSError, ValueError): + pass + + def test_returns_exact_bytes_despite_stderr_noise(self): + # \377 is invalid UTF-8 and stderr noise must not leak into the data + write_script(self.bin / 'mtype', r"printf 'R\377EADME-DATA'; printf 'vfat warning' >&2") + data = hil_test.read_disk_file('uid0', 0, 'README.TXT') + self.assertEqual(data, b'R\xffEADME-DATA') + + def test_failure_message_carries_mtype_stderr_and_fname(self): + write_script(self.bin / 'mtype', "printf 'mtype: cannot read' >&2; exit 1") + with self.assertRaises(AssertionError) as cm: + hil_test.read_disk_file('uid0', 0, 'README.TXT') + self.assertIn('cannot read', str(cm.exception)) + self.assertIn('README.TXT', str(cm.exception)) + + def test_empty_read_fails_immediately_with_fname(self): + # rc 0 with no data is a real answer (bad sectors, empty file), not "not ready": + # fail at once like the old assert did, naming the file — don't spin the budget + write_script(self.bin / 'mtype', 'exit 0') + t0 = time.monotonic() + with self.assertRaises(AssertionError) as cm: + hil_test.read_disk_file('uid0', 0, 'README.TXT') + self.assertLess(time.monotonic() - t0, 1.5) + self.assertIn('README.TXT', str(cm.exception)) + + def test_hung_mtype_cannot_hang_the_worker(self): + # a D-state child never exits; the bounded runner must give up without it + write_script(self.bin / 'mtype', f'echo $$ > {self.pidfile}; exec sleep 1000') + hil_test.MTYPE_TIMEOUT = 2 + finished, exc = run_bounded(lambda: hil_test.read_disk_file('uid0', 0, 'README.TXT'), 20) + self.assertTrue(finished, 'read_disk_file hung on a stuck mtype') + self.assertIsInstance(exc, AssertionError) + + +class CompactOutput(unittest.TestCase): + def test_strips_workflow_command_markers(self): + """Defense-in-depth: the historical marker source was worker-side run_cmd + (now suppressed at the emitter); anything future that pipes markers into a + captured stdout would land them mid-row where GitHub renders them literally.""" + raw = '::group::COMMAND TIMEOUT (1s): x\nboom\n::endgroup::\ntail' + self.assertEqual(hil_test.compact_output(raw), 'COMMAND TIMEOUT (1s): x | boom | tail') + + +class UsbtestRecovery(unittest.TestCase): + def test_recovery_flags_and_flash_bound_fit_the_reserve(self): + """The post-hang reflash plumbing: the CLI flags exist, and the bounded reflash + plus the fixed recovery costs (60s case timeout + 5s kill wait + 5s settle) + fits inside USBTEST_RECOVERY_BUDGET -- otherwise the outer run_cmd kill lands + mid-flash and orphans the flasher (own session) on the probe.""" + import subprocess + hil_dir = Path(TEST_DIR).parents[0] + r = subprocess.run([sys.executable, str(hil_dir / 'usbtest.py'), '--help'], + capture_output=True, text=True, timeout=30) + self.assertEqual(r.returncode, 0, r.stderr) + for flag in ('--recover-board', '--recover-fw', '--outer-timeout'): + self.assertIn(flag, r.stdout) + + def test_the_bounded_reflash_actually_fits_the_reserve(self): + """The arithmetic the docstring above claims but never checked -- the two + constants never met in any test, so bumping either silently broke the promise. + Overrun means run_cmd's outer kill lands MID-FLASH and orphans the flasher + (start_new_session, so killpg misses it) holding the probe.""" + import re + import usbtest + hil_dir = Path(TEST_DIR).parents[0] + # read the case timeout hil_test actually passes, so this cannot drift silently + src = (hil_dir / 'hil_test.py').read_text() + m = re.search(r'--timeout (\d+) --budget', src) + self.assertIsNotNone(m, 'usbtest invocation changed shape; re-derive this bound') + case_timeout = int(m.group(1)) + kill_wait, settle, time_left_reserve = 5, 5, 35 # usbtest.py's fixed costs + worst = (case_timeout + kill_wait + usbtest.RECOVER_FLASH_TIMEOUT + + settle + time_left_reserve) + self.assertLessEqual( + worst, hil_test.USBTEST_RECOVERY_BUDGET, + f'a HUNG case needs {worst}s to recover but only ' + f'{hil_test.USBTEST_RECOVERY_BUDGET}s is reserved') + + +class UsbtestRunHelper(unittest.TestCase): + """usbtest.run() is the bounded replacement for subprocess.run: sysfs_write feeds it + input=, and every battery calls that before case 1.""" + + def setUp(self): + import usbtest + self.usbtest = usbtest + + def test_input_kwarg_is_honoured(self): + r = self.usbtest.run(['cat'], input='payload', timeout=10) + self.assertEqual(r.returncode, 0) + self.assertEqual(r.stdout, 'payload') + + def test_capture_output_kwarg_is_accepted(self): + r = self.usbtest.run(['printf', 'x'], capture_output=True, timeout=10) + self.assertEqual(r.stdout, 'x') + + def test_timeout_is_bounded_and_raises(self): + import subprocess + t0 = time.monotonic() + with self.assertRaises(subprocess.TimeoutExpired): + self.usbtest.run(['sleep', '30'], timeout=1) + self.assertLess(time.monotonic() - t0, 15) + + +class BuildBoardContract(unittest.TestCase): + def test_every_return_path_is_a_pair(self): + """main() unpacks `_, nfail = build_board(board)`; a bare int on any path + (the timeout path did) raises TypeError before the pool exists.""" + import ast + src = (Path(TEST_DIR).parents[0] / 'hil_test.py').read_text() + fn = next(n for n in ast.walk(ast.parse(src)) + if isinstance(n, ast.FunctionDef) and n.name == 'build_board') + for node in ast.walk(fn): + if isinstance(node, ast.Return) and node.value is not None: + self.assertIsInstance(node.value, ast.Tuple, + f'build_board returns a non-tuple at line {node.lineno}') + + +class RemoteStaging(unittest.TestCase): + def test_import_closure_is_staged_to_the_rig(self): + # hil_ci.sh stages an explicit scp whitelist; a module that is not on it exists + # locally and in CI checkouts but silently never reaches the remote rig (how + # mtp_test.py was first missed). Walk the local-import closure of everything + # the rig executes and require each file's exact scp entry — a bare-substring + # match would be satisfied by a mention in a comment or the run line. + import ast + hil_dir = Path(TEST_DIR).parents[0] + staged = (hil_dir / 'hil_ci.sh').read_text() + + def imported_paths(pyfile): + # ast, not regex: an earlier regex walker went silently vacuous on a + # multi-line import. ast also sees function-local deferred imports + # (usbtest.py's `import hil_flash` inside the recovery branch). + for node in ast.walk(ast.parse(pyfile.read_text())): + if isinstance(node, ast.Import): + for a in node.names: + yield a.name.replace('.', '/') + '.py' + elif isinstance(node, ast.ImportFrom) and node.module: + if node.module == 'helper': + for a in node.names: + yield f'helper/{a.name}.py' + else: + yield node.module.replace('.', '/') + '.py' + + seeds = ['hil_test.py', 'usbtest.py', 'mtp_test.py'] # CLI + spawned helpers + for f in seeds: # a renamed seed must fail loudly, not fall out of the walk + self.assertTrue((hil_dir / f).exists(), f'stale RemoteStaging seed: {f}') + todo, seen = list(seeds), set() + while todo: + f = todo.pop() + if f in seen or not (hil_dir / f).exists(): + continue # stdlib/site-packages imports have no test/hil file + seen.add(f) + todo += list(imported_paths(hil_dir / f)) + for f in sorted(seen): + self.assertIn(f'"$ROOT_DIR/test/hil/{f}"', staged, + f'{f} runs on the rig but hil_ci.sh does not scp it') + + +class _MtpFakeRig: + """The fake rig shared by the MTP cases: a udev-marker tree under one tmp root and + the scripted pymtp on PYTHONPATH. A plain mixin, NOT a TestCase -- subclassing a + TestCase to reuse a fixture re-runs every inherited test in each subclass.""" + + @classmethod + def setUpClass(cls): + # both file fixtures come from the example's sources, so drift there fails here: + # file id 1 is README.TXT (C define), file id 2 is logo.png (C byte array) + import hashlib + import re + src = Path(TEST_DIR).parents[2] / 'examples/device/mtp/src' + m = re.search(r'#define README_TXT_CONTENT "([^"]+)"', (src / 'mtp_fs_example.c').read_text()) + assert m, 'README_TXT_CONTENT define not found in mtp_fs_example.c' + cls.readme = m.group(1) + data = bytes(int(x, 16) for x in + re.findall(r'0x([0-9a-fA-F]{2})', (src / 'tinyusb_logo_png.h').read_text())) + assert hashlib.md5(data).hexdigest() == '40ef23fc2891018d41a05d4a0d5f822f' + cls.logo = data + + def setUp(self): + self.tmp = TemporaryDirectory() + tmp = Path(self.tmp.name) + self.addCleanup(self.tmp.cleanup) + logo = tmp / 'logo.bin' + logo.write_bytes(self.logo) + self.board = {'uid': 'CAFE01', 'name': 'fakeboard'} + # addCleanup, not tearDown: tearDown does NOT run when setUp raises, and a leaked + # chdir into a deleted temp dir breaks every test after it. + self.saved_env = {k: os.environ.get(k) for k in + ('FAKE_PYMTP_MODE', 'FAKE_PYMTP_UID', 'FAKE_PYMTP_LOGO', + 'FAKE_PYMTP_FILE1', 'PYTHONPATH', 'PYTHONSAFEPATH', + 'HIL_MTP_FAKE_ROOT', 'FAKE_PYMTP_ERRED_MARKER')} + self.addCleanup(self._restore_env) + # A udev-ready marker tree: libmtp-runtime publishes /dev/libmtp-<sysname> only + # after mtp-probe accepts a device, and mtp_test opens THAT device directly rather + # than probing every MTP device on the rig (the parallel-probe race #3790 fixed). + # mirrors the real layout under one root, so <tmp>/sys/bus/usb/devices/1-1 reads + # as the stand-in for /sys/bus/usb/devices/1-1 that it is + dev = tmp / 'sys/bus/usb/devices/1-1' + usbdev = tmp / 'dev/bus/usb/001' + markers = tmp / 'dev' # created by usbdev's parents=True + dev.mkdir(parents=True); usbdev.mkdir(parents=True) + (dev / 'idVendor').write_text('cafe\n') + (dev / 'idProduct').write_text('4017\n') + (dev / 'serial').write_text(self.board['uid'] + '\n') + (dev / 'busnum').write_text('1\n') + (dev / 'devnum').write_text('2\n') + node = usbdev / '002' + node.write_bytes(b'') + (markers / 'libmtp-1-1').symlink_to(node) + os.environ['HIL_MTP_FAKE_ROOT'] = str(tmp) + os.environ['FAKE_PYMTP_ERRED_MARKER'] = str(tmp / 'erred') + os.environ['FAKE_PYMTP_UID'] = self.board['uid'] + os.environ['FAKE_PYMTP_LOGO'] = str(logo) + os.environ['FAKE_PYMTP_FILE1'] = self.readme + stubs = os.path.join(TEST_DIR, 'stubs') + pp = self.saved_env['PYTHONPATH'] + os.environ['PYTHONPATH'] = stubs if not pp else f'{stubs}:{pp}' + # pymtp is vendored next to mtp_test.py, and a script's own dir (sys.path[0]) + # outranks PYTHONPATH — safe-path mode (3.11+) drops it so the fake wins there + os.environ['PYTHONSAFEPATH'] = '1' + for name in ('_enum_timeout', 'MTP_SESSION_MARGIN'): + self.addCleanup(setattr, hil_test, name, getattr(hil_test, name)) + hil_test._enum_timeout = 2 + # the session scratch files land in cwd + self.addCleanup(os.chdir, os.getcwd()) + os.chdir(tmp) + + def _restore_env(self): + for k, v in self.saved_env.items(): + if v is None: + os.environ.pop(k, None) + else: + os.environ[k] = v + + [email protected](os.name == 'nt', 'POSIX shell fakes') [email protected](sys.version_info < (3, 11), 'fake-pymtp steering needs PYTHONSAFEPATH') +class DeviceMtp(_MtpFakeRig, unittest.TestCase): + """test_device_mtp end to end: the real mtp_test.py subprocess under run_cmd, + with the scripted pymtp fake steered in via PYTHONPATH.""" + + def test_mtp_session_passes_against_scripted_device(self): + os.environ['FAKE_PYMTP_MODE'] = 'ok' + hil_test.test_device_mtp(self.board) # no exception + + def test_absent_device_fails_cleanly(self): + os.environ['FAKE_PYMTP_MODE'] = 'absent' + finished, exc = run_bounded(lambda: hil_test.test_device_mtp(self.board), 30) + self.assertTrue(finished) + self.assertIsInstance(exc, AssertionError) + self.assertIn('MTP device not found', str(exc)) + + def test_libmtp_error_on_one_poll_retries_instead_of_dying(self): + """pymtp raises for USB_LAYER/PTP_LAYER errors -- routine on the first poll + after a flash. An unguarded raise skipped the whole enumeration budget.""" + os.environ['FAKE_PYMTP_MODE'] = 'error_then_ok' + hil_test.test_device_mtp(self.board) # retries past the error, then passes + + def test_libmtp_error_every_poll_fails_cleanly(self): + os.environ['FAKE_PYMTP_MODE'] = 'error' + finished, exc = run_bounded(lambda: hil_test.test_device_mtp(self.board), 30) + self.assertTrue(finished) + self.assertIsInstance(exc, AssertionError) + + def test_hung_mtp_stack_cannot_hang_the_worker(self): + # in-process libmtp blocking in a usbfs ioctl (D state) hangs whatever thread + # made the call, forever — the session must be somewhere disposable + os.environ['FAKE_PYMTP_MODE'] = 'hang' + hil_test.MTP_SESSION_MARGIN = 3 + finished, exc = run_bounded(lambda: hil_test.test_device_mtp(self.board), 25) + self.assertTrue(finished, 'test_device_mtp hung on a wedged MTP stack') + self.assertIsInstance(exc, AssertionError) + + +class ConvoySafeFlasher(unittest.TestCase): + """hil_flash.convoy_safe decides whether a board gets post-HUNG recovery at all. + + It must be true ONLY for flashers that can reach their probe without opening the + poisoned usbfs node: openocd pinned with a roster vid_pid (filters on kernel-cached + sysfs descriptors) and esptool (delivers to a named tty, never enumerates usbfs). + Anything else enumerates by opening nodes, would block in D state on the wedged one + and become a second stray -- JLinkExe included, whose selection is serial-only and + so cannot be pinned at all.""" + + def setUp(self): + import hil_flash + self.f = hil_flash.convoy_safe + + def test_pinned_openocd_is_safe(self): + self.assertTrue(self.f({'name': 'openocd', 'vid_pid': '0x2e8a 0x000c'})) + + def test_unpinned_openocd_is_not(self): + self.assertFalse(self.f({'name': 'openocd'})) + self.assertFalse(self.f({'name': 'openocd', 'vid_pid': ''})) + + def test_esptool_is_safe_without_a_pin(self): + """Delivery is `-p <ttyACM>`; there is no usbfs walk to poison.""" + self.assertTrue(self.f({'name': 'esptool'})) + + def test_enumerating_flashers_are_not(self): + for name in ('jlink', 'stlink', 'lm4flash', 'dfu-util'): + self.assertFalse(self.f({'name': name, 'vid_pid': '0x1366 0x1024'}), + f'{name} must not be treated as convoy-safe') + + def test_missing_or_odd_name_is_not_safe(self): + for flasher in ({}, {'name': None}, {'name': ''}): + self.assertFalse(self.f(flasher)) + + +class BoundedOpen(unittest.TestCase): + """hil_util.bounded_open must return rather than block, and must not leak the fd if + the open completes after we gave up (usblp_open takes the device mutex before it + consults O_NONBLOCK, so a wedged node blocks the open uninterruptibly).""" + + def setUp(self): + from helper import hil_util + self.hil_util = hil_util + self.tmp = TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + # bounded_open counts its stranded threads now, and the counter is process-global + # with no decrement: three wedged-FIFO tests here reach SYSFS_STUCK_MAX and every + # later test in this file reads SYSFS_UNKNOWN for perfectly good attributes + self.addCleanup(setattr, hil_util, '_sysfs_stuck', hil_util._sysfs_stuck) + + def test_opens_a_normal_file(self): + f = Path(self.tmp.name) / 'plain' + f.write_text('x') + fd = self.hil_util.bounded_open(str(f), os.O_RDONLY, 5) + self.assertIsNotNone(fd) + os.close(fd) + + def test_missing_path_returns_none_without_raising(self): + self.assertIsNone(self.hil_util.bounded_open( + str(Path(self.tmp.name) / 'nope'), os.O_RDONLY, 5)) + + @unittest.skipIf(os.name == 'nt', 'POSIX fifo') + def test_blocking_open_gives_up_and_does_not_leak_fds(self): + """A reader-less FIFO blocks open(O_WRONLY) forever -- the closest portable + stand-in for a wedged usblp node.""" + fifo = Path(self.tmp.name) / 'fifo' + os.mkfifo(fifo) + before = len(os.listdir('/proc/self/fd')) + t0 = time.monotonic() + for _ in range(5): + self.assertIs(self.hil_util.bounded_open(str(fifo), os.O_WRONLY, 0.2), + self.hil_util.SYSFS_UNKNOWN) + self.assertLess(time.monotonic() - t0, 10, 'bounded_open did not bound') + self.assertLessEqual(len(os.listdir('/proc/self/fd')) - before, 1, + 'bounded_open leaked fds on the blocking path') + + @unittest.skipIf(os.name == 'nt', 'POSIX fifo') + def test_open_completing_during_the_abandon_does_not_leak(self): + """The window the handoff lock exists for: the worker is at its store-or-close + decision when the caller gives up and drains the box. + + The `abandoned` Event is instrumented to park the worker there, because timing + alone never reaches that window -- 1500 tries against the unlocked version leaked + nothing, so a test that merely completes the open late proves nothing. An empty + `hit` means the instrumentation no longer bites and the window is untested.""" + hil_util = self.hil_util + fifo = Path(self.tmp.name) / 'fifo' + os.mkfifo(fifo) + caller = threading.current_thread() + drained, hit = threading.Event(), [] + + class RacingEvent(threading.Event): + def is_set(self): + v = super().is_set() + if not v and not hit and threading.current_thread() is not caller: + hit.append(True) + # bounded: the fixed bounded_open holds the lock across this call, so + # the caller cannot reach its abandon (and set drained) until we return + drained.wait(0.3) + return v + + shim = types.ModuleType('threading_shim') + shim.__dict__.update(threading.__dict__) + shim.Event = RacingEvent + hil_util.threading = shim + self.addCleanup(setattr, hil_util, 'threading', threading) + + before = len(os.listdir('/proc/self/fd')) + rd = os.open(fifo, os.O_RDONLY | os.O_NONBLOCK) # the O_WRONLY open completes at once + try: + self.assertIs(hil_util.bounded_open(str(fifo), os.O_WRONLY, 0.05), + hil_util.SYSFS_UNKNOWN) + drained.set() + time.sleep(0.1) # let an abandoned worker act on what it saw + self.assertTrue(hit, 'the abandon window was never entered') + self.assertLessEqual(len(os.listdir('/proc/self/fd')) - before, 1, + 'bounded_open stored the fd after the caller drained the box') + finally: + drained.set() + os.close(rd) + + +class SysfsUnknownIsNotAbsent(unittest.TestCase): + """read_sysfs must tell "no such attribute" (a fact) from "the read did not answer" + (not a fact). Every caller that concluded absence from the latter reported a healthy + board as a firmware regression.""" + + def setUp(self): + from helper import hil_util + self.hil_util = hil_util + self.saved = (hil_util._sysfs_stuck, hil_util._sysfs_blind_logged) + self.tmp = TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + + def tearDown(self): + # a blocked read strands a counted daemon thread; leaving the count raised would + # blind every later test in this process + self.hil_util._sysfs_stuck, self.hil_util._sysfs_blind_logged = self.saved + + def test_readable_attribute_returns_its_value(self): + p = Path(self.tmp.name) / 'serial' + p.write_text('CAFE01\n') + self.assertEqual(self.hil_util.read_sysfs(str(p)), 'CAFE01') + + def test_missing_attribute_is_none(self): + self.assertIsNone(self.hil_util.read_sysfs(str(Path(self.tmp.name) / 'nope'))) + + @unittest.skipIf(os.name == 'nt', 'POSIX fifo') + def test_blocking_read_is_unknown_not_absent(self): + """A reader-less FIFO stands in for the wedged device whose sysfs read never + returns; None here would read as "the board is gone".""" + fifo = Path(self.tmp.name) / 'fifo' + os.mkfifo(fifo) + t0 = time.monotonic() + v = self.hil_util.read_sysfs(str(fifo), grace=0.3) + self.assertLess(time.monotonic() - t0, 10, 'read_sysfs did not bound') + self.assertIs(v, self.hil_util.SYSFS_UNKNOWN) + self.assertIsNotNone(v) + + def test_blind_process_answers_unknown_for_a_readable_attribute(self): + p = Path(self.tmp.name) / 'serial' + p.write_text('CAFE01') + self.hil_util._sysfs_stuck = self.hil_util.SYSFS_STUCK_MAX + self.assertTrue(self.hil_util.sysfs_blind()) + self.assertIs(self.hil_util.read_sysfs(str(p)), self.hil_util.SYSFS_UNKNOWN) + self.assertIn('blind', self.hil_util.sysfs_blind_note()) + + def test_unknown_is_falsy_but_not_none(self): + # call sites use `(v or '')` idioms; the sentinel must keep working there while + # still being distinguishable from a real absence + self.assertFalse(self.hil_util.SYSFS_UNKNOWN) + self.assertIsNotNone(self.hil_util.SYSFS_UNKNOWN) + + +class UsbtestEnumerationVerdict(unittest.TestCase): + """test_device_usbtest must not report a healthy board as "no cafe:4010 device" just + because its own sysfs reads stopped answering.""" + + def setUp(self): + from helper import hil_util + self.hil_util = hil_util + self.td = TemporaryDirectory() + self.addCleanup(self.td.cleanup) + # a real device dir: usb_scan reads idVendor/idProduct with a plain open (they are + # lock-free descriptor fields), and only `serial` through the bounded reader + dev = Path(self.td.name) / '1-2' + dev.mkdir() + (dev / 'idVendor').write_text('cafe\n') + (dev / 'idProduct').write_text('4010\n') + (dev / 'serial').write_text('CAFE01\n') + for obj, name, val in ((hil_util, 'read_sysfs', hil_util.read_sysfs), + (hil_util, 'glob', hil_util.glob), + (hil_util, '_sysfs_stranded', {}), + (hil_test, '_enum_timeout', 1)): + self.addCleanup(setattr, obj, name, getattr(obj, name)) + setattr(obj, name, val) + hil_util.glob = types.SimpleNamespace(glob=lambda pat: [str(dev)]) + + def _fail(self, reader): + self.hil_util.read_sysfs = reader + with self.assertRaises(hil_test.TestFail) as cm: + hil_test.test_device_usbtest({'uid': 'CAFE01', 'name': 'fake', 'flasher': {}}) + return str(cm.exception) + + def test_unknown_reads_do_not_claim_the_device_is_absent(self): + msg = self._fail(lambda p, *a, **kw: self.hil_util.SYSFS_UNKNOWN) + self.assertNotIn('no cafe:4010 device', msg) + self.assertIn('did not answer', msg) + + def test_a_readable_bus_without_the_device_still_says_absent(self): + msg = self._fail(lambda p, *a, **kw: 'OTHERUID') + self.assertIn('no cafe:4010 device', msg) + + +class UnresolvedControllerBucket(unittest.TestCase): + """An unresolved controller must budget in ONE bucket. Taking a permit on every slot + serialized the whole fleet the moment a worker went blind.""" + + def setUp(self): + import threading + from helper import hil_lock + self.hil_lock = hil_lock + self.saved = (hil_lock.controller_map, hil_lock.controller_meta, + hil_lock.controller_hints, hil_lock.log) + hil_lock.controller_map, hil_lock.controller_meta = {}, threading.Lock() + hil_lock.controller_hints, hil_lock.log = {}, lambda *a, **k: None + + def tearDown(self): + (self.hil_lock.controller_map, self.hil_lock.controller_meta, + self.hil_lock.controller_hints, self.hil_lock.log) = self.saved + + def _slots(self, uid, warn): + import threading + sems = self.hil_lock.make_permit_sems(threading.Semaphore, 2) + return self.hil_lock.controller_permit(sems, uid, warn_unknown=warn).slots + + def test_unresolved_boards_share_one_slot(self): + for warn in (False, True): + slots = self._slots('NOSUCHUID', warn) + self.assertEqual(len(slots), 1, 'unresolved uid took more than one slot') + self.assertEqual(slots, self._slots('OTHERUID', warn), + 'unresolved boards must share the bucket, not spread over it') + + def test_the_semaphore_array_is_long_enough_for_the_unknown_slot(self): + """UNKNOWN_SLOT indexes one PAST the real slots. An array sized to + CONTROLLER_SLOTS IndexErrors on the first unresolved board, inside a pool worker, + which map_async turns into a total loss of every board's results.""" + import threading + sems = self.hil_lock.make_permit_sems(threading.Semaphore, 2) + self.assertGreater(len(sems), self.hil_lock.UNKNOWN_SLOT) + + def test_the_unknown_bucket_never_lends_a_controller_a_second_budget(self): + """A private FULL budget let 2 unknown batteries join 2 resolved ones on the same + physical controller -- 4 where the width is 2. One at a time caps that at +1.""" + import threading + sems = self.hil_lock.make_permit_sems(threading.Semaphore, 2) + first = self.hil_lock.controller_permit(sems, 'NOSUCHUID') + first.__enter__() + self.addCleanup(first.__exit__) + second = self.hil_lock.controller_permit(sems, 'OTHERUID') + self.assertFalse(sems[second.slots[0]].acquire(blocking=False), + 'a second unresolved board got in alongside the first') + + def test_every_real_slot_keeps_the_full_width(self): + import threading + sems = self.hil_lock.make_permit_sems(threading.Semaphore, 2) + for s in sems[:self.hil_lock.CONTROLLER_SLOTS]: + self.assertTrue(s.acquire(blocking=False) and s.acquire(blocking=False)) + self.assertFalse(s.acquire(blocking=False)) + + +class ThroughputPayloadBound(unittest.TestCase): + """An unknown link speed must pick the FS payload, and each dd must be bounded by the + payload actually requested.""" + + def test_only_a_read_high_speed_gets_the_big_payload(self): + from helper import hil_util + for speed in (None, hil_util.SYSFS_UNKNOWN, '12', '1.5'): + self.assertTrue(hil_test.link_is_fs(speed), f'{speed!r} must scale as FS') + for speed in ('480', '5000', '10000'): + self.assertFalse(hil_test.link_is_fs(speed)) + + def test_dd_bound_scales_with_the_payload_and_stays_bounded(self): + self.assertGreater(hil_test.dd_timeout(16), hil_test.dd_timeout(1)) + self.assertGreaterEqual(hil_test.dd_timeout(1), 30) # setup + flush floor + # still an INNER bound: run_cmd's own timeout must stay the outer one + self.assertLess(hil_test.dd_timeout(16), hil_test.hil_util.CMD_TIMEOUT) + + +class FindDeviceCache(unittest.TestCase): + """usbtest.find_device's cache is keyed by sysname, a bus-topology path: after a + renumber it can name a different cafe:4010 board, and idVendor/idProduct are identical + on every one of them. Only `serial` tells them apart.""" + + def setUp(self): + import usbtest + self.usbtest = usbtest + self.tmp = TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.saved_sys_usb = usbtest.SYS_USB + usbtest.SYS_USB = Path(self.tmp.name) + usbtest._DEV_CACHE.clear() + self._dev('1-2', 'AAAA', devnum=2) + self._dev('1-3', 'BBBB', devnum=3) + + def tearDown(self): + self.usbtest.SYS_USB = self.saved_sys_usb + self.usbtest._DEV_CACHE.clear() + + def _dev(self, sysname, serial, devnum): + d = Path(self.tmp.name) / sysname + d.mkdir() + for name, val in (('idVendor', self.usbtest.VID), ('idProduct', self.usbtest.PID), + ('serial', serial), ('busnum', '1'), ('devnum', str(devnum)), + ('speed', '480'), ('bcdDevice', '0104')): + (d / name).write_text(val + '\n') + + def test_cached_sysname_with_another_boards_serial_is_rejected(self): + self.usbtest._DEV_CACHE['bbbb'] = '1-2' # renumbered: 1-2 is board AAAA now + dev = self.usbtest.find_device('BBBB') + self.assertEqual(dev['sysname'], '1-3') + self.assertEqual(dev['serial'], 'BBBB') + self.assertEqual(self.usbtest._DEV_CACHE['bbbb'], '1-3') + + def test_cached_sysname_with_the_right_serial_is_kept(self): + self.usbtest._DEV_CACHE['bbbb'] = '1-3' + dev = self.usbtest.find_device('BBBB') + self.assertEqual((dev['sysname'], dev['serial']), ('1-3', 'BBBB')) + + def test_a_cached_device_that_vanished_falls_back_to_the_scan(self): + self.usbtest._DEV_CACHE['bbbb'] = '1-9' # gone from sysfs + self.assertEqual(self.usbtest.find_device('BBBB')['sysname'], '1-3') + + +class EnumPollDoesNotReReadAWedgedPath(unittest.TestCase): + """usbtest_enumerated re-globs every device each 0.2 s pass. One wedged peer therefore + strands a fresh bounded reader thread per pass, and SYSFS_STUCK_MAX=4 of those blind + the WHOLE worker for the rest of the run -- measured at 8 s of polling. A path that + already stranded is known-unknown; reading it again buys nothing and costs the + blindness budget.""" + + def test_a_stranded_path_is_read_at_most_once(self): + from contextlib import contextmanager + from helper import hil_lock, hil_util + + td = TemporaryDirectory() + self.addCleanup(td.cleanup) + # A REAL device dir: usb_scan reads idVendor/idProduct with a plain open and + # `continue`s on OSError, so a bare FIFO is skipped before the bounded read is ever + # reached -- this test passed identically with the memo deleted until the ids were + # added. The FIFO must be the `serial` of a device that survives the cheap filter. + devdir = Path(td.name) / '1-2' + devdir.mkdir() + (devdir / 'idVendor').write_text('cafe\n') + (devdir / 'idProduct').write_text('4010\n') + wedged = devdir / 'serial' + os.mkfifo(wedged) # open() blocks forever: no writer, ever + + def patch(obj, name, value): + self.addCleanup(setattr, obj, name, getattr(obj, name)) + setattr(obj, name, value) + + def _permit(uid): + yield + + from helper import hil_util as _hu2 + patch(_hu2, 'glob', types.SimpleNamespace(glob=lambda p: [str(devdir)])) + patch(_hu2, '_sysfs_stranded', {}) + patch(hil_lock, 'usbtest_permit', contextmanager(_permit)) + # Long enough for several 2 s reads, but under the blindness cap -- past the cap + # sysfs_blind() short-circuits reads on its own and would mask the memo entirely. + patch(hil_test, '_enum_timeout', 8) + # the blindness counter is process-global and never decrements: restore it or this + # test blinds every test that runs after it + patch(hil_util, '_sysfs_stuck', hil_util._sysfs_stuck) + + # Count LEAKED THREADS, not _sysfs_stuck: a strand is booked only the first time a + # path is seen, so the counter is deduped by the memo's own bookkeeping and stays 1 + # even when the memo is broken. Each re-read blocks a fresh thread on the FIFO + # forever and leaks its fd -- which is the cost the memo exists to avoid, and the + # only thing here that actually moves when it regresses. + before = threading.active_count() + with self.assertRaises(hil_test.TestFail): # never enumerates, by construction + hil_test.test_device_usbtest({'name': 'b', 'uid': 'UID1', + 'flasher': {'name': 'openocd'}}) + self.assertLessEqual(threading.active_count() - before, 1, + 'the poll re-read a path it already knew was stranded') + + +class ReRunSpecNamesOnlyWhatFailed(unittest.TestCase): + """The pool-guard path used to leave this unwritten -- and a fresh run has already + unlinked it -- so build.yml's re-run step found nothing and GitHub re-tested all ~26 + boards to find the one that wedged.""" + + def test_only_failed_boards_and_their_failed_tests(self): + with TemporaryDirectory() as td: + d = Path(td) + spec = d / 'cfg.failed' + hil_test._write_failed_spec(spec, d, [ + ('good', 0, [], None, 1.0), + ('bad', 2, ['device/cdc_msc'], None, 1.0), + ('wedged', 1, [], None, 0.0), # never reported: no test list + ]) + got = spec.read_text() + self.assertIn('-b bad', got) + self.assertIn('-bt bad:device/cdc_msc', got) + self.assertIn('-b wedged', got) + self.assertNotIn('good', got) + + def test_an_all_green_run_removes_a_stale_spec(self): + with TemporaryDirectory() as td: + d = Path(td) + spec = d / 'cfg.failed' + spec.write_text('--accumulate -b stale') + hil_test._write_failed_spec(spec, d, [('good', 0, [], None, 1.0)]) + self.assertFalse(spec.exists(), 'a stale spec would re-run last time\'s boards') + + +class WedgedPidsFailsClosed(unittest.TestCase): + """A scan that could not SEE the holder must not report "no holder". The holder is + root-owned (run_case uses sudo -n when the node is not writable) and that is exactly + what a hidepid/ProtectProc mount hides — so an unreadable /proc reading as clear + clears unrecovered_hang and lets cleanup unbind a device whose usbfs lock is still + held, which deadlocks the bus rather than one board.""" + + def test_returns_a_completeness_flag_not_just_pids(self): + import usbtest + got = usbtest.wedged_pids('/dev/bus/usb/999/999') + self.assertIsInstance(got, tuple) + self.assertEqual(len(got), 2, 'the caller needs (pids, complete)') + + def test_a_restricted_proc_is_reported_incomplete(self): + import usbtest + self.addCleanup(setattr, usbtest.os, 'geteuid', usbtest.os.geteuid) + self.addCleanup(setattr, usbtest.os, 'access', usbtest.os.access) + usbtest.os.geteuid = lambda: 1000 # not root + usbtest.os.access = lambda p, m: False # /proc/1/cmdline unreadable + _, complete = usbtest.wedged_pids('/dev/bus/usb/999/999') + self.assertFalse(complete, 'a hidden holder was reported as absent') + + [email protected](os.name == 'nt', 'POSIX shell fakes') [email protected](sys.version_info < (3, 11), 'fake-pymtp steering needs PYTHONSAFEPATH') +class StrandMemoRemembersUnstattablePaths(unittest.TestCase): + """A stranded path whose inode could not be read is stored as None -- which dict.get() + also returns for a MISS. Testing `is not None` therefore treats 'known stranded' as + 'never seen', and every later call strands ANOTHER permanent thread and fd on a path we + already know is wedged. That is the exact unbounded growth SYSFS_STUCK_MAX exists to + stop, and it is invisible: `first = path not in _sysfs_stranded` is False, so the + blindness counter does not advance either.""" + + def setUp(self): + from helper import hil_util + self.hil_util = hil_util + self.addCleanup(hil_util._sysfs_stranded.clear) + hil_util._sysfs_stranded.clear() + self.addCleanup(setattr, hil_util, '_sysfs_stuck', hil_util._sysfs_stuck) + hil_util._sysfs_stuck = 0 + self.td = TemporaryDirectory(); self.addCleanup(self.td.cleanup) + self.fifo = os.path.join(self.td.name, 'serial') + os.mkfifo(self.fifo) # open() succeeds, read() never returns + + def test_an_unstattable_strand_is_not_re_read(self): + self.hil_util._sysfs_stranded[self.fifo] = None # as the record path stores it + before = threading.active_count() + self.assertIs(self.hil_util.read_sysfs(self.fifo, grace=0.5), + self.hil_util.SYSFS_UNKNOWN) + self.assertEqual(threading.active_count(), before, + 'a known-stranded path was re-read, stranding another thread') + + def test_a_live_strand_is_still_re_read_when_the_node_is_replaced(self): + """The memo must not become permanent blindness: a NEW inode at the same path is a + different device and has to be read.""" + self.hil_util._sysfs_stranded[self.fifo] = 999999999 # inode that is not this one + with open(os.path.join(self.td.name, 'other'), 'w') as f: + f.write('ok\n') + os.replace(os.path.join(self.td.name, 'other'), self.fifo) + self.assertEqual(self.hil_util.read_sysfs(self.fifo, grace=0.5), 'ok') + + +class MtpGioOrdering(_MtpFakeRig, unittest.TestCase): + """gio must not run until the device is READY. + + gvfs claims an MTP device only AFTER udev probing, so the mount this unmounts cannot + exist before /dev/libmtp-<sysname> is published -- an unmount issued earlier is a + guaranteed no-op that still forks a process, and it leaves the window between the + unmount and the open unprotected, which is the hang it exists to prevent. Running it + per poll iteration also forks one gio per second of the enumeration budget.""" + + def setUp(self): + super().setUp() + tmp = Path(self.tmp.name) + self.gio_log = tmp / 'gio.log' + binn = tmp / 'bin'; binn.mkdir() + (binn / 'gio').write_text('#!/bin/sh\necho "$@" >> "$GIO_LOG"\n') + (binn / 'gio').chmod(0o755) + for k in ('PATH', 'GIO_LOG'): + old = os.environ.get(k) + self.addCleanup(lambda k=k, v=old: os.environ.__setitem__(k, v) + if v is not None else os.environ.pop(k, None)) + os.environ['GIO_LOG'] = str(self.gio_log) + os.environ['PATH'] = f'{binn}:{os.environ["PATH"]}' + + def _gio_calls(self): + return self.gio_log.read_text().splitlines() if self.gio_log.exists() else [] + + def test_gio_does_not_run_before_the_device_is_ready(self): + (Path(self.tmp.name) / 'dev' / 'libmtp-1-1').unlink() # never becomes ready + os.environ['FAKE_PYMTP_MODE'] = 'absent' + run_bounded(lambda: hil_test.test_device_mtp(self.board), 30) + calls = self._gio_calls() + self.assertEqual(calls, [], f'gio ran {len(calls)}x with no device ready: {calls}') + + def test_gio_still_runs_once_the_device_is_ready(self): + """The guard must delay the unmount, not delete it.""" + os.environ['FAKE_PYMTP_MODE'] = 'ok' + hil_test.test_device_mtp(self.board) + self.assertTrue(self._gio_calls(), 'gio never ran for a ready device') + + +class MtpGioFallthrough(unittest.TestCase): + """The missing-gio path must fall THROUGH to detection. `continue` there skips the + deadline check and the sleep as well, spinning at 100% CPU until the caller's outer + kill — reported as a wedged DUT for a missing apt package.""" + + def test_a_missing_gio_still_bounds_the_session(self): + import subprocess + with TemporaryDirectory() as td: + env = {**os.environ, 'PATH': td, # no gio, no anything + 'PYTHONPATH': os.path.join(TEST_DIR, 'stubs'), + 'FAKE_PYMTP_MODE': 'none', 'PYTHONSAFEPATH': '1'} + t0 = time.monotonic() + r = subprocess.run([sys.executable, + str(Path(TEST_DIR).parents[0] / 'mtp_test.py'), + '--uid', 'CAFE01', '--timeout', '3'], + capture_output=True, text=True, timeout=60, env=env) + elapsed = time.monotonic() - t0 + self.assertLess(elapsed, 30, f'did not honour --timeout 3 ({elapsed:.1f}s)') + self.assertNotEqual(r.returncode, 0) + # The assertions above are satisfied by an immediate CRASH, which is exactly what + # shipped through this test once: `pass` left gio unbound and the next line + # dereferenced it. Assert the behaviour the docstring names -- it POLLED for the + # device (so it spent its budget) and did not die on a traceback. + self.assertGreater(elapsed, 2.0, + f'exited without polling ({elapsed:.1f}s) -- it crashed') + self.assertNotIn('Traceback', r.stderr) + self.assertIn('MTP device not found', r.stdout + r.stderr) + + +class RunWhileContract(unittest.TestCase): + """The read-while-we-write runner. Its child can still outlast SIGKILL -- but unlike + the thread it replaced, an abandoned child is a real process in its own session, so + the containment sweep finds it and the report names it.""" + + def setUp(self): + from helper import hil_util + self.hil_util = hil_util + + def test_an_error_in_work_is_not_swallowed(self): + """A `return` inside the reap's `finally` discarded it: an assert in the CDC + write half vanished and the caller went on to compare data it never sent.""" + def boom(): + raise AssertionError('the write failed') + with self.assertRaises(AssertionError): + self.hil_util.run_alongside(['sh', '-c', 'printf X'], boom, 5) + + def test_the_child_is_reaped_even_when_work_raises(self): + seen = {} + + def boom(): + raise AssertionError('x') + with self.assertRaises(AssertionError): + self.hil_util.run_alongside(['sleep', '20'], boom, 1) + # nothing of ours is left running: the reap ran on the error path too + import subprocess + out = subprocess.run(['pgrep', '-f', '^sleep 20'], capture_output=True, text=True) + seen['strays'] = [p for p in out.stdout.split() if p] + self.assertEqual(seen['strays'], [], 'work() raising leaked the child') + + def test_an_abandoned_child_is_in_its_own_session(self): + """killpg on it reaps whatever it spawned, and it cannot take our group with it.""" + import subprocess + pgids = {} + + def check(): + time.sleep(0.2) + pgids['child'] = os.getpgid(self._proc_pid) + + real_popen = subprocess.Popen + + def spy(argv, **kw): + p = real_popen(argv, **kw) + self._proc_pid = p.pid + return p + self.addCleanup(setattr, subprocess, 'Popen', real_popen) + subprocess.Popen = spy + self.hil_util.run_alongside(['sleep', '0.5'], check, 5) + subprocess.Popen = real_popen + self.assertNotEqual(pgids['child'], os.getpgid(0)) + + +class StrandedPathMemoInvalidates(unittest.TestCase): + """The memo lives in read_sysfs, so every bounded reader gets it -- call-site memos + meant each new scanner had to remember (get_printer_dev and the throughput probe did + not). And it MUST expire on re-enumeration: the key is a bus path, which does not + change when a device comes back on the same port, so a memo that never invalidates + makes a board the branch's own HUNG reflash just recovered permanently invisible.""" + + def setUp(self): + from helper import hil_util + self.hil_util = hil_util + self.td = TemporaryDirectory() + self.addCleanup(self.td.cleanup) + for name in ('_sysfs_stranded', '_sysfs_stuck'): + self.addCleanup(setattr, hil_util, name, getattr(hil_util, name)) + hil_util._sysfs_stranded = {} + hil_util._sysfs_stuck = 0 + + def test_a_stranded_path_is_not_re_read(self): + f = Path(self.td.name) / 'serial' + os.mkfifo(f) # never answers + self.assertIs(self.hil_util.read_sysfs(str(f), 0.3), self.hil_util.SYSFS_UNKNOWN) + after_first = self.hil_util._sysfs_stuck + t0 = time.monotonic() + for _ in range(3): + self.assertIs(self.hil_util.read_sysfs(str(f), 0.3), + self.hil_util.SYSFS_UNKNOWN) + self.assertLess(time.monotonic() - t0, 0.3, 'the memo did not short-circuit') + self.assertEqual(self.hil_util._sysfs_stuck, after_first, + 'repeat reads spent more of the blindness budget') + + def test_re_enumeration_clears_it(self): + """A new device on the same busport gets a fresh sysfs node, hence a fresh inode. + Without this the memo outlives the wedge it recorded.""" + f = Path(self.td.name) / 'serial' + os.mkfifo(f) + self.assertIs(self.hil_util.read_sysfs(str(f), 0.3), self.hil_util.SYSFS_UNKNOWN) + f.unlink() + f.write_text('CAFE01\n') # same path, new inode = re-enumerated + self.assertEqual(self.hil_util.read_sysfs(str(f), 0.3), 'CAFE01', + 'a recovered device stayed invisible') + + def test_a_vanished_path_is_not_remembered_as_stranded(self): + f = Path(self.td.name) / 'serial' + os.mkfifo(f) + self.assertIs(self.hil_util.read_sysfs(str(f), 0.3), self.hil_util.SYSFS_UNKNOWN) + f.unlink() + self.assertIsNone(self.hil_util.read_sysfs(str(f), 0.3)) + + +class UsbScanIsTheOneWalk(unittest.TestCase): + """Three call sites each had a different subset of the three things this must get + right; none had all three. The expensive read is `serial` -- served under the device + lock a wedged usbfs ioctl holds -- so it must come LAST, only for devices the free + descriptor fields could not rule out, and never twice for a path that stranded.""" + + def setUp(self): + from helper import hil_util + self.hil_util = hil_util + self.td = TemporaryDirectory() + self.addCleanup(self.td.cleanup) + self.root = Path(self.td.name) + self.reads = [] + real = hil_util.read_sysfs + + def counting(path, *a, **k): + self.reads.append(path) + return real(path, *a, **k) + self.addCleanup(setattr, hil_util, 'read_sysfs', real) + hil_util.read_sysfs = counting + self.addCleanup(setattr, hil_util, '_sysfs_stranded', + dict(hil_util._sysfs_stranded)) + self.addCleanup(setattr, hil_util, '_sysfs_stuck', hil_util._sysfs_stuck) + + def _dev(self, name, vid, pid, serial='S1', fifo=False): + d = self.root / name + d.mkdir() + (d / 'idVendor').write_text(vid + '\n') + (d / 'idProduct').write_text(pid + '\n') + if fifo: + os.mkfifo(d / 'serial') # a read that never answers + else: + (d / 'serial').write_text(serial + '\n') + return d + + def _scan(self, **kw): + import glob as _g + real_glob = _g.glob + self.addCleanup(setattr, self.hil_util.glob, 'glob', real_glob) + self.hil_util.glob.glob = lambda pat: [str(p) for p in self.root.iterdir()] + return self.hil_util.usb_scan(**kw) + + def test_a_mismatched_vid_pid_costs_no_serial_read(self): + self._dev('1-1', '1234', '5678') + self._dev('1-2', 'cafe', '4010', serial='UID1') + devs, unknown = self._scan(vid_pid=('cafe', '4010')) + self.assertEqual([d['serial'] for d in devs], ['UID1']) + self.assertFalse(unknown) + # the ruled-out device's locked attribute was never touched + self.assertNotIn(str(self.root / '1-1' / 'serial'), self.reads) + + def test_a_wedged_device_stays_unproven_on_every_scan(self): + """The memo lives in read_sysfs now, so usb_scan still CALLS it each pass -- what + must not repeat is the cost. StrandedPathMemoInvalidates covers the short-circuit; + here the invariant is that the device stays out of the results and absence stays + unproven, however many times we look.""" + from helper import hil_util + self._dev('1-1', 'cafe', '4010', fifo=True) + first = None + t0 = time.monotonic() + for _ in range(3): + devs, unknown = self._scan() + self.assertTrue(unknown, 'a stranded read must leave absence unproven') + self.assertEqual(devs, []) + if first is None: + first = hil_util._sysfs_stuck + self.assertEqual(hil_util._sysfs_stuck, first, + 'repeat scans spent more of the blindness budget') + self.assertLess(time.monotonic() - t0, 3.0, 'repeat scans re-paid the grace') + + +class BoundedOpenTellsAbsentFromUnknown(unittest.TestCase): + """Same contract as read_sysfs, in the sibling function of the same file: a real + OSError is a FACT (EBUSY, ENOENT, EACCES), a blocked open is UNKNOWN. Folding both + into None made an ordinary EBUSY report as a USB wedge, sending the operator to + usb-kernel-recover for healthy hardware -- and left the stranded thread uncounted, + so the cap that exists to stop the fd/thread ceiling never saw it.""" + + def setUp(self): + from helper import hil_util + self.hil_util = hil_util + self.td = TemporaryDirectory() + self.addCleanup(self.td.cleanup) + + def test_a_real_oserror_is_a_fact(self): + missing = str(Path(self.td.name) / 'nope') + self.assertIsNone(self.hil_util.bounded_open(missing, os.O_RDONLY, 1)) + + def test_a_blocked_open_is_unknown_and_counted(self): + fifo = Path(self.td.name) / 'fifo' + os.mkfifo(fifo) # no reader: O_WRONLY blocks forever + self.addCleanup(setattr, self.hil_util, '_sysfs_stuck', + self.hil_util._sysfs_stuck) + before = self.hil_util._sysfs_stuck + got = self.hil_util.bounded_open(str(fifo), os.O_WRONLY, 0.3) + self.assertIs(got, self.hil_util.SYSFS_UNKNOWN) + self.assertEqual(self.hil_util._sysfs_stuck, before + 1, + 'a stranded open is invisible to the blindness budget') + + +class UsbtestSysfsReadIsCapped(unittest.TestCase): + """find_device re-scans every cafe:4010 peer after EVERY case, so the local twin -- + which had no SYSFS_STUCK_MAX -- stranded a thread and an fd per wedged peer per case. + Delegating to hil_util gets the cap, and the deferred import keeps usbtest.py + importable standalone.""" + + def test_a_stranded_read_counts_against_the_shared_cap(self): + import usbtest + from helper import hil_util + td = TemporaryDirectory() + self.addCleanup(td.cleanup) + wedged = Path(td.name) / 'serial' + os.mkfifo(wedged) # no writer: open() never returns + self.addCleanup(setattr, hil_util, '_sysfs_stuck', hil_util._sysfs_stuck) + before = hil_util._sysfs_stuck + # UNKNOWN, not None: folding them made a blinded scan read as "device dropped + # off the bus", which aborts past the HUNG reflash + self.assertIs(usbtest._read_sysfs_bounded(wedged, grace=0.5), + hil_util.SYSFS_UNKNOWN) + self.assertEqual(hil_util._sysfs_stuck, before + 1, + 'usbtest reads are invisible to the blindness budget') + + +class AbandonExitSurvivesAFailedFork(unittest.TestCase): + """Pool() forks, and after a convoy -- every stranded read holding a thread and an fd -- + that fork is what hits EAGAIN/ENOMEM. It now runs inside the try, so the finally can + reach _abandon_exit with pool and mgr still None.""" + + def test_none_pool_and_manager_still_write_the_banner(self): + # a subprocess, because _abandon_exit ends in os._exit: in-process it would take + # the test runner with it, before any assertion could run + import subprocess + with TemporaryDirectory() as td: + report = Path(td) / 'hil_report.md' + report.write_text('| board | test |\n|---|---|\n', encoding='utf-8') + src = ( + 'import sys, types\n' + f'sys.path.insert(0, {str(Path(TEST_DIR).parents[0])!r})\n' + 'st = types.ModuleType("serial")\n' + 'st.Serial = type("Serial", (), {})\n' + 'st.SerialException = type("SerialException", (Exception,), {})\n' + 'st.SerialTimeoutException = type("E2", (Exception,), {})\n' + 'sys.modules.setdefault("serial", st)\n' + 'import hil_test\n' + f'hil_test._abandon_exit(None, None, True, 1, __import__("pathlib")' + f'.Path({str(report)!r}))\n') + r = subprocess.run([sys.executable, '-c', src], capture_output=True, + text=True, timeout=120) + self.assertEqual(r.returncode, 1, r.stderr) + self.assertTrue(report.read_text().startswith('**HIL run abandoned'), + 'the abandon banner never reached the report') + + def test_kill_pool_children_tolerates_a_pool_that_never_existed(self): + from helper import hil_health + self.assertEqual(hil_health.kill_pool_children(None), 0) + self.assertEqual(hil_health.kill_pool_children(None, None), 0) + + +class UsbtestOuterBoundIsOneValue(unittest.TestCase): + """The bound usbtest is TOLD and the bound run_cmd ENFORCES must be the same number. + Three separate expressions disagreed: --skip-flash appended no --outer-timeout at all + (usbtest reads 0 as no limit), and the no-recovery branch narrowed only the CHILD's + view while run_cmd still waited for a recovery reserve nothing on that path can + spend -- a pool worker and its battery permit idle for the difference.""" + + def _invoke(self, flasher, skip_flash=False): + from contextlib import contextmanager + from helper import hil_lock, hil_util + + td = TemporaryDirectory() + self.addCleanup(td.cleanup) + dev = Path(td.name) / 'dev1' + dev.mkdir() + for attr, val in (('serial', 'UID1'), ('idVendor', 'cafe'), ('idProduct', '4010')): + (dev / attr).write_text(val + '\n') + + def patch(obj, name, value): + self.addCleanup(setattr, obj, name, getattr(obj, name)) + setattr(obj, name, value) + + def _permit(uid): + yield + + seen = {} + + def fake_run(cmd, **kw): + import subprocess + seen['cmd'], seen['timeout'] = cmd, kw.get('timeout') + return subprocess.CompletedProcess(cmd, 1, stdout=b'', stderr=b'stub') + + from helper import hil_util as _hu + patch(_hu, 'glob', types.SimpleNamespace(glob=lambda p: [str(dev)])) + # the blindness latch and the stranded memo are process-global: another class's + # wedged-FIFO test would otherwise make every read here answer SYSFS_UNKNOWN + patch(_hu, '_sysfs_stuck', 0) + patch(_hu, '_sysfs_stranded', {}) + patch(hil_lock, 'usbtest_permit', contextmanager(_permit)) + patch(hil_test, 'skip_flash', skip_flash) + patch(hil_test, '_current_fw', '/tmp/fw.elf') + patch(hil_util, 'run_cmd', fake_run) + with self.assertRaises(hil_test.TestFail): + hil_test.test_device_usbtest({'name': 'b', 'uid': 'UID1', 'flasher': flasher}) + return seen + + def _outer_flag(self, cmd): + toks = cmd.split() + self.assertIn('--outer-timeout', toks, 'usbtest reads a missing bound as UNLIMITED') + return int(toks[toks.index('--outer-timeout') + 1]) + + def test_a_recoverable_board_reserves_the_recovery_budget(self): + seen = self._invoke({'name': 'openocd', 'vid_pid': '0x1366 0x1024'}) + want = hil_test.USBTEST_BATTERY_BUDGET + hil_test.USBTEST_RECOVERY_BUDGET + self.assertEqual(self._outer_flag(seen['cmd']), want) + self.assertEqual(seen['timeout'], want) + + def test_a_board_with_no_recovery_does_not_pay_for_one(self): + seen = self._invoke({'name': 'stlink', 'uid': 'X'}) # never convoy_safe + outer = self._outer_flag(seen['cmd']) + self.assertEqual(seen['timeout'], outer, 'the two bounds disagree') + # It does not carry the RECOVERY reserve it cannot spend... + self.assertLess(outer, hil_test.USBTEST_BATTERY_BUDGET + + hil_test.USBTEST_RECOVERY_BUDGET) + # ...but it MUST still exceed the child's own --budget. The battery checks the + # budget before dispatching, so it can overshoot by one already-started case; an + # equal bound SIGKILLs it just as it goes to print, turning ~29 real per-case + # verdicts into "usbtest did not run" and re-paying the whole battery on retry. + toks = seen['cmd'].split() + budget = int(toks[toks.index('--budget') + 1]) + case_timeout = int(toks[toks.index('--timeout') + 1]) + self.assertGreaterEqual(outer - budget, case_timeout, + 'the outer kill can land mid-case, before the JSON') + + def test_skip_flash_still_bounds_the_child(self): + seen = self._invoke({'name': 'openocd', 'vid_pid': '0x1366 0x1024'}, skip_flash=True) + self.assertEqual(self._outer_flag(seen['cmd']), seen['timeout']) + + +class UsbtestRetryPolicy(unittest.TestCase): + """The pool guard bounds ONE battery; the retry loop multiplies it by max_retry. + So the loop must retry only what a retry can fix.""" + + def _patch(self, obj, name, value): + # addCleanup, not a finally: a failing assert must not leave the real module + # patched for whatever test runs next (max_retry only exists once main() ran, + # so restoring it means DELETING it again) + if hasattr(obj, name): + self.addCleanup(setattr, obj, name, getattr(obj, name)) + else: + self.addCleanup(delattr, obj, name) + setattr(obj, name, value) + + def _attempts(self, exc): + """How many times test_example runs the test fn before giving up.""" + import hil_flash + calls = [] + + def fake_test(board): + calls.append(1) + raise exc + + self._patch(hil_flash, 'find_firmware', lambda *a, **k: Path('/nonexistent/fw.elf')) + self._patch(hil_test, 'skip_flash', True) # no probe, no hardware + self._patch(hil_test, 'max_retry', 3) + self._patch(hil_test, 'log_line', lambda *a, **k: None) + hil_test.test_fake_example = fake_test + self.addCleanup(delattr, hil_test, 'test_fake_example') + hil_test.test_example({'name': 'b', 'uid': 'u', 'flasher': {'name': 'openocd'}}, + 'v', 'fake/example') + return len(calls) + + def test_a_per_case_verdict_is_not_retried(self): + # re-running the battery only re-observes a number the JSON already reported + self.assertEqual(self._attempts(hil_test.TestFail('29/30', parsed=True)), 1) + + def test_a_transient_failure_is_retried(self): + self.assertEqual(self._attempts(hil_test.TestFail('usbtest did not run')), 3) + + +class UsbtestOuterKillStaysRetryable(unittest.TestCase): + """rc 124 is run_cmd's timer expiring, NOT proof the DUT is wedged -- a healthy + battery can hit it under load. Suppressing the retry to save the budget also + suppresses the reflash test_example does before each attempt, which is the only + thing left to unpoison the DUT where usbtest's in-band recovery is off.""" + + def setUp(self): + from contextlib import contextmanager + from helper import hil_lock + self.td = TemporaryDirectory() + self.addCleanup(self.td.cleanup) + dev = Path(self.td.name) / 'dev1' + dev.mkdir() + # a real (readable) fake sysfs node, so the bounded reads run unmodified + for attr, val in (('serial', 'UID1'), ('idVendor', 'cafe'), ('idProduct', '4010')): + (dev / attr).write_text(val + '\n') + self.dev = dev + + def patch(obj, name, value): + saved = getattr(obj, name) + self.addCleanup(setattr, obj, name, saved) + setattr(obj, name, value) + + from helper import hil_util as _hu + patch(_hu, 'glob', types.SimpleNamespace(glob=lambda p: [str(dev)])) + def _permit(uid): # a real generator: a lambda returning an iterator has + yield # no .throw(), so any raise inside the `with` would + # surface as an AttributeError from contextlib instead + patch(hil_lock, 'usbtest_permit', contextmanager(_permit)) + patch(hil_test, 'skip_flash', True) + + def test_rc_124_stays_retryable(self): + import subprocess + from helper import hil_util + saved = hil_util.run_cmd + self.addCleanup(setattr, hil_util, 'run_cmd', saved) + hil_util.run_cmd = lambda *a, **k: subprocess.CompletedProcess( + 'usbtest', 124, stdout=b'', stderr=b'killed on the outer bound') + with self.assertRaises(hil_test.TestFail) as cm: + hil_test.test_device_usbtest({'name': 'b', 'uid': 'UID1', + 'flasher': {'name': 'openocd'}}) + self.assertFalse(cm.exception.parsed, + 'the retry is the last reflash a poisoned DUT gets') + + def test_a_crashed_tool_stays_retryable(self): + import subprocess + from helper import hil_util + saved = hil_util.run_cmd + self.addCleanup(setattr, hil_util, 'run_cmd', saved) + hil_util.run_cmd = lambda *a, **k: subprocess.CompletedProcess( + 'usbtest', 1, stdout=b'', stderr=b'ImportError: no module named usbtest') + with self.assertRaises(hil_test.TestFail) as cm: + hil_test.test_device_usbtest({'name': 'b', 'uid': 'UID1', + 'flasher': {'name': 'openocd'}}) + self.assertFalse(cm.exception.parsed) + + +class RemoteDirIsScreened(unittest.TestCase): + """REMOTE_DIR reaches the rig through `rm -rf`, an scp remote path and an rsync + remote path -- all re-split and expanded by the REMOTE shell, none of them + protectable by quoting the local variable. So the script screens the value once + instead: it must survive that re-split unchanged, and `~` must keep working.""" + + def _run(self, remote_dir, *args, keep_going=False): + import subprocess + with TemporaryDirectory() as td: + # real ssh/scp/rsync would reach the rig; these just record the argv. Exit 77 + # unless the caller needs the script to run on to the second ssh. + rc = 0 if keep_going else 77 + for tool in ('ssh', 'scp', 'rsync'): + write_script(Path(td) / tool, f'echo "stub-{tool} $*" >&2; exit {rc}') + env = {**os.environ, 'REMOTE_DIR': remote_dir, 'REMOTE': 'stub', + 'PATH': td + os.pathsep + os.environ['PATH']} + return subprocess.run( + ['bash', str(Path(TEST_DIR).parents[0] / 'hil_ci.sh'), *args], + capture_output=True, text=True, timeout=60, env=env) + + def test_whitespace_is_refused(self): + # unscreened, the remote `rm -rf -- "$1"` gets a TRUNCATED path and deletes + # the wrong tree + r = self._run('/tmp/hil dir') + self.assertNotEqual(r.returncode, 0) + self.assertIn('REMOTE_DIR', r.stderr) + + def test_command_substitution_is_refused(self): + r = self._run('/tmp/$(touch pwned)') + self.assertNotEqual(r.returncode, 0) + self.assertIn('REMOTE_DIR', r.stderr) + + def test_bare_root_is_refused(self): + r = self._run('/') + self.assertNotEqual(r.returncode, 0) + self.assertIn('REMOTE_DIR', r.stderr) + + def test_a_tilde_path_is_accepted(self): + """The one override %q broke: `~` must reach the remote shell UNESCAPED or it + creates a literal '~' directory in the login dir.""" + r = self._run('~/tinyusb-hil') + self.assertIn('~/tinyusb-hil', r.stderr) # got as far as the first ssh + self.assertNotIn('\\~', r.stderr) # %q escapes it; the remote shell won't + + def test_paths_that_would_rm_rf_something_huge_are_refused(self): + """Passing the tilde through UNESCAPED is what makes this dangerous: the remote + shell expands `~/` to the login dir, so `rm -rf -- "$1"` takes out $HOME -- one + typo away from the documented REMOTE_DIR=~/dir override. A bare root, a + no-component path and a foreign ~user are the same class.""" + for bad in ('~/', '~root/x', '~-', '//', '/.', '/tmp/hil/'): + with self.subTest(remote_dir=bad): + r = self._run(bad) + self.assertNotEqual(r.returncode, 0, f'{bad!r} was accepted') + self.assertIn('REMOTE_DIR', r.stderr) + + def test_an_arg_containing_a_space_survives_the_remote_resplit(self): + """ssh joins its argv into ONE string the remote shell re-splits, so an unquoted + `-t 'host/cdc msc'` arrives as two arguments and hil_test.py sees a stray word + where it expects the config path.""" + r = self._run('/tmp/tinyusb-hil', '-t', 'host/cdc msc', keep_going=True) + run_line = [l for l in r.stderr.splitlines() if 'bash -s --' in l][-1] + self.assertIn(r'host/cdc\ msc', run_line) + + +class CaveatSurvivesAccumulate(unittest.TestCase): + """CI reruns with --accumulate: the sidecar keeps every earlier attempt's cells, but the + banner was recomputed per attempt. A first attempt on a degraded rig and a clean rerun + therefore published the degraded attempt's PASSES with no caveat on them -- and the + generated .failed spec reruns only failures, so those cells are never re-earned.""" + + def _rows(self, board, cell): + return [(board, 0, 0, [(board, {cell: 'OK'}, '1s')], 0)] + + def test_an_earlier_attempts_caveat_is_still_on_the_report(self): + td = TemporaryDirectory() + self.addCleanup(td.cleanup) + rd = Path(td.name) + banner = '> **Rig note.** 2 process(es) in D state at start.\n' + + hil_test.accumulate_report(self._rows('boardA', 'cdc_msc'), rd, True, '', banner) + self.assertIn('Rig note', (rd / hil_test.REPORT_MD).read_text()) + + # the rerun: clean rig, so this attempt contributes no banner of its own + md = hil_test.accumulate_report(self._rows('boardB', 'cdc_msc'), rd, False, '', '') + self.assertIn('boardA', md) # the earlier cells are kept ... + self.assertIn('Rig note', md, + 'the caveat the earlier cells were collected under was dropped') + + def test_the_same_caveat_twice_is_not_stacked(self): + td = TemporaryDirectory() + self.addCleanup(td.cleanup) + rd = Path(td.name) + banner = '> **Rig note.** 2 process(es) in D state at start.\n' + hil_test.accumulate_report(self._rows('boardA', 'cdc_msc'), rd, True, '', banner) + md = hil_test.accumulate_report(self._rows('boardB', 'cdc_msc'), rd, False, '', banner) + self.assertEqual(md.count('Rig note'), 1) + + +class BlindWorkerReachesTheReport(unittest.TestCase): + """A worker that exhausts its bounded-read budget answers SYSFS_UNKNOWN for every + attribute, so its "device not found" means "could not tell". That reached the log and + the per-cell failure text but NOT the table -- and the table is what gets pasted into + the PR. Seen live: run 31794359407 went blind in 4 workers and published 26 red cells + with no mention of it, several of them caused by the blindness rather than the board.""" + + def test_no_note_when_every_worker_could_see(self): + mret = [('boardA', 0, [], [], 1.0, False), ('boardB', 0, [], [], 1.0, False)] + self.assertEqual(hil_test._blind_note(mret), '') + + def test_the_note_names_the_boards_whose_verdicts_are_not_evidence(self): + mret = [('boardA', 0, [], [], 1.0, True), ('boardB', 0, [], [], 1.0, False), + ('boardC', 1, [], [], 1.0, True)] + note = hil_test._blind_note(mret) + self.assertIn('boardA', note) + self.assertIn('boardC', note) + self.assertNotIn('boardB', note) # it could see; do not smear its result + self.assertTrue(note.endswith('\n'), 'banners are line-oriented') + + def test_both_row_widths_survive_the_report_writers(self): + """The blindness flag widened the worker's result tuple to 6, but the pool-timeout + path still synthesises 5-field rows for boards that never reported and feeds them + to the same two writers. A fixed-width unpack in either one raises INSIDE the + containment path, which is where a raise costs every board's results.""" + td = TemporaryDirectory() + self.addCleanup(td.cleanup) + rd = Path(td.name) + wide = ('boardA', 1, ['device/cdc_msc'], [('boardA', {'cdc_msc': '❌'}, '2s')], 2.0, True) + narrow = ('stuck', 1, [], None, 0) # what the timeout path builds + hil_test._write_failed_spec(rd / 'x.failed', rd, [wide, narrow]) + md = hil_test.accumulate_report([wide], rd, True, '', hil_test._blind_note([wide])) + self.assertIn('boardA', md) + self.assertIn('not all verdicts are evidence', md.lower()) + + def test_the_stray_note_names_the_board_and_survives_narrow_rows(self): + """Survivors ride back on the result tuple because main()'s own sweep runs after + the report is written on both abort paths -- the banner appended there was + computed and discarded.""" + wide = ('boardA', 0, [], [], 1.0, False, 2) + clean = ('boardB', 0, [], [], 1.0, False, 0) + note = hil_test._stray_note([wide, clean]) + self.assertIn('boardA', note) + self.assertNotIn('boardB', note) + self.assertIn('2', note) + self.assertEqual(hil_test._stray_note([clean]), '') + self.assertEqual(hil_test._stray_note([('stuck', 1, [], None, 0)]), '') + + def test_the_timeout_paths_synthetic_rows_do_not_crash_it(self): + """The pool-timeout path builds (name, 1, [], None, 0) for boards that never + reported -- five fields, no blindness to report -- and hands those around.""" + self.assertEqual(hil_test._blind_note([('stuck', 1, [], None, 0)]), '') + + +class PoolGuardKeepsWhatFinished(unittest.TestCase): + """The guard's 30-minute predecessor fired on 5 of the last 8 HIL jobs, so this is the + common failure, not an edge case: map_async discarded every board that had finished and + left the re-run spec unwritten, so CI re-tested all ~26 to find the one that wedged. + + Calls hil_test.drain_pool -- the loop main() actually runs. The predecessor of this test + built its own ThreadPool and its own drain loop and asserted on those, so deleting the + production drain outright left it green.""" + + class _It: + """Stands in for imap_unordered: yields, then blocks past any deadline.""" + + def __init__(self, ready): + self.ready, self.i = ready, 0 + + def next(self, timeout=None): + if self.i < len(self.ready): + self.i += 1 + return self.ready[self.i - 1] + raise MpTimeoutError + + def test_finished_rows_survive_a_guard_expiry(self): + boards = [{'name': 'fast1'}, {'name': 'fast2'}, {'name': 'wedged'}] + rows = [('fast1', 0, [], [], 1.0, False), ('fast2', 0, [], [], 1.0, False)] + with self.assertRaises(hil_test.PoolDrainTimeout) as cm: + hil_test.drain_pool(self._It(rows), boards, time.monotonic() + 5) + self.assertEqual([r[0] for r in cm.exception.finished], ['fast1', 'fast2']) + + def test_an_expired_deadline_stops_before_asking_for_more(self): + """Left <= 0 must not be handed to it.next() as a zero/negative timeout.""" + boards = [{'name': 'a'}, {'name': 'b'}] + it = self._It([('a', 0, [], [], 1.0, False)]) + with self.assertRaises(hil_test.PoolDrainTimeout) as cm: + hil_test.drain_pool(it, boards, time.monotonic() - 1) # already past + self.assertEqual(cm.exception.finished, []) + self.assertEqual(it.i, 0, 'asked the pool for a result after the deadline') + + def test_rows_collected_before_the_deadline_expires_are_kept_too(self): + """The OTHER raise site: boards finish, then the clock runs out between results. + Both sites must carry the rows -- a bare raise here loses a worker-width of rig + time just as map_async did, and the it.next() path alone does not prove it.""" + class Slow(self._It): + def next(self, timeout=None): + time.sleep(0.2) # each result eats into the deadline + return super().next(timeout) + + boards = [{'name': n} for n in ('a', 'b', 'c', 'd')] + rows = [(n, 0, [], [], 1.0, False) for n in ('a', 'b', 'c', 'd')] + with self.assertRaises(hil_test.PoolDrainTimeout) as cm: + hil_test.drain_pool(Slow(rows), boards, time.monotonic() + 0.3) + self.assertTrue(cm.exception.finished, 'rows collected before the expiry were lost') + + def test_every_board_finishing_returns_them_all(self): + boards = [{'name': 'a'}, {'name': 'b'}] + rows = [('a', 0, [], [], 1.0, False), ('b', 1, [], [], 2.0, False)] + got = hil_test.drain_pool(self._It(rows), boards, time.monotonic() + 5) + self.assertEqual(got, rows) + + +class WedgedBoardCosts(unittest.TestCase): + """Two decisions the containment latch makes, tested as decisions rather than through + test_board's loop -- the loop-level predecessor of these tests reimplemented that loop + and asserted on its own copy, which is how both defects survived it.""" + + def setUp(self): + self.addCleanup(setattr, hil_test, 'board_wedged', hil_test.board_wedged) + + def test_a_board_that_wedged_still_counts_as_an_error(self): + """It rendered a red cell but returned err_count 0, so main()'s sys.exit(err_count) + reported success and _write_failed_spec (`if err > 0`) left the board out of the + re-run entirely: a rig holding a D-state process published as a clean pass.""" + hil_test.board_wedged = 'usbtest HUNG' + # no real flasher: skip_flash isolates the accounting from hil_flash + self.addCleanup(setattr, hil_test, 'skip_flash', hil_test.skip_flash) + hil_test.skip_flash = True + # a firmware path must resolve or test_example returns 'skip (no binary)' before + # ever reaching the retry loop this is about + self.addCleanup(setattr, hil_flash, 'find_firmware', hil_flash.find_firmware) + hil_flash.find_firmware = lambda *a, **k: Path('fw.elf') + + def boom(*a, **k): + raise hil_test.TestFail('usbtest did not run') # unparsed: retryable + + self.addCleanup(setattr, hil_test, 'test_device_usbtest', hil_test.test_device_usbtest) + hil_test.test_device_usbtest = boom + board = {'name': 'b', 'uid': 'U', 'flasher': {'name': 'openocd'}, 'tests': []} + err, _status, _metric = hil_test.test_example(board, 'b', 'device/usbtest') + self.assertEqual(err, 1, 'a wedged board contributed nothing to the exit status') + + def test_the_teardown_park_does_not_flash_a_wedged_board(self): + """The park is a flash like any other: on a D-state-held node it blocks, survives + SIGKILL and leaves a stray -- added by the path that just declared the board wedged + and skipped every test for exactly that reason.""" + hil_test.board_wedged = '' + self.assertTrue(hil_test._should_park(False), 'a healthy board must still park') + hil_test.board_wedged = 'usbtest HUNG' + self.assertFalse(hil_test._should_park(False), + 'the teardown park would flash through the poisoned node') + self.assertFalse(hil_test._should_park(True), '--skip-flash must still suppress it') + + +class WedgeVerdictReachesTheLatch(unittest.TestCase): + """usbtest computes `unrecovered_hang` but never reported it, so hil_test inferred the + latch from `not recovery and 'HUNG' in out` and missed three cases: recovery ran and + FAILED (convoy-safe boards -- max32666fthr HUNG in the 08-14 run), the `inconclusive` + abort (which sets the flag but leaves no case at status HUNG), and an unparsable JSON, + which is the outer-timeout kill and the case where a wedge is most likely.""" + + def setUp(self): + self.addCleanup(setattr, hil_test, 'board_wedged', hil_test.board_wedged) + hil_test.board_wedged = '' + + def _run(self, stdout, rc=0): + from helper import hil_lock, hil_util + class R: + returncode = rc + stderr = b'' + R.stdout = stdout.encode() + self.addCleanup(setattr, hil_util, 'run_cmd', hil_util.run_cmd) + hil_util.run_cmd = lambda *a, **k: R() + # usbtest_enumerated is nested in test_device_usbtest, so stub what it calls + self.addCleanup(setattr, hil_util, 'usb_scan', hil_util.usb_scan) + hil_util.usb_scan = lambda **k: ([{'busport': '1-1', 'dir': '/x', 'vid': 'cafe', + 'pid': '4010', 'serial': 'U'}], False) + self.addCleanup(setattr, hil_lock, 'usbtest_permit', hil_lock.usbtest_permit) + from contextlib import contextmanager + hil_lock.usbtest_permit = contextmanager(lambda uid: iter([None])) + board = {'name': 'b', 'uid': 'U', 'flasher': {'name': 'openocd', 'vid_pid': '0x1 0x2'}} + try: + hil_test.test_device_usbtest(board) + except Exception: + pass + return hil_test.board_wedged + + def test_a_reported_wedge_latches_even_when_recovery_ran(self): + """`recovery` True means the flags were PASSED, not that they worked.""" + js = '{"serial":"U","speed":"480","tier":1,"passed":1,"failed":1,"notrun":0,' '"wedged":true,"cases":[{"num":1,"status":"FAIL"}]}' + self.assertTrue(self._run(js), 'a reported wedge did not latch') + + def test_no_wedge_reported_does_not_latch(self): + js = '{"serial":"U","speed":"480","tier":1,"passed":2,"failed":0,"notrun":0,' '"wedged":false,"cases":[]}' + self.assertFalse(self._run(js)) + + def test_an_unparseable_battery_that_mentions_HUNG_still_latches(self): + """rc 124 mid-print: no JSON to read, and this is the likeliest real wedge.""" + self.assertTrue(self._run('TEST 10 HUNG: device wedged mid-transfer', rc=124)) + + +class WedgedBoardCannotReportAPass(unittest.TestCase): + """The latch alone is not enough: it is set BEFORE the pass return, so an all-green + battery that still wedged returned `PASS 30/30`. That board then contributes 0 to + err_count, is omitted from the .failed re-run spec (which keys on err > 0), and the job + exits 0 with a D-state holder on the rig -- the exact silence this branch exists to end. + usbtest's `inconclusive` and `ambiguous` aborts fire AFTER the last case, so nothing + back-fills a BUDGET entry to make failed/notrun non-zero.""" + + def setUp(self): + self.addCleanup(setattr, hil_test, 'board_wedged', hil_test.board_wedged) + hil_test.board_wedged = '' + + def _cell(self, js): + """Returns ('pass', cell) or ('fail', message).""" + from helper import hil_lock, hil_util + class R: + returncode = 0 + stderr = b'' + R.stdout = js.encode() + self.addCleanup(setattr, hil_util, 'run_cmd', hil_util.run_cmd) + hil_util.run_cmd = lambda *a, **k: R() + self.addCleanup(setattr, hil_util, 'usb_scan', hil_util.usb_scan) + hil_util.usb_scan = lambda **k: ([{'busport': '1-1', 'dir': '/x', 'vid': 'cafe', + 'pid': '4010', 'serial': 'U'}], False) + self.addCleanup(setattr, hil_lock, 'usbtest_permit', hil_lock.usbtest_permit) + from contextlib import contextmanager + hil_lock.usbtest_permit = contextmanager(lambda uid: iter([None])) + board = {'name': 'b', 'uid': 'U', 'flasher': {'name': 'openocd', 'vid_pid': '0x1 0x2'}} + try: + return ('pass', hil_test.test_device_usbtest(board)) + except hil_test.TestFail as e: + return ('fail', str(e)) + + def test_an_all_pass_battery_that_wedged_is_not_a_pass(self): + kind, detail = self._cell('{"serial":"U","speed":"480","tier":1,"passed":30,' + '"failed":0,"notrun":0,"wedged":true,"cases":[]}') + self.assertEqual(kind, 'fail', f'a wedged board reported a green cell: {detail}') + self.assertIn('wedged', detail) + + def test_an_all_pass_battery_that_did_not_wedge_is_still_a_pass(self): + """The guard must key on the latch, not merely on having parsed a battery.""" + kind, cell = self._cell('{"serial":"U","speed":"480","tier":1,"passed":30,' + '"failed":0,"notrun":0,"wedged":false,"cases":[]}') + self.assertEqual(kind, 'pass', f'a healthy board was failed: {cell}') + self.assertIn('30/30', cell) + + +if __name__ == '__main__': + unittest.main() diff --git a/test/hil/test/test_hil_health.py b/test/hil/test/test_hil_health.py new file mode 100644 index 000000000..5695cad6d --- /dev/null +++ b/test/hil/test/test_hil_health.py @@ -0,0 +1,636 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +# Unit tests for hil_health.py — pure logic against a synthetic /proc, no hardware. A real +# wedge cannot be manufactured on demand, so the detectors are exercised against fabricated +# inputs. hil_health is stdlib-only on purpose, so all of this runs on a bare CI runner +# with nothing skipped. Run directly: +# python3 test/hil/test/test_hil_health.py +import os +import signal +import sys +import threading +import time +import subprocess +import unittest +from multiprocessing import Pool +from pathlib import Path +from tempfile import TemporaryDirectory + +# the module under test lives in the parent dir (test/hil), not here +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) +from helper import hil_health + +REAL_PROC = hil_health.PROC + + +def make_proc(root: Path, procs: dict, with_pid1: bool = True) -> None: + """Build a synthetic /proc. `procs` maps pid -> (comm, state, cmdline); a None comm or + cmdline omits that file. `with_pid1=False` simulates a restricted /proc (hidepid=2), + where an empty scan must not be read as an all-clear.""" + for pid, (comm, state, cmdline) in procs.items(): + d = root / str(pid) + d.mkdir() + if comm is not None: + (d / 'comm').write_text(comm + '\n') + if cmdline is not None: + (d / 'cmdline').write_bytes(cmdline) + # field 2 is comm in parens; the state letter follows it. Deliberately use a comm + # containing ')' so a naive split() would pick the wrong field. + (d / 'stat').write_text(f'{pid} (we)ird) {state} 1 1 0 0 -1 0 0\n') + if with_pid1 and 1 not in procs: + d = root / '1' + d.mkdir() + (d / 'comm').write_text('systemd\n') + (d / 'cmdline').write_bytes(b'/sbin/init\0') + (d / 'stat').write_text('1 (systemd) S 0 1 0 0 -1 0 0\n') + (root / 'not-a-pid').mkdir() + + +class PatchCase(unittest.TestCase): + """For classes that patch PROCESS-GLOBAL state (os.kill, time.sleep, subprocess.Popen). + + addCleanup, never tearDown: tearDown does NOT run when setUp raises, so a no-op + os.kill or time.sleep would survive into every later test in this blocking pre-commit + suite -- turning one setUp failure into a cascade of nonsense results.""" + + def patch(self, obj, name, value): + self.addCleanup(setattr, obj, name, getattr(obj, name)) + setattr(obj, name, value) + + def restore(self, obj, name): + """Same guarantee for state a TEST BODY assigns directly: register the restore + from setUp so it holds even when the assert between fails.""" + self.addCleanup(setattr, obj, name, getattr(obj, name)) + + +class ProcCase(unittest.TestCase): + """Every subclass repoints hil_health.PROC at a temp tree; restore it so a later test + cannot silently keep scanning a deleted directory.""" + + def tearDown(self): + hil_health.PROC = REAL_PROC + + +class ShutdownPool(unittest.TestCase): + def test_returns_true_when_the_pool_terminates(self): + pool = Pool(processes=1) + try: + self.assertTrue(hil_health.shutdown_pool(pool, grace=30)) + finally: + pool.terminate() + + def test_returns_false_instead_of_blocking_forever(self): + """The real failure is a worker in uninterruptible sleep, which cannot be created + from userspace. What matters is that shutdown_pool gives up on the deadline rather + than hanging, because the caller must then abandon the pool to free the job slot.""" + # Cancellable, not time.sleep(3600): shutdown_pool returns while its daemon thread + # is still inside terminate(), and an uninterruptible sleep there outlives the test. + # The next test alphabetically forks a real Pool, so the leaked thread made it + # fork-from-multithreaded ('DeprecationWarning: ... may lead to deadlocks in the + # child') and its result order-dependent. addCleanup releases it either way. + release = threading.Event() + self.addCleanup(release.set) + + class NeverDies: + def terminate(self): + release.wait(3600) + + start = time.monotonic() + self.assertFalse(hil_health.shutdown_pool(NeverDies(), grace=0.5)) + self.assertLess(time.monotonic() - start, 10) + + def test_a_raising_terminate_counts_as_failure(self): + """The thread dies on the exception, so is_alive() goes False -- which would report + success for a pool that is just as alive as if terminate() had hung.""" + class Explodes: + def terminate(self): + raise RuntimeError('boom') + + self.assertFalse(hil_health.shutdown_pool(Explodes(), grace=5)) + + +class ChildProcs(ProcCase): + """A pool worker's own group is OUR group (multiprocessing never setpgid's), so its + children can only be found by walking ppid -> pgrp in /proc.""" + + def test_grandchildren_are_swept_too(self): + """usbtest.py (child, own session) spawns its recovery reflash via run_cmd (own + session again): the flasher is a GRANDCHILD no direct-child walk covers, and a + pool-guard kill mid-recovery would orphan it on the probe.""" + got = self.scan([100], { + 100: ('worker', 1, 4242), + 200: ('usbtest.py', 100, 200), # child, own session + 300: ('openocd', 200, 300), # grandchild flasher, own session + 999: ('unrelated', 1, 999), + }) + self.assertEqual(sorted(got.get(100, [])), [(200, 200), (300, 300)]) + + def scan(self, pids, procs): + """`procs` maps pid -> (comm, ppid, pgrp); a None comm omits the stat file.""" + with TemporaryDirectory() as td: + root = Path(td) + for p, (comm, ppid, pgrp) in procs.items(): + d = root / str(p) + d.mkdir() + if comm is not None: + (d / 'stat').write_text(f'{p} ({comm}) S {ppid} {pgrp} 0 0 -1 0 0\n') + (root / 'not-a-pid').mkdir() + hil_health.PROC = root + return hil_health.child_procs(pids) + + def test_finds_direct_children_only(self): + got = self.scan([100], { + 100: ('python3', 1, 4242), # the worker itself + 201: ('openocd', 100, 201), # its detached flasher + 202: ('usbtest.py', 100, 202), # a second detached session + 303: ('unrelated', 7, 303), # someone else's child + }) + self.assertEqual({100: [(201, 201), (202, 202)]}, + {k: sorted(v) for k, v in got.items()}) + + def test_covers_every_parent_in_one_walk(self): + """One pass for all workers, not one pass each: this runs on the free-the-runner + path, and per-parent walks would also see different snapshots.""" + got = self.scan([100, 101], { + 201: ('openocd', 100, 201), + 202: ('JLinkExe', 101, 202), + }) + self.assertEqual(got, {100: [(201, 201)], 101: [(202, 202)]}) + + def test_parses_a_comm_containing_spaces_and_parens(self): + """A naive split() on the whole line would read the wrong fields.""" + got = self.scan([100], {500: ('we ) ird', 100, 500)}) + self.assertEqual(got, {100: [(500, 500)]}) + + def test_reports_a_child_that_shares_our_group(self): + """subprocess.run children (arecord, iperf) get no new session, so they land in + our group. They must still be REPORTED -- kill_pool_children signals them by pid, + since killpg on that group would take down the run itself.""" + got = self.scan([100], {201: ('arecord', 100, 4242)}) + self.assertEqual(got, {100: [(201, 4242)]}) + + def test_tolerates_unreadable_and_truncated_entries(self): + got = self.scan([100], { + 201: (None, 0, 0), # stat missing (exited mid-scan) + 202: ('openocd', 100, 202), # still found + }) + self.assertEqual(got, {100: [(202, 202)]}) + + def test_returns_empty_when_proc_is_unreadable(self): + hil_health.PROC = Path('/nonexistent-proc-for-test') + self.assertEqual(hil_health.child_procs([100]), {}) + + +class FakeProc: + """Stands in for a multiprocessing worker: kill_pool_children goes through + is_alive() and Process.kill(), whose internal returncode guard is what protects + against signalling a recycled pid.""" + + def __init__(self, pid, alive=True, wedged=False): + self.pid = pid + self._alive = alive + self._wedged = wedged # D state: ignores SIGKILL, so is_alive() stays True + self.killed = False + + def is_alive(self): + return self._alive + + def kill(self): + self.killed = True + # A signalled worker DIES unless it is wedged. Modelling every worker as an + # unkillable survivor sent all of them down the confirm/sudo ladder, which is + # what let literal pids reach the real os.kill. + if not self._wedged: + self._alive = False + + +class KillWorkerChildren(PatchCase): + # os.getpgid/killpg are stubbed for the whole class: FakeProc pids are literals like + # 101, which are live pids on a real machine, so an unstubbed killpg SIGKILLs a real + # process GROUP. That happened while writing this and killed the test run itself. + """What the workers spawned, killed while their parents are still alive. + + Verified premise: Pool.terminate() reaps a worker that is merely waiting in + communicate() on a wedged flasher, reparenting that flasher to init -- so this must + run BEFORE shutdown_pool(), or the ppid link is gone and a successful terminate() + skips the cleanup entirely.""" + + OWN_PGID = 4242 + + def setUp(self): + # _kill_and_confirm's grace poll must not touch the real /proc: fake pid + # 900 can be a live process on the host, which stalls the poll for the full grace + # and prints a false survivor warning into the blocking pre-commit hook. + self.proc_tmp = TemporaryDirectory() + self.addCleanup(self.proc_tmp.cleanup) + self.patch(hil_health, 'PROC', Path(self.proc_tmp.name)) # empty: unreadable -> gone + self.patch(hil_health, 'CONFIRM_KILL_GRACE', 0.05) + # the sweep runs two passes with a real gap; the fakes never respawn, so + # stub the wait rather than pay it in every test + self.patch(hil_health.time, 'sleep', lambda _s: None) + self.groups, self.pids = [], [] + self.children = {} # worker pid -> [(pid, pgid), ...] + self.patch(hil_health, 'child_procs', lambda pids: self.children) + self.patch(os, 'killpg', lambda pgid, sig: self.groups.append((pgid, sig))) + self.patch(os, 'kill', lambda pid, sig: self.pids.append((pid, sig))) + self.patch(os, 'getpgid', lambda pid: self.OWN_PGID) + # overwritten directly by some test bodies below (eperm/boom fakes) + self.restore(hil_health, '_kill_and_confirm') + + def test_kills_a_detached_child_by_group(self): + """Flashers are spawned with start_new_session=True, so one killpg also reaps + whatever they spawned; a plain kill would leave them holding the probe with no + timeout enforcer left alive.""" + w = FakeProc(101) + self.children = {101: [(900, 900)]} + + class FakePool: + _pool = [w] + self.assertEqual(hil_health.kill_worker_children(FakePool()), 0) # none survived + # by GROUP, so whatever the flasher spawned dies with it + self.assertEqual(self.groups, [(900, signal.SIGKILL)]) + # and then confirmed by pid: killpg reports success when it reached ANY member, + # so the group kill alone is not evidence this one died + self.assertIn((900, 0), self.pids) + self.assertFalse(w.killed) # the WORKER is not this one's job + + def test_a_root_owned_group_is_still_confirmed_and_reported(self): + """killpg on an all-root session raises EPERM: the sudo wrapper died and only its + root members remain. That is the one case this handler exists for, so it must + still reach the confirm step -- otherwise the holder that strands the NEXT job is + the one holder the report never names.""" + w = FakeProc(101) + self.children = {101: [(900, 900)]} + + def eperm(pgid, sig): + raise PermissionError + os.killpg = eperm + + class FakePool: + _pool = [w] + hil_health.kill_worker_children(FakePool()) + # confirmed by pid: a liveness probe on the member killpg could not touch + self.assertIn((900, 0), self.pids) + + def test_kills_a_same_group_child_by_pid(self): + """arecord/iperf/gio go through plain subprocess.run and stay in OUR group, where + killpg would take down the run itself -- but they must still die, or a blocked + arecord keeps holding the wedged device.""" + w = FakeProc(101) + self.children = {101: [(900, self.OWN_PGID)]} + + class FakePool: + _pool = [w] + hil_health.kill_worker_children(FakePool()) + self.assertEqual(self.groups, []) # never our own group + # SIGKILL, then a (pid, 0) probe: signalling is not dying, so the kill is always + # confirmed -- see _kill_and_confirm. + self.assertIn((900, signal.SIGKILL), self.pids) + self.assertIn((900, 0), self.pids) + + + def test_signals_pids_only_when_our_group_is_unknown(self): + """If getpgid(0) fails we cannot tell our group from a detached one, so killpg is + never safe -- fall back to per-pid signals rather than guessing.""" + def boom(pid): + raise OSError('no pgid') + os.getpgid = boom + w = FakeProc(101) + self.children = {101: [(900, 900)]} + + class FakePool: + _pool = [w] + hil_health.kill_worker_children(FakePool()) + self.assertEqual(self.groups, []) + self.assertIn((900, signal.SIGKILL), self.pids) + + def test_covers_a_dead_workers_orphans(self): + """A worker reaped between the snapshot and now leaves its flasher running. The + children are keyed off the snapshot, not off is_alive(), so they still die.""" + w = FakeProc(101, alive=False) + self.children = {101: [(900, 900)]} + + class FakePool: + _pool = [w] + self.assertEqual(hil_health.kill_worker_children(FakePool()), 0) # none survived + self.assertEqual(self.groups, [(900, signal.SIGKILL)]) + + def test_a_survivor_is_returned_so_the_report_can_say_the_rig_is_dirty(self): + """A stray that ignores SIGKILL is in D state on a usbfs node or holds a probe, and + it persists into the NEXT job. The count used to be discarded by the caller (the + return was the signalled-child count, which nothing read), so the only trace was a + line in the log -- and the run still published a table that looks clean.""" + w = FakeProc(101) + # TWO strays, only ONE unkillable: signalled=2, survivors=1, so this cannot pass + # by accident on the old return value + self.children = {101: [(900, 900), (901, 901)]} + self.patch(hil_health, '_kill_and_confirm', lambda pids: [p for p in pids if p == 901]) + + class FakePool: + _pool = [w] + self.assertEqual(hil_health.kill_worker_children(FakePool()), 1) + + def test_no_signal_when_the_workers_spawned_nothing(self): + w = FakeProc(101) # no self.children entry + + class FakePool: + _pool = [w] + self.assertEqual(hil_health.kill_worker_children(FakePool()), 0) + self.assertEqual((self.groups, self.pids), ([], [])) + + def test_includes_the_managers_children(self): + mgr_proc = FakeProc(402) + self.children = {402: [(900, 900)]} + + class FakePool: + _pool = [] + + class FakeManager: + _process = mgr_proc + self.assertEqual(hil_health.kill_worker_children(FakePool(), FakeManager()), 0) + + +class ConfirmTailIsOneGrace(PatchCase): + """The grace is ONE window for the whole set, not one per pid. Paid serially it + scaled with stray count: 30 strays x 16 workers spent ~154s inside the path whose + only job is to free the runner's single job slot -- which is exactly the + 'multi-stray convoy tail is minutes' the CI ceilings budget +30 min for.""" + + def setUp(self): + self.proc_tmp = TemporaryDirectory() + self.addCleanup(self.proc_tmp.cleanup) + root = Path(self.proc_tmp.name) + # every fake pid is alive and NOT a zombie, so all of them outlast the grace + make_proc(root, {900 + i: ('flasher', 'D', b'openocd\x00') for i in range(20)}) + self.patch(hil_health, 'PROC', root) + self.patch(hil_health, 'CONFIRM_KILL_GRACE', 0.3) + self.patch(os, 'kill', lambda pid, sig: None) # never signal a real pid + + def test_twenty_survivors_cost_one_grace_not_twenty(self): + pids = [900 + i for i in range(20)] + t0 = time.monotonic() + still = hil_health._kill_and_confirm(pids) + elapsed = time.monotonic() - t0 + self.assertEqual(sorted(still), pids) # all reported, none lost + self.assertLess(elapsed, 0.3 * 4, + f'the grace is paid per pid ({elapsed:.2f}s for 20)') + + +class KillPoolChildren(PatchCase): + """The worker processes themselves. + + Verified premise: an orphaned pool worker keeps the CI runner's stdout pipe open, so a + reader never sees EOF even after the parent exits. + + Fakes throughout: FakeProc.kill() only sets a flag, so nothing here can signal a real + process. That matters historically -- an earlier revision drove this through + os.pidfd_open with literal pids (101, 102), which exist on a real machine, so the suite + was asking the kernel to signal unrelated system processes and was saved only by EPERM. + Keep the fake in charge of kill(); never let a test reach os.kill/os.killpg with a + live pid. FakeProc.kill() alone is NOT enough for that: it leaves is_alive() True, so + the pid reaches the confirm/sudo ladder, which signals for real. Stub that too.""" + + def setUp(self): + # Pids 101/102/201 are ordinary user processes on a container or a fresh runner -- + # and pre-commit.yml runs this suite on GitHub's. Unstubbed, the ladder ran + # os.kill(101, SIGKILL) and forked `sudo -n kill -9 101` on an account with + # passwordless sudo, and the assertions passed only because those pids happen to + # be unkillable kernel threads here. + self.signals = [] + self.patch(os, 'kill', lambda pid, sig: self.signals.append((pid, sig))) + self.patch(os, 'killpg', lambda pgid, sig: self.signals.append((pgid, sig))) + self.proc_tmp = TemporaryDirectory() + self.addCleanup(self.proc_tmp.cleanup) + self.patch(hil_health, 'PROC', Path(self.proc_tmp.name)) # empty: unreadable -> gone + self.patch(hil_health, 'CONFIRM_KILL_GRACE', 0.05) + + def test_a_healthy_worker_never_reaches_the_signalling_ladder(self): + """The premise every assertion below rests on. Process.kill() is the fake's job; + only a worker that SURVIVES it goes on to raw os.kill/sudo, and these pids are + literals that belong to somebody else.""" + a, b = FakeProc(101), FakeProc(102) + + class FakePool: + _pool = [a, b] + hil_health.kill_pool_children(FakePool()) + self.assertEqual(self.signals, [], 'a literal pid reached the raw-signal ladder') + + def test_signals_every_live_worker(self): + a, b = FakeProc(101), FakeProc(102) + + class FakePool: + _pool = [a, b] + # 0, not 2: the RETURN is confirmed survivors, and workers that die to SIGKILL are + # not survivors. The operator verdict ("power-cycle the host") hangs off this. + self.assertEqual(hil_health.kill_pool_children(FakePool()), 0) + self.assertTrue(a.killed and b.killed) + + def test_a_wedged_worker_is_reported_as_a_survivor(self): + """The number the power-cycle verdict is worded on.""" + make_proc(Path(self.proc_tmp.name), {301: ('python3', 'D', b'python3 hil_test.py\x00')}) + wedged = FakeProc(301, wedged=True) + + class FakePool: + _pool = [wedged] + self.assertEqual(hil_health.kill_pool_children(FakePool()), 1) + + def test_skips_a_reaped_worker(self): + """Process.kill() re-checks returncode internally, but skipping a dead child keeps + the harness from signalling a pid the OS may have recycled.""" + live, dead = FakeProc(201), FakeProc(202, alive=False) + + class FakePool: + _pool = [live, dead] + self.assertEqual(hil_health.kill_pool_children(FakePool()), 0) + self.assertTrue(live.killed) + self.assertFalse(dead.killed) + + def test_also_kills_the_manager(self): + """Manager() is a separate child holding the same descriptors, and os._exit skips + its finalizer, so leaving it behind defeats the whole purpose. The RETURN is the + confirmed-survivor count (the caller words a power-cycle verdict on it), so a + clean kill of both reports 0.""" + worker, mgr_proc = FakeProc(401), FakeProc(402) + + class FakePool: + _pool = [worker] + + class FakeManager: + _process = mgr_proc + self.assertEqual(hil_health.kill_pool_children(FakePool(), FakeManager()), 0) + self.assertTrue(worker.killed and mgr_proc.killed) + self.assertTrue(mgr_proc.killed) + + def test_tolerates_a_pool_without_workers(self): + class NoPool: + _pool = None + self.assertEqual(hil_health.kill_pool_children(NoPool()), 0) + + +class WriteTimeoutReport(unittest.TestCase): + def test_prefix_carries_the_preflight_diagnosis(self): + """The timeout aborts before accumulate_report, so without the prefix the artifact + and the PR comment lose the one line saying WHY the pool never finished.""" + with TemporaryDirectory() as td: + d = Path(td) + hil_health.write_timeout_report(d, [{'name': 'b1'}], 4200, 'r.md', + prefix='> **wedged usb_hub_wq worker.**\n') + out = (d / 'r.md').read_text() + self.assertTrue(out.startswith('> **wedged usb_hub_wq worker.**')) + self.assertIn('timed out after 4200s', out) + self.assertIn('- b1', out) + + def test_writes_a_report_where_there_would_be_none(self): + with TemporaryDirectory() as td: + hil_health.write_timeout_report(Path(td), [{'name': 'ra6m5_ek'}], 4200, + 'hil_report.md') + md = (Path(td) / 'hil_report.md').read_text() + self.assertIn('4200s', md) + self.assertIn('ra6m5_ek', md) + + def test_keeps_a_previous_attempts_table(self): + with TemporaryDirectory() as td: + path = Path(td) / 'hil_report.md' + path.write_text('| board | cdc_msc |\n') + hil_health.write_timeout_report(Path(td), [{'name': 'b1'}], 4200, 'hil_report.md') + md = path.read_text() + self.assertIn('abandoned', md) + self.assertIn('| board | cdc_msc |', md) + self.assertLess(md.index('abandoned'), md.index('| board |')) + + def test_custom_banner_is_used(self): + with TemporaryDirectory() as td: + hil_health.write_timeout_report(Path(td), [], 0, 'hil_report.md', + banner='**refused to start.**\n') + self.assertIn('refused to start', (Path(td) / 'hil_report.md').read_text()) + + def test_unwritable_dir_does_not_raise(self): + """The caller may be about to os._exit; losing the report must not also lose the + exit path.""" + hil_health.write_timeout_report(Path('/proc/nonexistent/nope'), [], 0, 'x.md') + + +class WorkerSweepsItsOwnChildren(unittest.TestCase): + """maxtasksperchild=1 makes a worker exit the moment its task returns, so by the time + main()'s finally sweeps, the strays have been reparented to init and are off the pool's + ppid tree entirely. Measured over 4 tasks: pool._pool held two FRESH workers with zero + overlap with the four that ran, child_procs() returned {}, the sweep reported 0, and all + four strays were alive. Inside the worker the ppid link is still there.""" + + def test_a_detached_child_is_killed_and_confirmed(self): + kid = subprocess.Popen(['sleep', '120'], start_new_session=True) + self.addCleanup(lambda: kid.poll() is None and kid.kill()) + time.sleep(0.3) # let it appear in /proc + + stray = hil_health.kill_own_children() + + self.assertEqual(stray, 0, 'a killable stray was reported as a survivor') + kid.wait(timeout=5) # TimeoutExpired here means it outlived us + self.assertIsNotNone(kid.poll()) + + def test_no_children_is_not_an_error(self): + self.assertEqual(hil_health.kill_own_children(), 0) + + +class PermitReleasesOnlyWhatItTook(unittest.TestCase): + """The bounded acquire skips a slot it could not get ('proceeding over-subscribed') and + deliberately leaves it out of `taken`, but __exit__ released every slot in self.slots. + multiprocessing.Semaphore is unbounded, so each timeout permanently widened that + controller's permit -- the throttle this branch NARROWED (FLASH_PARALLEL 8->4, + USBTEST_PARALLEL 4->2) for xHCI bandwidth margin.""" + + def test_a_timed_out_slot_is_not_released_on_exit(self): + from helper import hil_lock + import multiprocessing + + sems = [multiprocessing.Semaphore(1)] + sems[0].acquire() # width 1, already held: the next wait times out + self.addCleanup(setattr, hil_lock, 'PERMIT_TIMEOUT', hil_lock.PERMIT_TIMEOUT) + hil_lock.PERMIT_TIMEOUT = 0.1 + + permit = hil_lock.controller_permit(sems, 'UID') + permit.slots = [0] + with permit: + pass + + # one holder still holds it, so a correct exit leaves it unavailable + self.assertFalse(sems[0].acquire(timeout=0.1), + 'the permit released a slot it never acquired: width grew') + + +class RecoveryPrefersResetOverReflash(unittest.TestCase): + """Probe reset is the preferred cure: non-destructive (the wedged firmware survives for + autopsy), no flash wear, no risk of a bad park image (a wfe/wfi park has bricked SWD on + mimxrt1064_evk and max32666fthr through a power cycle), and measured at 128-129 ms + against a full erase+program. It also fits in budgets a reflash does not.""" + + def setUp(self): + import usbtest # test/hil is already on sys.path (see top of file) + self.u = usbtest + + def test_reset_is_attempted_before_the_reflash(self): + steps = self.u.recovery_steps('openocd', time_left=600) + self.assertEqual([s[0] for s in steps], ['reset', 'flash']) + + def test_a_budget_too_small_to_reflash_still_gets_the_reset(self): + """The old gate skipped recovery whole when a reflash did not fit, leaving the + holder in place; a reset needs a fraction of the budget.""" + steps = self.u.recovery_steps('openocd', time_left=self.u.RECOVER_FLASH_TIMEOUT - 1) + self.assertEqual([s[0] for s in steps], ['reset']) + + def test_no_budget_at_all_yields_nothing(self): + self.assertEqual(self.u.recovery_steps('openocd', time_left=1), []) + + def test_a_flasher_with_no_reset_primitive_goes_straight_to_reflash(self): + steps = self.u.recovery_steps('nosuchflasher', time_left=600) + self.assertEqual([s[0] for s in steps], ['flash']) + + +class RecoveryDoesNotClaimAResetItDidNotDo(unittest.TestCase): + """reset_esptool and reset_lm4flash return rc 0 without resetting anything, so a plan + that includes them makes the log say "resetting <board> via <flasher>" for a step that + did nothing. wedged_pids() arbitrates, so behaviour was already right -- the record was + not, and a false record is what this branch keeps having to unpick.""" + + def setUp(self): + import usbtest + self.u = usbtest + + def test_a_no_op_reset_primitive_is_not_scheduled(self): + self.assertEqual([k for k, _ in self.u.recovery_steps('esptool', 600)], ['flash']) + self.assertEqual([k for k, _ in self.u.recovery_steps('lm4flash', 600)], ['flash']) + + def test_a_real_reset_primitive_still_is(self): + self.assertEqual([k for k, _ in self.u.recovery_steps('openocd', 600)], + ['reset', 'flash']) + + +class SudoSoftNeverRaises(unittest.TestCase): + """Two of its four call sites are inside run_case's timeout handler, where ANY raise + costs the HUNG verdict, the recovery and the JSON report -- and sudo() sys.exit()s on + 'a password is required', which is a raise like any other.""" + + def setUp(self): + import usbtest + self.u = usbtest + self.addCleanup(setattr, usbtest, 'sudo', usbtest.sudo) + + def _check(self, exc): + def boom(*a, **k): + raise exc + self.u.sudo = boom + r = self.u._sudo_soft(['dmesg']) # must not propagate + self.assertEqual(r.returncode, 1) + + def test_systemexit_from_a_password_prompt_is_contained(self): + self._check(SystemExit('sudo needs a password')) + + def test_oserror_is_contained(self): + self._check(OSError('no such binary')) + + def test_subprocess_error_is_contained(self): + self._check(subprocess.SubprocessError('timed out')) + + +if __name__ == '__main__': + unittest.main(verbosity=1) diff --git a/test/hil/test/test_hil_select.py b/test/hil/test/test_hil_select.py new file mode 100644 index 000000000..9a1261878 --- /dev/null +++ b/test/hil/test/test_hil_select.py @@ -0,0 +1,689 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +# Unit tests for hil_select.py — pure logic, no hardware, no git. Run directly: +# python3 test/hil/test/test_hil_select.py +# +# Imports stay stdlib + hil_select/hil_util/hil_flash ONLY: the pre-commit hil-test +# hook runs this suite, on GitHub's bare runner in the pre-commit workflow as well as +# locally, and that runner has no pyserial/pymtp. hil_flash is admissible because it +# is stdlib + hil_util only (test_hil_util.BottomLayer enforces the stdlib closure of +# both) and the roster-dispatch tests need its flash_* table; never import hil_test, +# which pulls pyserial. +import glob +import json +import os +import sys +import unittest + +# the modules under test live in the parent dir (test/hil), not here +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) +import hil_flash +from helper import hil_select +from helper.hil_util import device_tests, dual_tests + +REPO = os.path.dirname(os.path.dirname(os.path.dirname( + os.path.dirname(os.path.abspath(__file__))))) + + +def real_rosters(): + """The actual rig rosters, for regression tests that need real-world data + (a specific board/family/only-list) rather than the synthetic ROSTER above.""" + rosters = [] + for name in ('tinyusb.json', 'hfp.json'): + path = os.path.join(REPO, 'test/hil', name) + with open(path) as f: + rosters.append((f'test/hil/{name}', json.load(f)['boards'])) + return rosters + + +def roster_flashers(): + """(roster path, board) for every board in the live rosters, `boards-skip` + included: a parked board's flasher name must still dispatch, so that unparking it + is not what discovers the name went stale.""" + for name in ('tinyusb.json', 'hfp.json'): + path = os.path.join(REPO, 'test/hil', name) + with open(path) as f: + cfg = json.load(f) + for key in ('boards', 'boards-skip'): + for b in cfg.get(key, []): + yield f'test/hil/{name}', b + + +def on_roster(tc, *names): + """The subset of `names` currently in the live rig rosters, skipping the test + when none are, because parking/unparking a board is routine rig maintenance. + + That skip now matters MORE than it used to, not less: this suite is a blocking + pre-commit hook AND build.yml's selector steps gate on it (a failing suite falls + open to the full matrix), so an assertion that depends on a specific board being + present goes red on every PR -- including src/-only ones that never touched the + rig -- until someone fixes the roster. Keep roster-dependent assertions behind + on_roster.""" + have = {b['name'] for _, boards in real_rosters() for b in boards} + got = [n for n in names if n in have] + if not got: + tc.skipTest(f'not in the rig roster: {", ".join(names)}') + return got + + +ROSTER = [ + # device-only, rp2040 family + {'name': 'raspberry_pi_pico', 'uid': 'u1', 'flasher': {'name': 'openocd'}, + 'tests': {'device': True, 'host': True, 'dual': True}}, + # device-only, stm32f4 family + {'name': 'stm32f407disco', 'uid': 'u2', 'flasher': {'name': 'jlink'}, + 'tests': {'device': True, 'host': False, 'dual': False}}, + # host-only board + {'name': 'raspberry_pi_pico2', 'uid': 'u3', 'flasher': {'name': 'openocd'}, + 'tests': {'device': False, 'host': True, 'dual': False}}, + # only-list board (espressif-style), flashed by the CI leg that splits on esptool + {'name': 'espressif_s3_devkitm', 'uid': 'u4', 'flasher': {'name': 'esptool'}, + 'tests': {'only': ['device/cdc_msc_freertos', 'host/device_info']}}, +] +ROSTERS = [('test/hil/tinyusb.json', ROSTER)] + + +def sel(files): + return hil_select.classify(files, REPO, ROSTERS) + + +class TestPortRule(unittest.TestCase): + def test_dcd_rp2040_selects_pico_family_only(self): + s = sel(['src/portable/raspberrypi/rp2040/dcd_rp2040.c']) + self.assertFalse(s['full']) + self.assertIn('raspberry_pi_pico', s['boards']) + self.assertNotIn('stm32f407disco', s['boards']) + self.assertNotIn('espressif_s3_devkitm', s['boards']) + # device role: no host tests in pico's list + self.assertTrue(all(not t.startswith('host/') for t in s['boards']['raspberry_pi_pico'])) + # host-only boards drop out entirely on a device-role change + self.assertNotIn('raspberry_pi_pico2', s['boards']) + + def test_shared_port_file_is_both_roles(self): + s = sel(['src/portable/synopsys/dwc2/dwc2_common.c']) + self.assertFalse(s['full']) + self.assertNotIn('raspberry_pi_pico', s['boards']) # rp2040 is not a dwc2 family + self.assertIn('stm32f407disco', s['boards']) # stm32f4 is + + +class TestCoreRoleRule(unittest.TestCase): + def test_usbd_selects_all_device_tests_everywhere(self): + s = sel(['src/device/usbd.c']) + self.assertFalse(s['full']) + self.assertNotIn('raspberry_pi_pico2', s['boards']) # host-only board dropped + pico = s['boards']['raspberry_pi_pico'] + self.assertTrue(set(device_tests).issubset(set(pico))) + self.assertTrue(set(dual_tests).issubset(set(pico))) # dual survives device role + self.assertTrue(all(not t.startswith('host/') for t in pico)) + # only-list board: selection intersects its only-list + esp = s['boards']['espressif_s3_devkitm'] + self.assertEqual(esp, ['device/cdc_msc_freertos']) + + def test_host_change_drops_device(self): + s = sel(['src/host/usbh.c']) + self.assertFalse(s['full']) + self.assertIn('raspberry_pi_pico2', s['boards']) + self.assertNotIn('stm32f407disco', s['boards']) # device-only board dropped + + +class TestClassRule(unittest.TestCase): + def test_cdc_device_selects_cdc_examples_only(self): + s = sel(['src/class/cdc/cdc_device.c']) + self.assertFalse(s['full']) + pico = s['boards']['raspberry_pi_pico'] + self.assertIn('device/cdc_msc', pico) + self.assertIn('device/cdc_dual_ports', pico) + self.assertNotIn('device/msc_dual_lun', pico) # CFG_TUD_CDC 0 there + self.assertNotIn('device/usbtest', pico) # CFG_TUD_CDC 0 there + self.assertTrue(all(not t.startswith('host/') for t in pico)) + + def test_msc_host_selects_host_side(self): + s = sel(['src/class/msc/msc_host.c']) + self.assertFalse(s['full']) + self.assertNotIn('stm32f407disco', s['boards']) # device-only board + pico2 = s['boards']['raspberry_pi_pico2'] + self.assertIn('host/msc_file_explorer', pico2) + self.assertTrue(all(not t.startswith('device/') for t in pico2)) + + +class TestClassIncludeEdges(unittest.TestCase): + """A class header another class includes reaches that class's examples too. + src/class/midi/midi{,2}_{device,host}.h include class/audio/audio.h, so + midi_test's firmware contains audio.h - but the class rule derives macros from + the directory name alone, so an audio.h change used to select only + device/audio_test_freertos. On boards that skip that example the per-board + intersection emptied and an audio.h-only PR ran ZERO HIL on them.""" + def test_edges_derived_from_includes(self): + edges = hil_select.class_include_edges(REPO) + self.assertEqual(edges.get('audio/audio.h'), {'midi'}) + self.assertEqual(edges.get('cdc/cdc.h'), {'net'}) + + def test_audio_header_selects_midi_example(self): + s = hil_select.classify(['src/class/audio/audio.h'], REPO, real_rosters()) + self.assertFalse(s['full']) + # every board that runs device/midi_test at all must run it here (boards with + # a tests.only list, e.g. espressif, run the freertos examples instead) + by_name = {b['name']: b for _, bs in real_rosters() for b in bs} + checked = 0 + for name, tests in s['boards'].items(): + if 'device/midi_test' in hil_select.board_tests(by_name[name]): + self.assertIn('device/midi_test', tests, name) + checked += 1 + self.assertTrue(checked) + + def test_audio_header_reaches_boards_that_skip_audio(self): + # both skip device/audio_test_freertos: without the midi edge their + # intersection is empty and they drop out of the selection entirely + boards = on_roster(self, 'metro_m4_express', 'nrf54lm20dk') + s = hil_select.classify(['src/class/audio/audio.h'], REPO, real_rosters()) + for board in boards: + self.assertEqual(s['boards'].get(board), ['device/midi_test'], board) + + def test_edge_is_per_header_not_per_class(self): + # midi includes audio.h, not audio_device.h: an audio_device change must + # not drag midi's examples in + s = hil_select.classify(['src/class/audio/audio_device.c'], REPO, real_rosters()) + self.assertFalse(s['full']) + for tests in s['boards'].values(): + if tests != 'all': + self.assertNotIn('device/midi_test', tests) + + +class TestFallbackRules(unittest.TestCase): + def test_unknown_tool_is_full(self): + s = sel(['tools/random_new_script.py']) + self.assertTrue(s['full']) + + def test_docs_only_is_empty_not_full(self): + s = sel(['docs/info/contributing.rst', 'README.rst']) + self.assertFalse(s['full']) + self.assertEqual(s['boards'], {}) + + def test_bsp_family_selects_family_boards(self): + s = sel(['hw/bsp/rp2040/family.cmake']) + self.assertFalse(s['full']) + self.assertIn('raspberry_pi_pico', s['boards']) + self.assertEqual(s['boards']['raspberry_pi_pico'], 'all') + self.assertNotIn('stm32f407disco', s['boards']) + + def test_bsp_board_narrows_to_board(self): + s = sel(['hw/bsp/rp2040/boards/raspberry_pi_pico/board.h']) + self.assertFalse(s['full']) + self.assertEqual(list(s['boards'].keys()), ['raspberry_pi_pico']) + + def test_example_change_selects_that_example(self): + s = sel(['examples/device/cdc_msc/src/main.c']) + self.assertFalse(s['full']) + self.assertEqual(s['boards']['raspberry_pi_pico'], ['device/cdc_msc']) + + def test_core_common_is_full(self): + for f in ['src/tusb.c', 'src/common/tusb_fifo.c', 'src/osal/osal_freertos.h']: + self.assertTrue(sel([f])['full'], f) + + def test_board_test_example_is_full(self): + # board_test is the park/teardown firmware hil_test.py flashes on every board, + # not an unlisted example: a regression there must not skip the whole rig + for f in ['examples/device/board_test/src/main.c', + 'examples/device/board_test/CMakeLists.txt']: + self.assertTrue(sel([f])['full'], f) + + def test_harness_is_full(self): + for f in ['test/hil/hil_test.py', '.github/workflows/build.yml', 'hw/mcu/nxp/x.c', 'lib/foo/x.c']: + self.assertTrue(sel([f])['full'], f) + + def test_mixed_roles_no_pruning(self): + s = sel(['src/device/usbd.c', 'src/host/usbh.c']) + self.assertFalse(s['full']) + self.assertIn('raspberry_pi_pico2', s['boards']) + self.assertIn('stm32f407disco', s['boards']) + + def test_cmakelists_and_requirements_are_full(self): + for f in ['src/CMakeLists.txt', 'examples/CMakeLists.txt', + 'examples/device/CMakeLists.txt', 'test/hil/requirements.txt']: + self.assertTrue(sel([f])['full'], f) + + def test_docs_txt_is_noncode(self): + s = sel(['docs/info/changelog.txt']) + self.assertFalse(s['full']) + self.assertEqual(s['boards'], {}) + + +class TestArgsEmission(unittest.TestCase): + def test_args_for_scoped_selection(self): + s = sel(['src/portable/raspberrypi/rp2040/dcd_rp2040.c']) + args = hil_select.selection_args(s, ROSTERS) + a = args['tinyusb.json'] + self.assertIn('-b raspberry_pi_pico', a) + self.assertNotIn('stm32f407disco', a) + self.assertIn('-bt raspberry_pi_pico:', a) # device-only subset of a device+host board + + def test_args_full_is_empty(self): + s = sel(['tools/random_new_script.py']) + self.assertEqual(hil_select.selection_args(s, ROSTERS), {'tinyusb.json': ''}) + + def test_args_all_board_gets_bare_b(self): + s = sel(['hw/bsp/rp2040/boards/raspberry_pi_pico/board.h']) + a = hil_select.selection_args(s, ROSTERS)['tinyusb.json'] + self.assertIn('-b raspberry_pi_pico', a) + self.assertNotIn('-bt', a) + + def test_args_by_flasher_splits_esp_from_the_rest(self): + s = sel(['src/device/usbd.c']) + per = hil_select.selection_args_by_flasher(s, ROSTERS)['tinyusb.json'] + self.assertIn('espressif_s3_devkitm', per['esptool']) + self.assertIn('raspberry_pi_pico', per['openocd']) + self.assertNotIn('espressif_s3_devkitm', per.get('openocd', '') + per.get('jlink', '')) + + def test_args_by_flasher_omits_a_flasher_with_no_selected_board(self): + # the esp CI leg must see no args at all here, not a filter matching zero boards + s = sel(['hw/bsp/rp2040/boards/raspberry_pi_pico/board.h']) + per = hil_select.selection_args_by_flasher(s, ROSTERS)['tinyusb.json'] + self.assertEqual(per, {'openocd': '-b raspberry_pi_pico'}) + + def test_args_by_flasher_full_is_empty(self): + s = sel(['tools/random_new_script.py']) + self.assertEqual(hil_select.selection_args_by_flasher(s, ROSTERS), {'tinyusb.json': {}}) + + def test_cli_diff_file(self): + import subprocess, tempfile, json as j + with tempfile.NamedTemporaryFile('w', suffix='.txt', delete=False) as f: + f.write('src/class/cdc/cdc_device.c\n') + path = f.name + r = subprocess.run([sys.executable, os.path.join(REPO, 'test/hil/helper/hil_select.py'), + '--diff-file', path, os.path.join(REPO, 'test/hil/tinyusb.json')], + capture_output=True, text=True) + self.assertEqual(r.returncode, 0, r.stderr) + out = j.loads(r.stdout) + self.assertFalse(out['full']) + self.assertIn('tinyusb.json', out['args']) + self.assertTrue(any('cdc_device' in line for line in out['reasons'])) + # A core-class diff must select boards THROUGH THE CLI: the in-process tests + # inject their own repo root, so only this subprocess path catches a broken + # repo_root derivation -- which once made every repo-relative glob match + # nothing and turned this exact diff into a silent full-HIL skip. + self.assertTrue(out['boards'], + 'CLI selected zero boards for a src/class change: repo_root broken?') + os.unlink(path) + + +class TestRealRosterPortFamilies(unittest.TestCase): + """Regression for port_families() missing espressif's dwc2 reference, which + lives in a component CMakeLists.txt rather than family.cmake/family.mk.""" + def test_dwc2_change_selects_espressif_boards(self): + boards = on_roster(self, 'espressif_s3_devkitm', 'espressif_p4_function_ev') + s = hil_select.classify(['src/portable/synopsys/dwc2/dcd_dwc2.c'], REPO, real_rosters()) + self.assertFalse(s['full']) + for board in boards: + self.assertIn(board, s['boards']) + + +class TestOptionGatedPort(unittest.TestCase): + """Regression: family_support.cmake compiles some ports from a build option + (MAX3421_HOST=1 -> hcd_max3421.c), so a board's family file never names them.""" + # host-side option board (max3421 as host controller), off any max3421 family + OPT_ROSTER = [('test/hil/opt.json', [ + {'name': 'fake_dual_board', 'uid': 'o1', 'flasher': {'name': 'jlink'}, + 'build': {'args': ['MAX3421_HOST=1']}, + 'tests': {'device': True, 'host': False, 'dual': True}}, + {'name': 'fake_host_board', 'uid': 'o2', 'flasher': {'name': 'jlink'}, + 'variant': [{'name': 'fake_host_board', 'flags': '-DMAX3421_HOST=1'}], + 'tests': {'device': False, 'host': True, 'dual': False}}, + {'name': 'fake_off_board', 'uid': 'o3', 'flasher': {'name': 'jlink'}, + 'variant': [{'name': 'fake_off_board', 'defines': ['MAX3421_HOST=0']}], + 'tests': {'device': True, 'host': True, 'dual': True}}, + ])] + + def test_real_roster_max3421_selects_option_board(self): + boards = on_roster(self, 'metro_m4_express') + s = hil_select.classify(['src/portable/analog/max3421/hcd_max3421.c'], REPO, real_rosters()) + self.assertFalse(s['full']) + for board in boards: + self.assertIn(board, s['boards']) + + def test_option_selects_via_args_defines_and_flags(self): + s = hil_select.classify(['src/portable/analog/max3421/hcd_max3421.c'], REPO, self.OPT_ROSTER) + self.assertFalse(s['full']) + self.assertIn('fake_dual_board', s['boards']) # build.args + self.assertIn('fake_host_board', s['boards']) # variant flags + self.assertNotIn('fake_off_board', s['boards']) # variant defines, but =0 + + def test_device_role_port_does_not_pull_host_only_option_board(self): + s = hil_select.classify(['src/portable/analog/max3421/dcd_max3421.c'], REPO, self.OPT_ROSTER) + self.assertFalse(s['full']) + self.assertNotIn('fake_host_board', s['boards']) # host-only board, device change + self.assertIn('fake_dual_board', s['boards']) # device-capable option board + + def test_gates_parsed_from_family_support(self): + self.assertEqual(hil_select.port_option_gates(REPO).get('analog/max3421'), + {'MAX3421_HOST'}) + + def test_board_cmake_option_counts(self): + """A board can enable a gated port in its own BSP rather than via the roster + (hw/bsp/espressif/boards/*/board.cmake -> set(MAX3421_HOST 1)); board_options() + must see those too, or such a board joining the roster is silently dropped.""" + self.assertIn('MAX3421_HOST', + hil_select.bsp_board_options('adafruit_feather_esp32s3', REPO)) + self.assertIn('CFG_TUH_RPI_PIO_USB', + hil_select.bsp_board_options('adafruit_fruit_jam', REPO)) + # commented-out `# set(MAX3421_HOST 1)` must not count + self.assertNotIn('MAX3421_HOST', + hil_select.bsp_board_options('feather_nrf52840_express', REPO)) + + def test_board_cmake_option_selects_off_family_board(self): + # adafruit_feather_esp32s3 is not on any rig roster; stand it in as one to + # prove the BSP-sourced option alone pulls a max3421 change onto the board + roster = [('test/hil/opt.json', [ + {'name': 'adafruit_feather_esp32s3', 'uid': 'o1', 'flasher': {'name': 'esptool'}, + 'tests': {'device': False, 'host': True, 'dual': False}}])] + s = hil_select.classify(['src/portable/analog/max3421/hcd_max3421.c'], REPO, roster) + self.assertFalse(s['full']) + self.assertIn('adafruit_feather_esp32s3', s['boards']) + + def test_board_mk_option_is_ignored(self): + """Make-only options must not select: HIL CI builds with CMake exclusively, so + hw/bsp/nrf/boards/nrf5340dk/board.mk's MAX3421_HOST compiles nothing here.""" + roster = [('test/hil/opt.json', [ + {'name': 'nrf5340dk', 'uid': 'o1', 'flasher': {'name': 'jlink'}, + 'tests': {'device': False, 'host': True, 'dual': False}}])] + s = hil_select.classify(['src/portable/analog/max3421/hcd_max3421.c'], REPO, roster) + self.assertFalse(s['full']) + self.assertEqual(s['boards'], {}) + + +class TestPortFamiliesCmakeOnly(unittest.TestCase): + """port_families() is CMake-only (HIL CI never builds with Make) and matches on + 'port_dir/' so a port dir is not a prefix of a sibling.""" + def test_make_only_family_is_not_a_family(self): + # hw/bsp/pic32mz has family.mk but no family.cmake + self.assertEqual(hil_select.port_families('microchip/pic32mz', REPO), set()) + + def test_prefix_port_does_not_inherit_sibling_families(self): + # bare-substring matching let 'microchip/pic' match '.../microchip/pic32mz/...' + self.assertEqual(hil_select.port_families('microchip/pic', REPO), set()) + + def test_make_only_port_forces_full(self): + s = sel(['src/portable/microchip/pic32mz/dcd_pic32mz.c']) + self.assertTrue(s['full']) + self.assertTrue(any('no board family' in r for r in s['reasons']), s['reasons']) + + def test_cmake_families_still_found(self): + self.assertEqual(hil_select.port_families('raspberrypi/rp2040', REPO), {'rp2040'}) + self.assertIn('stm32f4', hil_select.port_families('synopsys/dwc2', REPO)) + + +class TestPortFamiliesCoverage(unittest.TestCase): + """Systematic guard: every real dcd_*/hcd_* port directory should map to at + least one board family, so a future family.cmake/CMakeLists.txt layout that + port_families() doesn't scan fails loudly instead of silently dropping boards + (as espressif's dwc2 reference did - see TestRealRosterPortFamilies).""" + # Ports with no board family: not a bug, just not wired into any rig board. + # Add here (with a reason) only if port_families() legitimately can't find one. + # A port listed here force-fulls (fail-open), so it is never under-selected. + NO_FAMILY = { + 'template', # reference/example port, not built by any board + # hw/bsp/pic32mz has family.mk only (no family.cmake), and port_families() + # is CMake-only because HIL CI builds every board with CMake - so this port + # is compiled for no HIL board. + 'microchip/pic32mz', + 'microchip/pic', # same: only ever referenced from pic32mz's family.mk + } + + @staticmethod + def _dcd_hcd_ports(): + portable_root = os.path.join(REPO, 'src/portable') + ports = [] + for entry in sorted(os.listdir(portable_root)): + d = os.path.join(portable_root, entry) + if not os.path.isdir(d): + continue + if glob.glob(os.path.join(d, 'dcd_*.c')) or glob.glob(os.path.join(d, 'hcd_*.c')): + ports.append(entry) + continue + for sub in sorted(os.listdir(d)): + sd = os.path.join(d, sub) + if os.path.isdir(sd) and (glob.glob(os.path.join(sd, 'dcd_*.c')) or + glob.glob(os.path.join(sd, 'hcd_*.c'))): + ports.append(f'{entry}/{sub}') + return ports + + def test_every_port_maps_to_a_family(self): + ports = self._dcd_hcd_ports() + self.assertTrue(ports) # sanity: the scan itself found something + for port in ports: + if port in self.NO_FAMILY: + continue + fams = hil_select.port_families(port, REPO) + self.assertTrue(fams, f'{port}: no family references this port ' + f'(port_families() scan gap, or add to NO_FAMILY)') + + +class TestRealRosterOnlyListTests(unittest.TestCase): + """Regression for roster-only-list tests (e.g. espressif's hid_composite_freertos) + being invisible to the selector because it only knew the shared hil_util lists.""" + def test_only_list_example_change_selects_it(self): + boards = on_roster(self, 'espressif_s3_devkitm', 'espressif_p4_function_ev') + s = hil_select.classify(['examples/device/hid_composite_freertos/src/main.c'], REPO, real_rosters()) + self.assertFalse(s['full']) + for board in boards: + self.assertEqual(s['boards'][board], ['device/hid_composite_freertos']) + + def test_class_change_includes_only_list_boards(self): + boards = on_roster(self, 'espressif_s3_devkitm', 'espressif_p4_function_ev') + s = hil_select.classify(['src/class/hid/hid_device.c'], REPO, real_rosters()) + self.assertFalse(s['full']) + for board in boards: + self.assertIn(board, s['boards']) + + +class TestPortAndCoreRoleUseExtras(unittest.TestCase): + """Regression: the port rule and core-role rule must thread the roster-only + test universe (extras) the same way the class rule already does, so a DCD + or device-stack change doesn't silently drop espressif's only-list tests + (e.g. hid_composite_freertos) that aren't in the shared device_tests list.""" + def test_dcd_change_includes_only_list_test(self): + boards = on_roster(self, 'espressif_s3_devkitm', 'espressif_p4_function_ev') + s = hil_select.classify(['src/portable/synopsys/dwc2/dcd_dwc2.c'], REPO, real_rosters()) + self.assertFalse(s['full']) + for board in boards: + tests = s['boards'][board] + self.assertIn('device/hid_composite_freertos', tests) + self.assertIn('device/cdc_msc_freertos', tests) + self.assertIn('device/audio_test_freertos', tests) + self.assertIn('device/usbtest', tests) + + def test_core_device_change_includes_only_list_test(self): + boards = on_roster(self, 'espressif_s3_devkitm', 'espressif_p4_function_ev') + s = hil_select.classify(['src/device/usbd.c'], REPO, real_rosters()) + self.assertFalse(s['full']) + for board in boards: + tests = s['boards'][board] + self.assertIn('device/hid_composite_freertos', tests) + self.assertIn('device/cdc_msc_freertos', tests) + self.assertIn('device/audio_test_freertos', tests) + self.assertIn('device/usbtest', tests) + + def test_host_change_does_not_leak_device_only_list_test(self): + s = hil_select.classify(['src/host/usbh.c'], REPO, real_rosters()) + self.assertFalse(s['full']) + for board, tests in s['boards'].items(): + if tests == 'all': + continue + self.assertNotIn('device/hid_composite_freertos', tests, board) + + +class TestFamilies(unittest.TestCase): + """`families` exists for consumers that build (not just test) the diff: most + families have no rig board, so `boards` alone would compile nothing for them.""" + def test_off_rig_port_still_reports_family(self): + s = sel(['src/portable/microchip/samx7x/dcd_samx7x.c']) + self.assertFalse(s['full']) + self.assertEqual(s['boards'], {}) # no same7x board on the rig + self.assertEqual(s['families'], ['same7x']) + + def test_port_families_are_reported(self): + s = sel(['src/portable/raspberrypi/rp2040/dcd_rp2040.c']) + self.assertIn('rp2040', s['families']) + + def test_bsp_family_and_board_report_family(self): + self.assertEqual(sel(['hw/bsp/rp2040/family.cmake'])['families'], ['rp2040']) + self.assertEqual(sel(['hw/bsp/rp2040/boards/raspberry_pi_pico/board.h'])['families'], + ['rp2040']) + + def test_docs_only_has_no_families(self): + self.assertEqual(sel(['docs/info/contributing.rst'])['families'], []) + + def test_full_selection_still_reports_families(self): + """A full-matrix file must not hide the families of the other changed files: + consumers that build from `families` (e.g. /pre-pr) ignore `boards` when full.""" + s = sel(['src/common/tusb_fifo.c', 'src/portable/microchip/samx7x/dcd_samx7x.c']) + self.assertTrue(s['full']) + self.assertIn('same7x', s['families']) + # full stays full: every roster board, and no args to narrow the run + self.assertEqual(set(s['boards']), {b['name'] for b in ROSTER}) + self.assertTrue(all(v == 'all' for v in s['boards'].values())) + self.assertEqual(hil_select.selection_args(s, ROSTERS), {'tinyusb.json': ''}) + self.assertEqual(hil_select.selection_args_by_flasher(s, ROSTERS), {'tinyusb.json': {}}) + + def test_family_order_does_not_matter(self): + # same as above with the full-matrix file last (was the only order that worked) + s = sel(['src/portable/microchip/samx7x/dcd_samx7x.c', 'src/common/tusb_fifo.c']) + self.assertTrue(s['full']) + self.assertIn('same7x', s['families']) + + +class TestGitDiffArgv(unittest.TestCase): + def test_diff_disables_rename_detection(self): + """Without --no-renames git reports only a rename's destination, so moving an + HIL-relevant file to a non-code path would be classified as non-code only.""" + self.assertIn('--no-renames', hil_select.GIT_DIFF_ARGV) + + +class TestPortWithoutFamilyIsFull(unittest.TestCase): + """A port dir no family file references must widen (full matrix), not silently + contribute zero boards — the fail-open contract.""" + def test_unreferenced_port_forces_full(self): + orig = hil_select.port_families + hil_select.port_families = lambda port_dir, repo_root: set() + try: + s = sel(['src/portable/vendor/newip/dcd_newip.c']) + finally: + hil_select.port_families = orig + self.assertTrue(s['full']) + self.assertTrue(any('no board family' in r for r in s['reasons']), s['reasons']) + + +class TestOpenocdVidPid(unittest.TestCase): + """The roster's optional flasher `vid_pid` field (openocd-verbatim, e.g. + "0x1a86 0x8010", more pairs appended) pins openocd's probe discovery so it + never opens foreign usbfs nodes. It must be emitted BEFORE the args: the + rescue cfgs run `init` internally (rp2350-rescue.cfg errors on any + config-stage command after its init; rp2040.cfg under RESCUE scans before a + trailing flag is even parsed), and no rig cfg sets a competing list + (the 2026-08-10 convoy mechanism).""" + + def test_vid_pid_flag_precedes_args(self): + cmd = hil_flash._openocd_cmd_base( + {'uid': 'S1', 'args': '-f target/wch-riscv.cfg', 'vid_pid': '0x1a86 0x8010'}) + self.assertIn('-c "adapter usb vid_pid 0x1a86 0x8010" -f target/wch-riscv.cfg', cmd) + self.assertTrue(cmd.endswith('-f target/wch-riscv.cfg'), cmd) + + def test_rescue_cfg_command_keeps_vid_pid_before_init(self): + """rescue_openocd swaps the target cfg for one that runs `init` internally; + a vid_pid flag after the args would error there (rp2350) or be skipped + (rp2040) -- in exactly the wedged-rig scenario the pin exists for.""" + flasher = {'name': 'openocd', 'uid': 'S1', 'vid_pid': '0x2e8a 0x000c', + 'args': '-c "set RESCUE 1" -f target/rp2040.cfg'} + cmd = hil_flash._openocd_cmd_base(flasher) + self.assertLess(cmd.index('adapter usb vid_pid'), cmd.index('-f target/'), cmd) + + def test_vid_pid_multiple_pairs(self): + cmd = hil_flash._openocd_cmd_base( + {'uid': 'S1', 'args': '-f i.cfg', 'vid_pid': '0x2e8a 0x000c 0x2e8a 0x000d'}) + self.assertIn('-c "adapter usb vid_pid 0x2e8a 0x000c 0x2e8a 0x000d"', cmd) + + def test_no_field_no_flag_but_warns(self): + # the roster lint only covers the committed rosters; a dev PC's local.json entry + # without the field must at least say what it is giving up -- on STDERR, since + # hil_test captures stdout per test and would swallow it on a passing run + import io + from contextlib import redirect_stderr + hil_flash._VID_PID_WARNED.discard('S-warn') + cap = io.StringIO() + with redirect_stderr(cap): + cmd = hil_flash._openocd_cmd_base({'uid': 'S-warn', 'args': '-f i.cfg'}) + self.assertNotIn('vid_pid', cmd) + self.assertIn('vid_pid', cap.getvalue()) + + def test_roster_openocd_entries_all_pin_vid_pid(self): + # every openocd probe on the rig has a known VID/PID; a new entry without the + # pin silently reintroduces open-everything discovery + for path, board in roster_flashers(): + f = board['flasher'] + # tinyusb.json only: hfp.json is the hifiphile rig owner's file, and a + # blocking repo-wide lint over someone else's roster would red every PR the + # moment they add an openocd board (hil_flash treats the field as optional) + if f['name'] == 'openocd' and path.endswith('tinyusb.json'): + self.assertIn('vid_pid', f, + f"{path}: {board['name']} openocd flasher lacks vid_pid") + self.assertNotIn('vid_pid', f.get('args', ''), + f"{path}: {board['name']} packs vid_pid into args; use the field") + + +class TestRosterFlashersDispatch(unittest.TestCase): + """hil_test and hil_pool_check resolve a board's flasher with a bare + getattr(hil_flash, f'flash_{name}'), and hil_test does it inside a redirect_stdout — + so a renamed or typo'd roster name raises an AttributeError whose output is swallowed, + with nothing pointing at the roster as the thing to edit. Renaming a flash_*/reset_* + pair without updating every roster must fail here instead.""" + + def test_flash_and_reset_exist_for_every_roster_flasher(self): + for path, board in roster_flashers(): + name = board['flasher']['name'].lower() + for fn in (f'flash_{name}', f'reset_{name}'): + self.assertTrue(callable(getattr(hil_flash, fn, None)), + f'{path}: {board["name"]} uses flasher "{name}" ' + f'but hil_flash.{fn} does not exist') + + def test_firmware_suffix_known_for_every_roster_flasher(self): + """find_firmware falls back to accepting .elf-or-.bin when a flasher is missing + from FLASHER_SUFFIX, silently restoring the mismatch that map exists to catch.""" + for path, board in roster_flashers(): + name = board['flasher']['name'].lower() + self.assertIn(name, hil_flash.FLASHER_SUFFIX, + f'{path}: {board["name"]} uses flasher "{name}" ' + f'with no hil_flash.FLASHER_SUFFIX entry') + + +class FlasherRecoverEntry(unittest.TestCase): + """Optional roster key: a SECOND flasher used only to deliver recovery while a usbfs + node is poisoned. Boards whose primary flasher cannot get past a convoy (jlink, + stlink, lm4flash) name an openocd entry here instead of changing how they are + normally flashed.""" + + def test_recover_flasher_prefers_the_optional_entry(self): + prim = {'name': 'jlink', 'uid': 'X', 'args': '-device MIMXRT1064xxx6A'} + rec = {'name': 'openocd', 'uid': 'X', 'args': '-f interface/jlink.cfg -f target/foo.cfg'} + self.assertEqual(hil_flash.recover_flasher({'flasher': prim, 'flasher_recover': rec}), rec) + self.assertEqual(hil_flash.recover_flasher({'flasher': prim}), prim) + + def test_openocd_over_jlink_is_convoy_safe_without_a_pin(self): + """libjaylink discovery returns early unless idVendor == 0x1366 (SEGGER) and the PID + is in its table, and only THEN calls libusb_open (discovery_usb.c) -- it never opens + a foreign node. `adapter usb vid_pid` is a no-op for this driver: jlink.c reads + adapter_serial / usb address / usb location, never the vid/pid.""" + self.assertTrue(hil_flash.convoy_safe( + {'name': 'openocd', 'args': '-f interface/jlink.cfg -f target/stm32f4x.cfg'})) + + def test_openocd_with_neither_a_pin_nor_jlink_is_not_safe(self): + self.assertFalse(hil_flash.convoy_safe( + {'name': 'openocd', 'args': '-f interface/stlink.cfg -f target/stm32h7x.cfg'})) + + def test_the_existing_rules_are_unchanged(self): + self.assertTrue(hil_flash.convoy_safe( + {'name': 'openocd', 'vid_pid': '0x2e8a 0x000c', 'args': '-f interface/cmsis-dap.cfg'})) + self.assertFalse(hil_flash.convoy_safe({'name': 'jlink', 'uid': 'X'})) + self.assertTrue(hil_flash.convoy_safe({'name': 'esptool'})) + + +if __name__ == '__main__': + unittest.main(verbosity=1) diff --git a/test/hil/test/test_hil_util.py b/test/hil/test/test_hil_util.py new file mode 100644 index 000000000..9c3d5edef --- /dev/null +++ b/test/hil/test/test_hil_util.py @@ -0,0 +1,230 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: MIT +# Unit tests for hil_util.run_cmd's binary/split_stderr/quiet modes — real subprocesses, no +# hardware. Stdlib + hil_util only (hil_util is stdlib-only), so the pre-commit hil-test +# hook can run this on GitHub's bare runner. Run directly: +# python3 test/hil/test/test_hil_util.py +import io +import os +import shutil +import tempfile +import sys +import time +import unittest +from contextlib import redirect_stdout +from pathlib import Path + +# the module under test lives in the parent dir's helper/ package +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) +from helper import hil_util + + [email protected](os.name == 'nt', 'POSIX shell commands') +class RunCmdModes(unittest.TestCase): + def test_default_mode_unchanged(self): + r = hil_util.run_cmd('printf out; printf err >&2') + self.assertEqual(r.returncode, 0) + self.assertIsInstance(r.stdout, str) + # stderr merged into stdout, as every existing caller expects + self.assertIn('out', r.stdout) + self.assertIn('err', r.stdout) + + def test_binary_stdout_is_exact_bytes(self): + # \xff is not valid UTF-8: text mode would mangle it via errors='replace' + r = hil_util.run_cmd(r"printf 'a\377\000b'", binary=True) + self.assertEqual(r.returncode, 0) + self.assertEqual(r.stdout, b'a\xff\x00b') + + def test_split_stderr_keeps_stdout_clean(self): + r = hil_util.run_cmd('printf out; printf err >&2', split_stderr=True) + self.assertEqual(r.returncode, 0) + self.assertEqual(r.stdout, 'out') + self.assertEqual(r.stderr, 'err') + + def test_binary_split_stderr_timeout_returns_124(self): + t0 = time.monotonic() + r = hil_util.run_cmd(r"printf 'p\377re'; printf warn >&2; sleep 30", + binary=True, split_stderr=True, timeout=1) + self.assertEqual(r.returncode, 124) + # killpg + bounded communicate: well under sleep 30 + self.assertLess(time.monotonic() - t0, 15) + self.assertIn(b'p\xffre', r.stdout or b'') + # stderr collected before the timeout must survive the kill + self.assertIn(b'warn', r.stderr or b'') + + def test_text_mode_timeout_stdout_stays_str(self): + r = hil_util.run_cmd('sleep 30', timeout=1) + self.assertEqual(r.returncode, 124) + # a text-mode caller must never get bytes back, even empty + self.assertIsInstance(r.stdout, str) + + def test_failed_banner_includes_split_stderr(self): + # with split_stderr the diagnostic is in .stderr; the banner must not go blank. + # The text travels via env, not the command string — the banner title echoes the + # command, which would make a literal assertion pass vacuously. + os.environ['RUN_CMD_TEST_ERR'] = 'diagnostic-xyzzy' + self.addCleanup(os.environ.pop, 'RUN_CMD_TEST_ERR', None) + cap = io.StringIO() + with redirect_stdout(cap): + r = hil_util.run_cmd('printf "$RUN_CMD_TEST_ERR" >&2; exit 3', split_stderr=True) + self.assertEqual(r.returncode, 3) + self.assertIn('COMMAND FAILED', cap.getvalue()) + self.assertIn('diagnostic-xyzzy', cap.getvalue()) + + def test_no_group_markers_when_stdout_is_captured(self): + # GitHub folds ::group:: only at line start of the JOB's real stdout. Pool + # workers run tests under redirect_stdout and compact the capture into one + # row line, where the markers land mid-line and render as literal noise. + saved_ci = os.environ.get('CI') # pre-exists on GitHub runners: restore, not pop + os.environ['CI'] = '1' + self.addCleanup(lambda: os.environ.update({'CI': saved_ci}) if saved_ci is not None + else os.environ.pop('CI', None)) + cap = io.StringIO() + with redirect_stdout(cap): + r = hil_util.run_cmd('printf boom; exit 3') + self.assertEqual(r.returncode, 3) + self.assertIn('COMMAND FAILED', cap.getvalue()) + self.assertNotIn('::group::', cap.getvalue()) + self.assertNotIn('::endgroup::', cap.getvalue()) + + def test_quiet_suppresses_failed_banner(self): + # retry-loop callers report failures themselves; per-poll banners are noise + cap = io.StringIO() + with redirect_stdout(cap): + r = hil_util.run_cmd('printf boom >&2; exit 3', quiet=True) + self.assertEqual(r.returncode, 3) + self.assertNotIn('COMMAND FAILED', cap.getvalue()) + + +class BottomLayer(unittest.TestCase): + def test_bad_timeout_env_falls_back(self): + # hil_select (the PR-diff selector) imports hil_util for the example rosters; + # a malformed HIL_CMD_TIMEOUT must not crash the selector at import and knock + # CI back to the full-matrix fallback + import subprocess + r = subprocess.run( + [sys.executable, '-c', 'from helper import hil_util; print(hil_util.CMD_TIMEOUT)'], + cwd=os.path.dirname(os.path.dirname(os.path.abspath(__file__))), + env={**os.environ, 'HIL_CMD_TIMEOUT': 'bogus'}, + capture_output=True, text=True, timeout=30) + self.assertEqual(r.returncode, 0, r.stderr) + # the warning must NOT be on stdout: hil_select's stdout is machine-read JSON + self.assertEqual(r.stdout.strip(), '180') + self.assertIn('warning', r.stderr) # but a silent fallback hides the misconfiguration + + def test_tinyusb_root_is_the_repo_root(self): + # the constant is derived from __file__ parents[N]; moving hil_util.py without + # adjusting N silently re-points every firmware/build path (it happened) + self.assertTrue((hil_util.TINYUSB_ROOT / 'examples').is_dir(), hil_util.TINYUSB_ROOT) + self.assertTrue((hil_util.TINYUSB_ROOT / 'test' / 'hil').is_dir(), hil_util.TINYUSB_ROOT) + + def test_hil_util_is_a_single_module_instance(self): + # helper modules must be imported via the helper package everywhere: a plain + # `import hil_util` from inside helper/ creates a SECOND module object, and + # state like `verbose` set on one copy never reaches the other + import hil_flash + from helper import hil_pool_check + self.assertIs(hil_flash.hil_util, hil_util) + self.assertIs(hil_pool_check.hil_util, hil_util) + self.assertIs(hil_pool_check.hil_flash, hil_flash) + + def test_bare_runner_modules_stay_stdlib_only(self): + # hil_examples.py used to make this structural (a list of strings cannot grow a + # dependency); with the rosters folded into hil_util the invariant needs teeth: + # everything the bare GitHub runner imports (selector + this suite) must stay + # stdlib + local. Adding pyserial/pymtp here breaks hil_select on CI. + import ast + hil_dir = Path(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + # ONLY the modules the bare runner can import -- not every stem in the tree. + # Globbing the directory allowed `import pymtp` (and hil_test, usbtest, + # mtp_test) through, so the pymtp case this test names could never fail: that + # module runs ctypes.CDLL(find_library('mtp')) at import and raises where there + # is no libmtp, taking hil_select down with it. + local = {'helper', 'hil_util', 'hil_select', 'hil_flash', + 'hil_health', 'hil_lock', 'hil_pool_check'} + allowed = set(sys.stdlib_module_names) | local + # hil_pool_check included: test_hil_util_is_a_single_module_instance imports it + # on the bare runner, and its `import serial` is function-local for exactly + # this reason -- hoisting it must fail HERE, not on every PR's pre-commit CI + for mod in ('helper/hil_util', 'hil_flash', 'helper/hil_select', + 'helper/hil_health', 'helper/hil_lock', 'helper/hil_pool_check'): + tree = ast.parse((hil_dir / f'{mod}.py').read_text()) + # module level only: a deferred import inside a function cannot break + # importability (hil_pool_check keeps `import serial` function-local + # for exactly that reason) + for node in tree.body: + roots = [] + if isinstance(node, ast.Import): + roots = [a.name.split('.')[0] for a in node.names] + elif isinstance(node, ast.ImportFrom) and node.module: + roots = [node.module.split('.')[0]] + for root in roots: + self.assertIn(root, allowed, + f'{mod}.py imports {root}, not stdlib/local - breaks the bare CI runner') + + +class BoundedReadBookkeeping(unittest.TestCase): + """Two ways the strand accounting lied, both of which cost a blindness credit -- and + the process goes blind after four.""" + + def test_a_value_that_arrived_at_the_deadline_is_not_a_strand(self): + """join() returns, is_alive() is still True, but the reader HAS deposited its + value. read_sysfs booked a strand from is_alive() alone, so a merely-slow healthy + read was memoised as unreadable forever. bounded_open already gets this right.""" + import threading, time as _t + before = hil_util._sysfs_stuck + self.addCleanup(setattr, hil_util, '_sysfs_stuck', before) + real_thread = threading.Thread + + class Lingering(real_thread): + """Deposits the value, then outlives the join by a hair.""" + def run(self): + super().run() + _t.sleep(0.6) # still alive when join(grace) returns + + self.addCleanup(setattr, threading, 'Thread', real_thread) + threading.Thread = Lingering + with tempfile.NamedTemporaryFile('w', suffix='_attr', delete=False) as fh: + fh.write('cafe\n') + path = fh.name + self.addCleanup(os.unlink, path) + hil_util.read_sysfs(path, grace=0.2) + self.assertEqual(hil_util._sysfs_stuck, before, + 'a value that arrived was still counted as a strand') + + def test_bounded_open_does_not_re_strand_a_known_path(self): + """Same rule read_sysfs has: re-opening a path known to hang costs another thread, + another fd and another blindness credit to learn what we already know. The printer + test re-opens ONE lp node on every retry.""" + d = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, d, True) + fifo = os.path.join(d, 'lp0') + os.mkfifo(fifo) # open() blocks: no writer, ever + self.addCleanup(setattr, hil_util, '_sysfs_stuck', hil_util._sysfs_stuck) + self.addCleanup(setattr, hil_util, '_sysfs_stranded', dict(hil_util._sysfs_stranded)) + before = hil_util._sysfs_stuck + for _ in range(3): + hil_util.bounded_open(fifo, os.O_WRONLY, 0.3) + self.assertLessEqual(hil_util._sysfs_stuck - before, 1, + 'each retry spent another blindness credit on the same path') + + +class RunAlongsideKeepsStderrOffThePayload(unittest.TestCase): + """test_device_printer_to_cdc byte-compares run_alongside's stdout against the payload + it wrote. Merging stderr into that stream turns any stray child stderr byte -- a + PYTHONWARNINGS chirp, a sitecustomize print, a venv .pth deprecation -- into + 'CDC->Printer wrong data', sending a maintainer after the printer class driver for an + interpreter warning. hil_ci.sh runs python3 with no isolating flags.""" + + def test_child_stderr_does_not_contaminate_stdout(self): + from helper import hil_util + argv = [sys.executable, '-c', + 'import sys; sys.stderr.write("noise\\n"); sys.stdout.write("PAYLOAD")'] + r = hil_util.run_alongside(argv, lambda: time.sleep(0.2), timeout=20) + self.assertEqual(r.returncode, 0) + self.assertEqual(r.stdout, b'PAYLOAD', + 'child stderr leaked into the payload stream') + + +if __name__ == '__main__': + unittest.main() diff --git a/test/hil/tinyusb.json b/test/hil/tinyusb.json index 59ab57a06..6f552f126 100644 --- a/test/hil/tinyusb.json +++ b/test/hil/tinyusb.json @@ -1,6 +1,36 @@ { "boards": [ { + "name": "frdm_k64f", + "uid": "FFFF1A00FFFFFFFF1031454E0F000280", + "tests": { + "device": false, + "host": true, + "dual": false, + "dev_attached": [ + { + "vid_pid": "1a86_55d4", + "serial": "52D2023934", + "is_cdc": true, + "comment": "CH9102 USB-serial (TX-RX loopback)" + }, + { + "vid_pid": "21c4_0cc7", + "serial": "9000588268687E25", + "is_msc": true, + "block_size": 512, + "block_count": 60620800, + "msc_inquiry": "Lexar USB Flash Drive PMAP" + } + ] + }, + "flasher": { + "name": "jlink", + "uid": "000621000000", + "args": "-device MK64FN1M0xxx12" + } + }, + { "name": "ek_tm4c123gxl", "uid": "010105186C60A110", "tests": { @@ -26,6 +56,7 @@ "device/cdc_msc_freertos", "device/hid_composite_freertos", "device/audio_test_freertos", + "device/usbtest", "host/device_info", "host/msc_file_explorer_freertos" ], @@ -65,6 +96,7 @@ "device/cdc_msc_freertos", "device/hid_composite_freertos", "device/audio_test_freertos", + "device/usbtest", "host/device_info", "host/msc_file_explorer_freertos" ], @@ -115,9 +147,11 @@ "dual": false }, "flasher": { - "name": "openocd_adi", + "name": "openocd", "uid": "E6614C311B597D32", - "args": "-f interface/cmsis-dap.cfg -f target/max32665.cfg" + "vid_pid": "0x2e8a 0x000c", + "args": "-f interface/cmsis-dap.cfg -f target/max32665.cfg", + "verify": true } }, { @@ -149,8 +183,8 @@ } }, { - "name": "mimxrt1015_evk", - "uid": "DC28F865D2111D228D00B0543A70463C", + "name": "lpcxpresso11u37", + "uid": "17121919", "tests": { "device": true, "host": false, @@ -158,43 +192,13 @@ }, "flasher": { "name": "jlink", - "uid": "000726284213", - "args": "-device MIMXRT1015DAF5A" - } - }, - { - "name": "mimxrt1064_evk", - "uid": "BAE96FB95AFA6DBB8F00005002001200", - "tests": { - "device": true, - "host": true, - "dual": true, - "dev_attached": [ - { - "vid_pid": "10c4_ea60", - "serial": "0001", - "is_cdc": true, - "comment": "cp2102" - }, - { - "vid_pid": "21c4_0cc7", - "serial": "900058874D871F66", - "is_msc": true, - "block_size": 512, - "block_count": 60620800, - "msc_inquiry": "Lexar USB Flash Drive PMAP" - } - ] - }, - "flasher": { - "name": "jlink", - "uid": "000725299165", - "args": "-device MIMXRT1064xxx6A" + "uid": "000724441579", + "args": "-device LPC11U37/401" } }, { - "name": "lpcxpresso11u37", - "uid": "17121919", + "name": "lpcxpresso55s28", + "uid": "2BF1839A7D51F553A15AB03FD08F70AB", "tests": { "device": true, "host": false, @@ -202,8 +206,8 @@ }, "flasher": { "name": "jlink", - "uid": "000724441579", - "args": "-device LPC11U37/401" + "uid": "000727031389", + "args": "-device LPC55S28" } }, { @@ -251,7 +255,9 @@ "flasher": { "name": "openocd", "uid": "E6614103E72C1D2F", - "args": "-f interface/cmsis-dap.cfg -f target/rp2040.cfg -c \"adapter speed 5000\"" + "vid_pid": "0x2e8a 0x000c", + "args": "-f interface/cmsis-dap.cfg -f target/rp2040.cfg -c \"adapter speed 5000\"", + "verify": true } }, { @@ -280,7 +286,9 @@ "flasher": { "name": "openocd", "uid": "E6633861A3819D38", - "args": "-f interface/cmsis-dap.cfg -f target/rp2040.cfg -c \"adapter speed 5000\"" + "vid_pid": "0x2e8a 0x000c", + "args": "-f interface/cmsis-dap.cfg -f target/rp2040.cfg -c \"adapter speed 5000\"", + "verify": true }, "comment": "Test native host" }, @@ -305,7 +313,9 @@ "flasher": { "name": "openocd", "uid": "E6633861A3978538", - "args": "-f interface/cmsis-dap.cfg -f target/rp2350.cfg -c \"adapter speed 5000\"" + "vid_pid": "0x2e8a 0x000c", + "args": "-f interface/cmsis-dap.cfg -f target/rp2350.cfg -c \"adapter speed 5000\"", + "verify": true } }, { @@ -334,7 +344,9 @@ "flasher": { "name": "openocd", "uid": "E663AC91D3359B38", - "args": "-f interface/cmsis-dap.cfg -f target/rp2350.cfg -c \"adapter speed 5000\"" + "vid_pid": "0x2e8a 0x000c", + "args": "-f interface/cmsis-dap.cfg -f target/rp2350.cfg -c \"adapter speed 5000\"", + "verify": true } }, { @@ -379,13 +391,14 @@ "dual": false, "dev_attached": [ { - "vid_pid": "1a86_55d4", - "serial": "52D2003414", - "is_cdc": true + "vid_pid": "10c4_ea60", + "serial": "0001", + "is_cdc": true, + "comment": "cp2102" }, { "vid_pid": "21c4_0cc7", - "serial": "90005893730A1A63", + "serial": "900058874D871F66", "is_msc": true, "block_size": 512, "block_count": 60620800, @@ -413,9 +426,9 @@ "dual": false }, "flasher": { - "name": "openocd", + "name": "stlink", "uid": "004C00343137510F39383538", - "args": "-f interface/stlink.cfg -f target/stm32h7x.cfg" + "verify": true } }, { @@ -427,9 +440,9 @@ "dual": false }, "flasher": { - "name": "openocd", + "name": "stlink", "uid": "066FFF495087534867063844", - "args": "-f interface/stlink.cfg -f target/stm32g0x.cfg" + "verify": true }, "comment": "32-bit scheme, 2KB USB SRAM" }, @@ -474,9 +487,11 @@ "dual": false }, "flasher": { - "name": "openocd_wch", - "uid": "EBCA8F0670AF", - "args": "" + "name": "openocd", + "uid": "A76D8F062C2A", + "vid_pid": "0x1a86 0x8010", + "args": "-f target/wch-riscv.cfg", + "verify": false } }, { @@ -489,9 +504,32 @@ "dual": false }, "flasher": { - "name": "openocd_wch", + "name": "openocd", "uid": "BC4954081051", - "args": "" + "vid_pid": "0x1a86 0x8010", + "args": "-f target/wch-riscv.cfg", + "verify": false + } + }, + { + "name": "ch32v307v_r1_1v0", + "uid": "DE6B3E263B3857CAFFFFFFFF", + "toolchain": "riscv-gcc", + "variant": [ + {"name": "ch32v307v_r1_1v0-usbhs", "defines": ["SPEED=high"]}, + {"name": "ch32v307v_r1_1v0-usbfs", "defines": ["SPEED=full"]} + ], + "tests": { + "device": true, + "host": false, + "dual": false + }, + "flasher": { + "name": "openocd", + "uid": "BC5DA47360D0", + "vid_pid": "0x1a86 0x8010", + "args": "-f target/wch-riscv.cfg", + "verify": false } }, { @@ -504,14 +542,76 @@ "dual": false }, "flasher": { - "name": "openocd_wch", - "uid": "7FD88F0604B5", - "args": "" + "name": "openocd", + "uid": "57468F06DC03", + "vid_pid": "0x1a86 0x8010", + "args": "-f target/wch-riscv.cfg", + "verify": false + } + }, + { + "name": "mimxrt1064_evk", + "uid": "BAE96FB95AFA6DBB8F00005002001200", + "tests": { + "device": true, + "host": true, + "dual": true, + "dev_attached": [ + { + "vid_pid": "1a86_55d4", + "serial": "52D2003414", + "is_cdc": true + }, + { + "vid_pid": "21c4_0cc7", + "serial": "90005893730A1A63", + "is_msc": true, + "block_size": 512, + "block_count": 60620800, + "msc_inquiry": "Lexar USB Flash Drive PMAP" + } + ] + }, + "flasher": { + "name": "jlink", + "uid": "000725299165", + "args": "-device MIMXRT1064xxx6A" + } + }, + { + "name": "nrf54lm20dk", + "uid": "899C3DE5B0F4D5CA", + "tests": { + "device": true, + "host": false, + "dual": false, + "skip": ["device/audio_test_freertos"], + "comment": "board new to HIL: audio_test_freertos never reaches dcd_init (FreeRTOS itself runs; cdc_msc_freertos and usbtest pass) - example-level issue on nRF54L, fix separately" + }, + "flasher": { + "name": "jlink", + "uid": "1051856258", + "args": "-device NRF54LM20A_M33" } } ], "boards-skip": [ { + "name": "ra8m1_ek", + "uid": "797D142D36345030364E1737922E4B4E", + "comment": "USBHS bring-up pending (HS chirp completes digitally but terminations never switch; FS-forced build passed usbtest 30/30). Parked until fixed", + "tests": { + "device": true, + "host": false, + "dual": false + }, + "flasher": { + "name": "jlink", + "uid": "001083115236", + "args": "-device R7FA8M1AH" + } + }, + { "name": "stm32f769disco", "uid": "21002F000F51363531383437", "variant": [ @@ -528,6 +628,23 @@ "uid": "000778170924", "args": "-device stm32f769ni" } + }, + { + "name": "ra6m5_ek", + "uid": "8419032D32363657364EF4622D294B4E", + "comment": "Unstable in CI: intermittent usbtest failures plus cdc_dual_ports/hid_boot_interface/midi_test/mtp/printer_to_cdc flapping. Parked until diagnosed", + "tests": { + "device": true, + "host": false, + "dual": false, + "skip": ["device/cdc_msc_throughput", "device/msc_dual_lun"], + "comment": "MSC writes wedge the uPD720201 host (URBs queued, zero wire activity, bus-15 ctrl xfers time out until the device's URBs are killed); reproduced identically with master firmware - device side armed+BUF and exonerated. MSC reads and usbtest bulk (15.8 MB/s) are fine" + }, + "flasher": { + "name": "jlink", + "uid": "000831915224", + "args": "-device R7FA6M5BH" + } } ] } diff --git a/test/hil/usbtest.py b/test/hil/usbtest.py new file mode 100755 index 000000000..485e9e0e4 --- /dev/null +++ b/test/hil/usbtest.py @@ -0,0 +1,873 @@ +#!/usr/bin/env python3 +"""Run the Linux kernel usbtest/testusb battery against a TinyUSB usbtest device. + +Device firmware: examples/device/usbtest (VID:PID cafe:4010). The firmware +advertises its capability tier in bcdDevice low byte; the battery is selected +accordingly (see examples/device/usbtest/README.md). + +Requires: usbtest kernel module (CONFIG_USB_TEST), testusb binary (built from +kernel tools/usb/testusb.c), sudo for driver binding + usbfs ioctls. + +testusb reporting quirks this script works around: +- its exit code is always 0 when the device exists: results are parsed from stdout +- a case gated off by the driver's capability profile (or an in-kernel parameter + check) returns -EOPNOTSUPP, which testusb silently skips: a missing result line + means NOT RUN, and is reported as a failure since every case in the selected + battery is expected to run. + +Binding uses the 5-field new_id form referencing Gadget Zero (0525:a4a0) so the +dynamic id inherits its capability profile (autoconf + ctrl_out + iso + intr). +Never register a plain "vid pid" dynamic id with usbtest: the dynid then has +driver_info == 0 and usbtest_probe() dereferences it without a NULL check +(kernel oops). autoconf is also what enables bulk endpoint discovery; the +capability flags only unlock cases, they don't require the endpoints to exist. +""" + +import argparse +import json +from contextlib import redirect_stdout +import os +import pathlib +import re +import shutil +import subprocess +import sys +import time +from pathlib import Path + +sys.path.append(os.path.dirname(os.path.abspath(__file__))) # PYTHONSAFEPATH drops it + +VID = 'cafe' +PID = '4010' +GZ_REF = '0525 a4a0' # copy Gadget Zero's capability profile (ctrl_out+iso+intr) +SYS_USB = Path('/sys/bus/usb/devices') +DRIVER = Path('/sys/bus/usb/drivers/usbtest') +PATTERN_PARAM = Path('/sys/module/usbtest/parameters/pattern') +RECOVER_FLASH_TIMEOUT = 90 # bound on the post-hang reflash; typical flash is 10-20s +RECOVER_RESET_TIMEOUT = 30 # bound on the post-hang probe reset; ResetTarget measures ~130ms + + +def recovery_steps(flasher_name: str, time_left: float) -> list: + """Ordered (kind, bound) recovery attempts that fit in `time_left`. + + RESET FIRST, reflash second. A probe reset fails the in-flight URB at the source just + as a park-flash does, but it is non-destructive -- the firmware under test survives, so + the wedge can still be autopsied -- writes no flash, and cannot brick SWD the way a bad + park image has on mimxrt1064_evk and max32666fthr (survived a power cycle). Measured + 128-129 ms against a full erase+program, and it works on i.MX RT and on DWC2 alike + (stm32f407disco, 2026-08-16: `r; g` -> USB disconnect, re-enumerated 325 ms later). + + The reset also fits budgets a reflash does not: the old gate skipped recovery entirely + when RECOVER_FLASH_TIMEOUT did not fit, which left the holder in place for the next + job. Whether either worked is decided by wedged_pids(), never by the exit code -- a + clean flash only proves the probe wrote the MCU. + """ + import hil_flash + steps = [] + reset_fn = getattr(hil_flash, f'reset_{flasher_name.lower()}', None) + if getattr(reset_fn, 'no_op', False): + reset_fn = None # a stub that returns rc 0 without resetting: do not claim it + if reset_fn and time_left >= RECOVER_RESET_TIMEOUT: + steps.append(('reset', RECOVER_RESET_TIMEOUT)) + if time_left >= RECOVER_FLASH_TIMEOUT: + steps.append(('flash', RECOVER_FLASH_TIMEOUT)) + return steps +HELPER_TIMEOUT = 30 # default bound for sudo helpers (dmesg/modprobe/setpci/tee) + +# Battery per tier, in run order: control sanity, simple bulk, queued, unaligned, unlink, +# halt/toggle, throughput last. +TIER_CASES = { + 1: [0, 9, 10, 1, 2, 3, 4, 5, 6, 7, 8, 17, 18, 19, 20, 11, 12, 24, 13, 29, 27, 28], + 2: [14, 21], + 3: [25, 26], + 4: [15, 16, 22, 23], +} + +# Per-case testusb parameters (full speed / high speed). All -s/-v values are multiples +# of 512 so transfers stay packet-aligned at both speeds: the device streams whole max-size +# packets and a non-aligned IN length would babble. 14/21 must never run with defaults +# (vary >= length is -EINVAL in the kernel). +PARAMS = { + 0: ('-c 1', '-c 1'), + 9: ('-c 256', '-c 1000'), + 10: ('-c 64 -g 16', '-c 256 -g 16'), + **{n: ('-c 128 -s 1024 -v 512', '-c 512 -s 1024 -v 512') for n in (1, 2, 3, 4, 17, 18, 19, 20)}, + **{n: ('-c 8 -s 1024 -g 8', '-c 32 -s 1024 -g 16') for n in (5, 6, 7, 8)}, + **{n: ('-c 64 -s 1024 -g 8', '-c 256 -s 1024 -g 8') for n in (11, 12, 24)}, + 13: ('-c 16 -s 512', '-c 64 -s 512'), + 29: ('-c 16 -s 512', '-c 64 -s 512'), + 27: ('-c 16 -s 1024 -g 32', '-c 128 -s 1024 -g 32'), + 28: ('-c 16 -s 1024 -g 32', '-c 128 -s 1024 -g 32'), + 14: ('-c 64 -s 512 -v 61', '-c 256 -s 512 -v 61'), + 21: ('-c 64 -s 512 -v 61', '-c 256 -s 512 -v 61'), + 25: ('-c 32 -s 512', '-c 256 -s 1024'), + 26: ('-c 32 -s 512', '-c 256 -s 1024'), + **{n: ('-c 16 -s 512 -g 8', '-c 64 -s 1024 -g 8') for n in (15, 16, 22, 23)}, +} + +CASE_NAMES = { + 0: 'NOP', 1: 'bulk write', 2: 'bulk read', 3: 'bulk write vary', 4: 'bulk read vary', + 5: 'bulk sg write', 6: 'bulk sg read', 7: 'bulk sg write vary', 8: 'bulk sg read vary', + 9: 'ch9 subset', 10: 'queued control', 11: 'unlink reads', 12: 'unlink writes', + 13: 'ep halt set/clear', 14: 'ctrl_out write/read', 15: 'iso write', 16: 'iso read', + 17: 'bulk write unaligned', 18: 'bulk read unaligned', 19: 'bulk write premapped', + 20: 'bulk read premapped', 21: 'ctrl_out unaligned', 22: 'iso write unaligned', + 23: 'iso read unaligned', 24: 'unlink queued writes', 25: 'int write', 26: 'int read', + 27: 'bulk write perf', 28: 'bulk read perf', 29: 'toggle clear', +} + +RE_PASS = re.compile(r'test (\d+),\s*(\d+)\.(\d+) secs') +RE_FAIL = re.compile(r'test (\d+) --> (\d+) \((.*)\)') + + +def run(cmd, **kw): + # NOT subprocess.run(timeout=): CPython's post-timeout path is an UNBOUNDED wait() that + # never returns on a D-state child -- the hang sysfs_write's timeout exists to catch. + timeout = kw.pop('timeout', None) + data = kw.pop('input', None) # subprocess.run-only kwarg; Popen takes stdin + kw.pop('capture_output', None) # ditto: expressed by the PIPEs below + kw.setdefault('text', True) + kw.setdefault('encoding', 'utf-8') + kw.setdefault('errors', 'replace') # strict decode would raise out of _sudo_soft + # NO start_new_session: these helpers (dmesg, modprobe, setpci, tee) must stay in our + # process group so hil_test's outer killpg reaps them with us. + timeout = timeout if timeout is not None else HELPER_TIMEOUT + proc = subprocess.Popen(cmd, stdin=subprocess.PIPE if data is not None else None, + stdout=subprocess.PIPE, stderr=subprocess.PIPE, **kw) + try: + out, err = proc.communicate(input=data, timeout=timeout) + return subprocess.CompletedProcess(cmd, proc.returncode, out, err) + except subprocess.TimeoutExpired: + # Under sudo our child is only the wrapper; the root grandchild survives this and + # is left for the report and hil_pool_check to name. Close our pipe ends so an + # abandoned child costs no fds. + try: + proc.kill() # same group as us: never killpg, that would kill us too + except OSError: + pass + try: + proc.communicate(timeout=5) + except subprocess.TimeoutExpired: + for pipe in (proc.stdout, proc.stderr, proc.stdin): + try: + if pipe is not None: + pipe.close() + except OSError: + pass # unkillable: abandon it, the caller reports the timeout + raise + + +def sudo(cmd, **kw): + if os.geteuid() != 0: + cmd = ['sudo', '-n'] + cmd + r = run(cmd, **kw) + if r.returncode != 0 and 'password is required' in (r.stderr or ''): + sys.exit(f'sudo needs a password for: {" ".join(cmd)}\n' + 'Run as root, or grant this user NOPASSWD sudo.') + return r + + +def sysfs_write(path, data, check=True): + # A driver-registry write (new_id/remove_id/bind) blocks in D state when a wedged + # device holds its lock: fail fast instead of piling up unkillable writers -- the rig + # needs USB recovery first. + # + # Verified in v6.12.96: unbind_store -> device_driver_detach -> + # device_release_driver_internal -> __device_driver_lock (drivers/base/dd.c), which + # takes device_lock() -- the UNINTERRUPTIBLE variant, unlike the sysfs read path -- and + # ALSO device_lock(parent), because usb_bus_type sets need_parent_lock = true + # (drivers/usb/core/driver.c:2048). So one such write against a wedged device blocks + # unkillably while holding the HUB's lock: that is the mechanism by which a single + # wedged port takes its whole bus down, and why this fails fast instead. + try: + r = sudo(['tee', str(path)], input=data, timeout=15) + except subprocess.TimeoutExpired: + sys.exit(f'write "{data}" > {path} blocked >15s: USB subsystem is wedged ' + '(a D-state device lock exists). Recover the rig (usb-kernel-recover skill) ' + 'before running batteries.') + if check and r.returncode != 0: + sys.exit(f'write "{data}" > {path} failed: {r.stderr.strip()}') + return r.returncode == 0 + + +def _read_sysfs_bounded(path, grace=1.0): + """Bounded sysfs attribute read. The value, or None, or hil_util.SYSFS_UNKNOWN. + + Delegates to hil_util.read_sysfs (imported here, like every helper import in this + file) so both properties hold: the strand cap -- find_device re-scans after EVERY + case, so a 30-case battery against a wedged peer would otherwise strand dozens of + threads and fds -- and UNKNOWN kept distinct from None. Folding UNKNOWN into None made + a blinded scan read as "device dropped off the bus", which aborts down a path that + skips the HUNG recovery entirely. + """ + from helper import hil_util + return hil_util.read_sysfs(str(path), grace) + + +_DEV_CACHE: dict = {} # serial -> sysname, see find_device + + +def _reread(sysname, serial): + """Re-describe an already-resolved device, CONFIRMING its serial. + + idVendor/idProduct/busnum/devnum/speed are lock-free (sysfs.c:688-705), so they cannot + block on a wedged peer -- but every identical board answers them the same, so they + prove nothing about identity. `serial` does, at one bounded read: a sysname is a + topology path, and after a renumber (controller reset, reboot) it can name a DIFFERENT + cafe:4010 board whose verdicts would be filed under this one. Returns None when the + serial is gone, mismatched or unconfirmed -- caller falls back to a full scan. + """ + d = SYS_USB / sysname + try: + if ((d / 'idVendor').read_text().strip() != VID + or (d / 'idProduct').read_text().strip() != PID): + return None + dev_serial = _read_sysfs_bounded(d / 'serial') + if not isinstance(dev_serial, str) or dev_serial.lower() != serial.lower(): + return None # gone, mismatched, or unconfirmable -> full scan decides + return { + 'sysname': sysname, + 'serial': dev_serial, + 'node': '/dev/bus/usb/%03d/%03d' % (int((d / 'busnum').read_text()), + int((d / 'devnum').read_text())), + 'speed': (d / 'speed').read_text().strip(), + 'tier': int((d / 'bcdDevice').read_text().strip()[-2:], 16), + } + except (OSError, ValueError): + return None + + +def find_device(serial, first=False): + """Locate the usbtest device in sysfs, return info dict or None. + + Cached by serial: this is called after EVERY case, and a full scan pays a bounded + but real `serial` read for every cafe:4010 peer on the rig. With another board + wedged that cost lands on a HEALTHY battery ~30 times over, truncating it into + BUDGET entries. The fast path pays ONE bounded read -- our own device's serial, the + only attribute that tells identical boards apart (see _reread). + """ + if serial: + sysname = _DEV_CACHE.get(serial.lower()) + if sysname: + hit = _reread(sysname, serial) + if hit: + return hit + _DEV_CACHE.pop(serial.lower(), None) + matches, inconclusive = [], [] + for dev in SYS_USB.iterdir(): + try: + if (dev / 'idVendor').read_text().strip() != VID or \ + (dev / 'idProduct').read_text().strip() != PID: + continue + # BOUNDED: idVendor/idProduct are cached descriptors, but `serial` is served + # under device_lock(), so an unbounded read blocks us in D state on exactly the + # DUT whose hang we are here to report, losing every verdict collected so far. + dev_serial = _read_sysfs_bounded(dev / 'serial') + if dev_serial is not None and not isinstance(dev_serial, str): + inconclusive.append(dev.name) # unknown: NOT proof it is not ours + continue + if dev_serial is None: + continue + if serial and dev_serial.lower() != serial.lower(): + continue + matches.append({ + 'sysname': dev.name, + 'serial': dev_serial, + 'node': '/dev/bus/usb/%03d/%03d' % (int((dev / 'busnum').read_text()), + int((dev / 'devnum').read_text())), + 'speed': (dev / 'speed').read_text().strip(), + 'tier': int((dev / 'bcdDevice').read_text().strip()[-2:], 16), + }) + except (OSError, ValueError): + continue + if not matches: + # "could not tell" is not "gone". The caller aborts the battery on a falsy return + # and that path skips the HUNG reflash, so a blinded scan would report the wedge + # we exist to recover from as a physical disconnect. + return {'inconclusive': inconclusive} if inconclusive else None + if serial and len(matches) == 1: + _DEV_CACHE[serial.lower()] = matches[0]['sysname'] + if len(matches) > 1 and not first: + if serial: + # Dual-port parts (nanoch32v203, ch32v307) briefly enumerate BOTH ports with + # one serial around a variant reflash, and picking one could bind the stale + # port -- report ambiguity so the caller retries until it drops. + return {'ambiguous': sorted(m['sysname'] for m in matches)} + sys.exit(f'multiple {VID}:{PID} devices found, use --serial: ' + + ', '.join(m["serial"] for m in matches)) + return matches[0] + + +def check_host_compat(dev): + """Refuse to run when the DUT's upstream host controller is known-incompatible: + the MosChip MCS9990 (9710:9990) outright (buggy FRINDEX silicon: EHCI never + schedules int-OUT URBs and mangles unlinked reads - verified A/B 2026-07-09), + and the Renesas uPD720201/02 unless it runs firmware >= 2.0.2.6 (see below).""" + for attempt in range(3): + try: + root = Path(f"/sys/bus/usb/devices/usb{int(dev['node'].split('/')[-2])}") + drv = (root / '../driver').resolve().name + pci = (root / '..').resolve() + vid_did = ((pci / 'vendor').read_text().strip(), (pci / 'device').read_text().strip()) + break + except (OSError, ValueError): + # transient sysfs error (racing a re-enumeration): retry so a blip does not + # silently pass an incompatible host, then fail open but say so + if attempt == 2: + print('warning: cannot probe the upstream host controller; ' + 'skipping the host compatibility check', file=sys.stderr) + return + time.sleep(1) + if vid_did == ('0x9710', '0x9990'): + sys.exit(f'REFUSING to run: DUT is behind a MosChip MCS9990 ({pci.name}), which is ' + 'incompatible with usbtest: broken FRINDEX silicon - int-OUT URBs are never ' + 'placed in the EHCI periodic schedule and unlinked reads complete as short ' + 'transfers (EREMOTEIO). Move the DUT to an xHCI port.') + if drv.startswith('xhci') and vid_did in (('0x1912', '0x0014'), ('0x1912', '0x0015')): + # The Renesas uPD720201/uPD720202 must run firmware >= 2.0.2.6 (K2026090.mem; + # RAM-uploaded, so it reverts to ROM on every power cycle): on ROM firmware its + # command ring dies under unlink stress and the hub worker deadlocks holding the + # device lock, needing a host power cycle (ch32v307 2026-07-10; ra6m5 test 24, + # mimxrt1015 2026-07-11). Both parts expose the FW version at PCI config 0x6c. + # Necessary, not sufficient -- batteries have killed the controller on current + # firmware too, which per-board skips in the rig config handle. + fw = None + try: + r = _sudo_soft(['setpci', '-s', pci.name, '0x6c.l'], capture_output=True, text=True) + if r.returncode == 0: + fw = int(r.stdout.strip(), 16) + except (OSError, ValueError): + pass + if fw is None: + sys.exit(f'REFUSING to run: cannot read host xHCI Renesas ({pci.name}) firmware ' + 'version (setpci missing or not permitted) - usbtest requires verified ' + 'firmware >= 0x00202609 (2.0.2.6); on older firmware the command ring ' + 'dies under unlink stress. Install pciutils / fix sudo, or load the ' + 'firmware and re-check.') + if fw < 0x00202609: + sys.exit(f'REFUSING to run: host xHCI Renesas ({pci.name}) firmware 0x{fw:08x} ' + '< 0x00202609 (2.0.2.6) - its command ring dies under usbtest unlink ' + 'stress. Load the latest firmware (K2026090.mem; it is RAM-uploaded and ' + 'reverts to ROM on every power cycle).') + + +def bind_usbtest(dev): + """Bind the device's interface 0 to the usbtest driver.""" + if not DRIVER.exists(): + r = _sudo_soft(['modprobe', 'usbtest']) + if r.returncode != 0 or not DRIVER.exists(): + sys.exit(f'cannot load usbtest module: {r.stderr.strip()}') + + # always re-register in case a stale dynamic id carries a different profile + intf = f'{dev["sysname"]}:1.0' + drv = SYS_USB / intf / 'driver' + stale_binding = drv.is_symlink() and drv.resolve().name == 'usbtest' + sysfs_write(DRIVER / 'remove_id', f'{VID} {PID}', check=False) + sysfs_write(DRIVER / 'new_id', f'{VID} {PID} 0 {GZ_REF}') + if stale_binding: + # it probed against the OLD dynamic id's capability profile; unbind once (the + # device is idle here) so the loop below reprobes the fresh one + sysfs_write(drv / 'unbind', intf, check=False) + + deadline = time.monotonic() + 3 + while time.monotonic() < deadline: + drv = SYS_USB / intf / 'driver' + if drv.is_symlink(): + if drv.resolve().name == 'usbtest': + return + # claimed by a foreign driver: steal the interface + sysfs_write(drv / 'unbind', intf) + sysfs_write(DRIVER / 'bind', intf, check=False) + time.sleep(0.2) + sys.exit(f'interface {intf} did not bind to usbtest') + + +def set_pattern(value): + try: + if PATTERN_PARAM.read_text().strip() != str(value): + sysfs_write(PATTERN_PARAM, str(value)) + except OSError as e: # FileNotFoundError (no param), PermissionError (root-only), ... + sys.exit(f'{PATTERN_PARAM} not usable ({e.strerror}): this usbtest module build may lack ' + 'the "pattern" param, or it is not readable') + + +def _sudo_soft(cmd, **kw): + """sudo() for calls whose failure must never abort the battery: run() re-raises + TimeoutExpired, and two of these are evaluated inside run_case's own timeout handler + -- a raise there loses the HUNG verdict, the recovery and the JSON report.""" + try: + return sudo(cmd, **kw) + except (OSError, ValueError, subprocess.SubprocessError, SystemExit) as e: + # SystemExit too: sudo() sys.exit()s on 'a password is required', unwinding out of + # run_case's timeout handler before the HUNG verdict is recorded -- which leaves + # unrecovered_hang False and lets the finally run the remove_id/unbind that must + # never happen while a D-state device lock is held + print(f'{cmd[0]}: {type(e).__name__}: {e}', file=sys.stderr) + return subprocess.CompletedProcess(cmd, 1, '', '') + + +def dmesg_tail(): + r = _sudo_soft(['dmesg']) + lines = [l for l in r.stdout.splitlines() if 'usbtest' in l] + return '\n'.join(lines[-8:]) + + + + +def wedged_pids(devnode): + """(pids, complete): pids still in D state on `devnode` after a recovery reflash. + + Matched by device node rather than by our child's pid because run_case() may wrap + testusb in sudo: the Popen pid is then the wrapper and the blocked process is its + child. A clean flash only proves the probe wrote the MCU, not that the D-state holder + let go -- this is what tells the two apart. + + FAIL CLOSED. `complete` is False when an entry could be HIDDEN from us, and the caller + must then keep treating the hang as unrecovered: the holder is root-owned (run_case + uses `sudo -n` whenever the node is not writable) and a hidepid/ProtectProc mount + hides exactly that entry. Reporting "no holder" from a scan that could not see it + clears unrecovered_hang and lets cleanup run remove_id/unbind against a device whose + usbfs lock is still held -- which deadlocks the bus, not just this board. + + Self-contained: /proc is plain text and this is one pass over it, so importing a + helper to do it would only add a failure mode on the recovery path. + """ + stuck, complete = [], True + # A restricted /proc hides other users' entries ENTIRELY -- no entry, so no + # PermissionError to catch -- and testusb runs under sudo, so the holder is exactly + # what is hidden. Detect the restriction itself rather than its symptom. + if os.geteuid() != 0 and not os.access('/proc/1/cmdline', os.R_OK): + complete = False + for d in pathlib.Path('/proc').glob('[0-9]*'): + try: + st = (d / 'stat').read_bytes() + if st[st.rindex(b')') + 2:st.rindex(b')') + 3] != b'D': + continue + if devnode.encode() in (d / 'cmdline').read_bytes(): + stuck.append(int(d.name)) + except PermissionError: + complete = False # cannot rule this pid out + except (OSError, ValueError): + continue # raced with exit + return stuck, complete + + +def run_case(num, dev, testusb, quick, timeout): + fs_hs = PARAMS[num][0 if dev['speed'] == '12' else 1] + if quick: + fs_hs = re.sub(r'-c (\d+)', lambda m: f'-c {max(1, int(m.group(1)) // 8)}', fs_hs) + cmd = [testusb, '-D', dev['node'], '-t', str(num)] + fs_hs.split() + # device nodes are usually opened directly (udev rule); sudo only if not + if not os.access(dev['node'], os.W_OK) and os.geteuid() != 0: + cmd = ['sudo', '-n'] + cmd + result = {'num': num, 'name': CASE_NAMES[num], 'params': fs_hs} + + # NO start_new_session: testusb must stay in OUR process group so the caller's outer + # killpg still reaps it; a sudo-wrapped child is escalated through sudo below instead. + # errors='replace': testusb output is not guaranteed UTF-8, and a strict decode would + # raise out of here and out of main(), printing no JSON at all (battery '0/30'). + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, + text=True, encoding='utf-8', errors='replace') + try: + out, _ = p.communicate(timeout=timeout) + except subprocess.TimeoutExpired: + # Under sudo we only kill the wrapper; its root-owned testusb keeps the inherited + # stdout pipe, so the reap below times out and the overrun is reported as HUNG. + # Accepted rather than escalated: the rig's udev rules make the device node + # writable, so sudo is the exception, and the harness must never sudo-kill a pid + # it cannot prove is its own. + try: + p.kill() + except OSError: + pass + try: + out, _ = p.communicate(timeout=5) + except subprocess.TimeoutExpired: + # SIGKILL had no effect: the child is in uninterruptible sleep on an in-kernel + # usbfs ioctl. Abandon it — waiting or re-signalling can never succeed. + result.update(status='HUNG', detail=f'testusb stuck in D state after {timeout}s', + dmesg=dmesg_tail()) + return result + result.update(status='FAIL', detail=f'timeout after {timeout}s', dmesg=dmesg_tail()) + return result + + m = RE_PASS.search(out) + if m and int(m.group(1)) == num: + secs = float(f'{m.group(2)}.{m.group(3)}') + result.update(status='PASS', secs=secs) + if num in (27, 28) and secs > 0: + opts = dict(zip(fs_hs.split()[::2], fs_hs.split()[1::2])) + total = int(opts['-c']) * int(opts['-s']) * int(opts['-g']) + result['mbps'] = round(total / secs / 1e6, 2) + return result + + m = RE_FAIL.search(out) + if m and int(m.group(1)) == num: + result.update(status='FAIL', detail=f'errno {m.group(2)} ({m.group(3)})', + dmesg=dmesg_tail()) + return result + + if cmd[0] == 'sudo' and ('password is required' in out or 'a terminal is required' in out): + result.update(status='FAIL', detail='sudo needs a password to run testusb: the device node ' + 'is not writable') + return result + + # no result line: the kernel returned -EOPNOTSUPP (capability profile or + # in-kernel parameter gate) and testusb skipped silently + result.update(status='NOTRUN', detail='case gated off: check binding profile/pattern', + stderr=out.strip()) + return result + + +def main(): + p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + p.add_argument('--serial', help='board uid (USB serial string) to select the device') + p.add_argument('--tier', type=int, choices=sorted(TIER_CASES), + help='override tier (default: from device bcdDevice)') + p.add_argument('--tests', help='comma-separated case numbers, overrides tier battery') + p.add_argument('--quick', action='store_true', help='divide iteration counts by 8') + p.add_argument('--json', action='store_true', help='machine-readable output on stdout') + p.add_argument('--keep-binding', action='store_true', help='leave usbtest dynamic id registered') + p.add_argument('--testusb', default=None, help='path to testusb binary') + p.add_argument('--timeout', type=int, default=120, help='per-case timeout in seconds') + p.add_argument('--recover-board', help='board JSON (name + flasher) for the post-hang ' + 'reflash recovery; without it a HUNG case leaves the device wedged') + p.add_argument('--recover-fw', help='firmware path reflashed by the post-hang recovery') + p.add_argument('--outer-timeout', type=int, default=0, + help='the caller\'s total bound on this process; a reflash that cannot ' + 'finish before it is skipped rather than orphaned mid-flash') + p.add_argument('--budget', type=int, default=0, + help='stop starting new cases after this many seconds (0 = no limit). ' + 'Callers that impose their own outer timeout set this to reserve ' + 'the remainder for the post-hang recovery path') + args = p.parse_args() + t_start = time.monotonic() + sys.stdout.reconfigure(line_buffering=True) # per-case results visible when piped/logged + + testusb = args.testusb or shutil.which('testusb') or os.path.expanduser('~/testusb') + if not os.access(testusb, os.X_OK): + sys.exit('testusb binary not found: build kernel tools/usb/testusb.c ' + 'and install it, or pass --testusb') + + # retry briefly: after a flash the enumeration may still be settling, and a dual-port + # part's stale same-serial node takes a moment to drop off (see find_device) + deadline = time.monotonic() + 8 + while True: + dev = find_device(args.serial) + # find_device is THREE-valued: a device, {'ambiguous': [...]}, or + # {'inconclusive': [...]} when bounded reads could not rule a device out. Screening + # only for 'ambiguous' let the inconclusive marker through as if it were a device, + # and the next statement subscripts dev['tier'] -> KeyError, no JSON on stdout, and + # hil_test reports "usbtest did not run / 0-30" for a merely-unreadable bus. + if dev and not ({'ambiguous', 'inconclusive'} & dev.keys()): + break + if time.monotonic() > deadline: + if dev and 'ambiguous' in dev: + sys.exit(f"multiple devices with serial {args.serial}: {', '.join(dev['ambiguous'])} " + '— stale enumeration from another port? replug or retry') + if dev and 'inconclusive' in dev: + from helper import hil_util as _hu + sys.exit(f"cannot tell whether {VID}:{PID} is present: bounded sysfs reads " + f"did not answer for {', '.join(dev['inconclusive'])}" + f"{_hu.sysfs_blind_note()}") + sys.exit(f'no {VID}:{PID} device' + (f' with serial {args.serial}' if args.serial else '')) + time.sleep(0.5) + + # a stale/foreign device advertising an out-of-range tier must not silently run an + # empty battery ('0/0 passed' would read as green in CI) + tier = args.tier or dev['tier'] + if not 1 <= tier <= max(TIER_CASES): + sys.exit(f"device advertises tier {tier} (bcdDevice ...{tier:02x}); reflash a usbtest build " + f"or pass --tier 1..{max(TIER_CASES)} — refusing to run an unknown/empty battery") + if args.tests: + cases = [] + for tok in args.tests.split(','): + tok = tok.strip() + if not tok.isdecimal() or int(tok) not in PARAMS: # isdecimal rejects unicode digits + sys.exit(f'--tests: {tok!r} is not a known case number (valid 0..{max(PARAMS)})') + cases.append(int(tok)) + else: + cases = [n for t in range(1, tier + 1) for n in TIER_CASES[t]] + + info = f"device {dev['serial']} {dev['node']} speed={dev['speed']} tier={tier}" + if not args.json: + print(info) + + # before touching the device: an incompatible host exits here, before any bind + check_host_compat(dev) + + results = [] + unrecovered_hang = False + try: + bind_usbtest(dev) + set_pattern(0) # tier 1 firmware sources zeros; also required by perf cases 27/28 + + abort_reason = None # set on any early exit; drives the BUDGET back-fill below + for idx, num in enumerate(cases): + # Only a HUNG case aborts the battery; an ordinary case timeout is a FAIL and + # the loop continues, each burning --timeout+5s, so without this the run can + # still be in the case loop when the outer timeout SIGKILLs it before it emits + # JSON. Checked before dispatch: worst overshoot is one case. + if args.budget and time.monotonic() - t_start > args.budget: + abort_reason = f'battery budget {args.budget}s exhausted' + break + results.append(run_case(num, dev, testusb, args.quick, args.timeout)) + r = results[-1] + if not args.json: + extra = f" {r.get('secs', '')}s" if r['status'] == 'PASS' else f" {r.get('detail', '')}" + extra += f" {r['mbps']} MB/s" if 'mbps' in r else '' + print(f"test {num:2d} {r['name']:22s} {r['status']:6s}{extra}") + if r['status'] == 'HUNG': + abort_reason = 'battery aborted on a kernel-side hang' + # Reflash, NEVER a root-port cycle: resetting the MCU through the DUT's own + # debug probe fails the in-flight URB at the source, so the ioctl returns, + # the queued kill lands and the cleanup below is lock-safe -- and it reaches + # exactly one board, where a root-port cycle bounces every fixture under the + # port (and could never remove power anyway; see usb-kernel-recover). + # Deliberately not gated on a hub-worker check: our own stuck testusb is + # what drives a hub worker into usb_lock_device(), so a pre-check reads + # wedged by construction. + # + # Assume unrecovered until proven otherwise, so any early exit from this + # block reaches the finally with the flag set instead of running the + # remove_id/unbind that must not happen while a device lock is held. + unrecovered_hang = True + print('aborting battery: kernel-side hang, device wedged mid-transfer', + file=sys.stderr) + if not (args.recover_board and args.recover_fw): + print('no --recover-board/--recover-fw: the device stays wedged and ' + 'cleanup is skipped', file=sys.stderr) + break + # The reflash is bounded to RECOVER_FLASH_TIMEOUT and skipped when the + # caller's outer bound cannot contain it: the flasher runs in its own + # session, so an outer killpg mid-flash would ORPHAN it on the probe. Gate + # each step on the time actually LEFT -- reserving for the worst case up + # front skipped recovery for nearly every real hang, since the hang-prone + # cases run late in the tier order. + def _time_left(): + if not args.outer_timeout: + return float('inf') + # what still runs after a step: run_cmd's post-kill reap (10s), + # the settle (5s), the sudo-escalated descendant reap run_case may + # have just paid (up to 7s) and the JSON write + return args.outer_timeout - (time.monotonic() - t_start) - 35 + + if _time_left() < RECOVER_RESET_TIMEOUT: + print('insufficient time before the outer bound for even a bounded ' + 'reset; the device stays wedged and cleanup is skipped', + file=sys.stderr) + break + try: + board = json.loads(args.recover_board) + bname, fname = board['name'], board['flasher']['name'] + import hil_flash # deferred: stdlib-only unless recovery actually runs + flash_fn = getattr(hil_flash, f'flash_{fname.lower()}') + reset_fn = getattr(hil_flash, f'reset_{fname.lower()}', None) + except Exception as e: # malformed/short json, import failure, unknown flasher + print(f'reflash recovery unavailable ({e})', file=sys.stderr) + break + # DELIVERY must be convoy-safe or the recovery makes things worse: our own + # testusb is D-state on this DUT's node, so a flasher that enumerates by + # OPENING usbfs nodes blocks on it, survives SIGKILL and is abandoned -- + # a SECOND stray, the budget spent, the device still wedged. On 2026-08-12 + # a vid_pid-pinned openocd was the only flasher that still reached its + # probe; JLinkExe's ShowEmuList returned zero. See hil_flash.convoy_safe. + if not hil_flash.convoy_safe(board['flasher']): + print(f'{fname} is not convoy-safe for delivery (it enumerates by ' + f'opening usbfs nodes, and this DUT has a D-state holder on ' + f'its own node): skipping the reflash rather than adding a ' + f'second stray. Pin the roster entry with vid_pid on an ' + f'openocd flasher to enable recovery for this board.', + file=sys.stderr) + break + # RESET FIRST (see recovery_steps). Non-destructive, ~130 ms, and it + # clears the wedge by the same mechanism as the reflash. wedged_pids is the + # arbiter: reset_esptool is a stub that returns rc 0 without resetting + # anything, so an exit code here proves nothing. + steps = recovery_steps(fname, _time_left()) + if reset_fn and any(k == 'reset' for k, _ in steps): + print(f'auto-recovering: resetting {bname} via {fname} probe ' + f'(non-destructive; reflash only if this does not clear it)', + file=sys.stderr) + try: + with redirect_stdout(sys.stderr): + reset_fn(board, timeout=RECOVER_RESET_TIMEOUT) + except TypeError: + with redirect_stdout(sys.stderr): + reset_fn(board) # older primitives take no bound + except Exception as e: + print(f'probe reset raised: {e}; falling through to the reflash', + file=sys.stderr) + time.sleep(5) # let the freed ioctl unwind + stuck, complete = wedged_pids(dev['node']) + if complete and not stuck: + print('probe reset cleared the wedge; skipping the reflash ' + '(firmware under test left intact for autopsy)', + file=sys.stderr) + unrecovered_hang = False + break + if _time_left() < RECOVER_FLASH_TIMEOUT: + print('reset did not clear it and no budget left for a reflash; ' + 'the device stays wedged', file=sys.stderr) + break + print(f'auto-recovering: reflashing {bname} via ' + f'{fname} (see .claude/skills/usb-kernel-recover). ' + f'Unbudgeted by flash_permit, like the root-cycle it replaced: the ' + f'per-controller semaphores live in hil_test\'s process.', + file=sys.stderr) + # run_cmd bounds the flash; its banners go to stdout, which in --json mode + # carries the result object -- keep them off it. A raising flasher (missing + # serial node, unwritable CWD) must not cost the battery its JSON report. + try: + with redirect_stdout(sys.stderr): + ret = flash_fn(board, args.recover_fw, timeout=RECOVER_FLASH_TIMEOUT) + except Exception as e: + print(f'reflash raised: {e}; the device may still be wedged', file=sys.stderr) + break + if ret.returncode != 0: + # a wedged RP DAP answers nothing and the probe has no reset line; + # POR it via the Rescue DP and retry once, exactly as the normal + # flash path does (no-op for every other board/failure) + out_txt = ret.stdout if isinstance(ret.stdout, str) else '' + # inside the redirect like its siblings (hil_test slices the result + # object from the first '{' on stdout), and only if a POR + retry + # still fits before the outer kill + rescued = False + try: + if _time_left() >= 2 * RECOVER_FLASH_TIMEOUT: + with redirect_stdout(sys.stderr): + rescued = hil_flash.rescue_openocd( + board, out_txt, timeout=RECOVER_FLASH_TIMEOUT) + if rescued: + print('DAP wedged; rescued via Rescue DP, retrying reflash', + file=sys.stderr) + with redirect_stdout(sys.stderr): + ret = flash_fn(board, args.recover_fw, + timeout=RECOVER_FLASH_TIMEOUT) + except Exception as e: + # guarded like the first flash: a raise here would unwind past the + # BUDGET back-fill and the JSON print + print(f'rescue/retry raised: {e}', file=sys.stderr) + if ret.returncode != 0: + print(f'reflash failed (rc {ret.returncode}); the device may still ' + f'be wedged', file=sys.stderr) + # settle even on a non-zero exit: the reset may have landed before the + # flasher failed, and the freed ioctl needs a moment to unwind before + # wedged_pids samples + time.sleep(5) + # Authoritative either way: a clean flash only proves the probe wrote the + # MCU, not that the D-state holder let go. + stuck, complete = wedged_pids(dev['node']) + if stuck: + print(f'{dev["sysname"]}: pid(s) {stuck} still in D state on ' + f'{dev["node"]} — the device lock was never released', file=sys.stderr) + # No hub-worker verdict here: our own testusb still holds the DUT's + # device lock, which is what drives a hub worker into usb_lock_device() + # -- any verdict from here is confounded by construction. + elif not complete: + print('cannot confirm recovery: /proc is only partly readable, so a ' + 'hidden D-state holder cannot be ruled out', file=sys.stderr) + else: + unrecovered_hang = False + break + # re-resolve: a mid-battery re-enumeration changes the devnum and so the node + # path. Match on the concrete serial (not args.serial, which may be None) so + # this can never retarget to another device sharing the VID:PID. + # first=False: the ambiguity guard exists because ONE serial can match two + # sysfs nodes on the dual-port WCH parts, and `dev = live` below makes any + # mistake stick for the rest of the battery -- including wedged_pids() then + # scanning the wrong node and clearing unrecovered_hang on a device it never + # checked. Ambiguous comes back as {'ambiguous': [...]}, handled below. + live = find_device(dev['serial']) + if live and live.get('ambiguous'): + # two nodes now answer to one serial (the dual-port WCH parts do this + # around a re-enumeration). Picking either would file the rest of the + # battery's verdicts under a device we cannot identify, so stop here and + # keep the recovery in play rather than guess. + abort_reason = (f'serial {dev["serial"]} matches more than one device ' + f'({", ".join(live["ambiguous"])}) after case {num}') + unrecovered_hang = True + break + if live and live.get('inconclusive'): + # bounded reads stopped answering, so we cannot say the device left -- + # treat it as the wedge it probably is, which keeps the HUNG reflash and + # the lock-safe cleanup in play + from helper import hil_util as _hu + abort_reason = ('cannot tell whether the device is still present: bounded ' + 'sysfs reads stopped answering' + _hu.sysfs_blind_note()) + unrecovered_hang = True + break + if not live: + # no second entry for `num`: run_case already recorded it, and a duplicate + # inflates the denominator (31/30) and reports a PASSing case as failed + abort_reason = f'device dropped off the bus after case {num}' + break + dev = live + if abort_reason and all(c in {r['num'] for r in results} for c in cases) \ + and 'dropped off the bus' in abort_reason and results: + # nothing left to back-fill (the drop happened during/after the LAST case), + # so the run would report a clean pass; the case it died on is not a pass + if results[-1].get('status') == 'PASS': + # only a PASS: a real FAIL/NOTRUN verdict names the actual regression + # (errno, dmesg) and must not be overwritten by the drop message + results[-1] = dict(results[-1], status='FAIL', detail=abort_reason) + if abort_reason: + # One BUDGET entry per case never dispatched, on EVERY abort path: a shrunken + # denominator (4/5 instead of 4/30) hides that most of the battery never + # executed and makes a regression in the skipped range read as "not the + # problem". + ran = {r['num'] for r in results} + results += [{'num': n, 'status': 'BUDGET', 'detail': f'not run: {abort_reason}'} + for n in cases if n not in ran] + finally: + # best-effort cleanup: a sudo/sysfs failure here (sudo() may sys.exit) must not replace + # an exception propagating out of the try body with a less useful one + try: + if unrecovered_hang: + # testusb still holds the device lock in a usbfs ioctl: remove_id/unbind + # would join the convoy and deadlock the bus (see usb-kernel-recover) + print('skipping cleanup after unrecovered hang: ask the operator for a full PVE host ' + 'power cycle (a VM reboot is not reliable — hubs latch up across the PCIe reset)', + file=sys.stderr) + elif not args.keep_binding: + sysfs_write(DRIVER / 'remove_id', f'{VID} {PID}', check=False) + # release every claimed interface: another device sharing the VID:PID + # (stale example firmware) may have been grabbed on probe and would stay + # bound to usbtest until re-plugged, hijacking the next test's device + for intf in DRIVER.glob('*:*'): + sysfs_write(DRIVER / 'unbind', intf.name, check=False) + except SystemExit: + pass + + # BUDGET, not NOTRUN: NOTRUN is taken, for a case the KERNEL gated off (-EOPNOTSUPP, + # see run_case) -- a real result that must stay in `failed` and keep its case number. + # BUDGET keeps the denominator honest without lying about the numerator: naming cases + # that never executed as failures sends a maintainer bisecting one of them. + notrun = [r for r in results if r['status'] == 'BUDGET'] + failed = [r for r in results if r['status'] not in ('PASS', 'BUDGET')] + ran = len(results) + if args.json: + # `wedged` is the verdict this process ALREADY computed; without it the caller had + # to infer one from 'HUNG' in our stdout, which misses a recovery that ran and + # failed, the inconclusive abort (no case reaches status HUNG), and any battery + # killed before it printed. + print(json.dumps({'serial': dev['serial'], 'speed': dev['speed'], 'tier': tier, + 'passed': ran - len(failed) - len(notrun), + 'failed': len(failed), 'notrun': len(notrun), + 'wedged': bool(unrecovered_hang), + 'cases': results}, indent=2)) + else: + print(f"{ran - len(failed) - len(notrun)}/{ran} passed" + + (f", {len(notrun)} not run" if notrun else "")) + for r in failed: + print(f" FAILED test {r['num']}: {r.get('detail', '')}") + if r.get('dmesg'): + print(' ' + r['dmesg'].replace('\n', '\n ')) + # NOTRUN counts toward the exit status even though it is reported separately: a + # standalone run whose cases were all skipped has NOT passed, and returning 0 hands a + # false success to any script driving this directly. + return len(failed) + len(notrun) + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/test/unit-test/test/device/usbd/test_usbd.c b/test/unit-test/test/device/usbd/test_usbd.c index 7f3c3f5b2..849097326 100644 --- a/test/unit-test/test/device/usbd/test_usbd.c +++ b/test/unit-test/test/device/usbd/test_usbd.c @@ -29,6 +29,7 @@ #include "tusb_fifo.h" #include "tusb.h" #include "usbd.h" +#include "device/usbd_pvt.h" TEST_SOURCE_FILE("usbd.c") // Mock File @@ -271,6 +272,90 @@ void test_usbd_control_in_zlp(void) } //--------------------------------------------------------------------+ +// SETUP dropped by full event queue +//--------------------------------------------------------------------+ + +// When the event queue is full, queue_event() drops the SETUP event. The queued-setup +// counter must not keep the dropped SETUP's increment: a leaked count makes the handler +// skip every later SETUP ("other SETUP in queue") forever, leaving EP0 permanently deaf. +void test_usbd_setup_dropped_by_full_queue_recovers(void) +{ + // fillers drain through usbd_reset -> class reset + mscd_reset_Ignore(); + + // fill the queue to the brim, then post one more SETUP: queue_event() drops it + for (unsigned i = 0; i < CFG_TUD_TASK_QUEUE_SZ; i++) { + dcd_event_bus_signal(rhport, DCD_EVENT_UNPLUGGED, false); + } + dcd_event_setup_received(rhport, (uint8_t*) &req_get_desc_device, false); + + // drain all fillers (each tud_task pass handles at most CFG_TUD_TASK_EVENTS_PER_RUN + // events); the dropped SETUP never arrives + for (unsigned i = 0; i < (CFG_TUD_TASK_QUEUE_SZ / CFG_TUD_TASK_EVENTS_PER_RUN) + 1; i++) { + tud_task(); + } + + // the next SETUP must still be answered + desc_device = (uint8_t const*) &data_desc_device; + dcd_event_setup_received(rhport, (uint8_t*) &req_get_desc_device, false); + + dcd_edpt_xfer_ExpectWithArrayAndReturn(rhport, 0x80, (uint8_t*) &data_desc_device, sizeof(tusb_desc_device_t), sizeof(tusb_desc_device_t), false, true); + dcd_event_xfer_complete(rhport, EDPT_CTRL_IN, sizeof(tusb_desc_device_t), 0, false); + + dcd_edpt_xfer_ExpectAndReturn(rhport, EDPT_CTRL_OUT, NULL, 0, false, true); + dcd_event_xfer_complete(rhport, EDPT_CTRL_OUT, 0, 0, false); + dcd_edpt0_status_complete_ExpectWithArray(rhport, &req_get_desc_device, 1); + + tud_task(); +} + +//--------------------------------------------------------------------+ +// Transfer completion dropped by full event queue +//--------------------------------------------------------------------+ + +// When the event queue is full, queue_event() drops the XFER_COMPLETE event. The endpoint's +// busy/claimed state must not survive the dropped completion: a leaked BUSY makes every later +// usbd_edpt_claim()/usbd_edpt_xfer() on that endpoint fail, so the class never re-arms it. +void test_usbd_xfer_complete_dropped_by_full_queue_recovers(void) +{ + // fillers drain through usbd_reset -> class reset + mscd_reset_Ignore(); + + // open + claim + arm a bulk OUT endpoint the way a class driver would + tusb_desc_endpoint_t desc_ep = { + .bLength = sizeof(tusb_desc_endpoint_t), + .bDescriptorType = TUSB_DESC_ENDPOINT, + .bEndpointAddress = 0x01, + .bmAttributes = { .xfer = TUSB_XFER_BULK }, + .wMaxPacketSize = 64, + .bInterval = 0 + }; + static uint8_t xfer_buf[64]; + + dcd_edpt_open_ExpectAndReturn(rhport, &desc_ep, true); + TEST_ASSERT_TRUE(usbd_edpt_open(rhport, &desc_ep)); + TEST_ASSERT_TRUE(usbd_edpt_claim(rhport, 0x01)); + dcd_edpt_xfer_ExpectAndReturn(rhport, 0x01, xfer_buf, 64, false, true); + TEST_ASSERT_TRUE(usbd_edpt_xfer(rhport, 0x01, xfer_buf, 64, false)); + + // fill the queue to the brim, then complete the transfer: queue_event() drops it + for (unsigned i = 0; i < CFG_TUD_TASK_QUEUE_SZ; i++) { + dcd_event_bus_signal(rhport, DCD_EVENT_UNPLUGGED, false); + } + dcd_event_xfer_complete(rhport, 0x01, 64, XFER_RESULT_SUCCESS, false); + + // the endpoint must be re-armable: the dropped completion must not leak busy/claimed + TEST_ASSERT_TRUE(usbd_edpt_claim(rhport, 0x01)); + dcd_edpt_xfer_ExpectAndReturn(rhport, 0x01, xfer_buf, 64, false, true); + TEST_ASSERT_TRUE(usbd_edpt_xfer(rhport, 0x01, xfer_buf, 64, false)); + + // drain the fillers so later tests start from an empty queue + for (unsigned i = 0; i < (CFG_TUD_TASK_QUEUE_SZ / CFG_TUD_TASK_EVENTS_PER_RUN) + 1; i++) { + tud_task(); + } +} + +//--------------------------------------------------------------------+ // Control OUT data stage host overrun //--------------------------------------------------------------------+ diff --git a/tools/88-tinyusb.rules b/tools/88-tinyusb.rules new file mode 100644 index 000000000..e0c5215b0 --- /dev/null +++ b/tools/88-tinyusb.rules @@ -0,0 +1,93 @@ +# Copy this file to the location of your distribution's udev rules: +# Then reload udev configuration by executing: +# sudo cp 88-tinyusb.rules /etc/udev/rules.d/ && sudo udevadm control --reload-rules && sudo udevadm trigger + +# Check SUBSYSTEM +SUBSYSTEMS=="hidraw", KERNEL=="hidraw*", MODE="0666", GROUP="dialout" +SUBSYSTEM=="usbmon", MODE="0640", GROUP="wireshark" + +# Rule applies to all TinyUSB example +ATTRS{idVendor}=="cafe", MODE="0666", GROUP="dialout" + +# Rule to make Trinket/Pro Trinket/Gemma/Flora programmable without running Arduino as root. +# Tested with Ubuntu 14.04 and 12.04. Other distributions might need to update GROUP="dialout" +# to another group value like "users". +SUBSYSTEM=="usb", ATTRS{idProduct}=="0c9f", ATTRS{idVendor}=="1781", MODE="0660", GROUP="dialout" + +# Rule to blacklist Adafruit USB CDC boards from being manipulated by ModemManager. +# Fixes issue with hanging references to /dev/ttyACM* devices on Ubuntu 15.04. +ATTRS{idVendor}=="239a", ENV{ID_MM_DEVICE_IGNORE}="1" + +# All Adafruit boards +ATTRS{idVendor}=="239a", MODE="0660", GROUP="adm" + +# All Espressif boards +ATTRS{idVendor}=="303a", MODE="0660", GROUP="adm" + +# All RaspberryPi boards +ATTRS{idVendor}=="2e8a", MODE="0660", GROUP="adm" + +# All NXP Boards +ATTRS{idVendor}=="1fc9", MODE="0660", GROUP="adm" + +# All ST +SUBSYSTEM=="usb", ATTRS{idVendor}=="0483", GROUP="adm" + +# Rule to blacklist TinyUSB example from being manipulated by ModemManager. +SUBSYSTEMS=="usb", ATTRS{idVendor}=="cafe", ENV{ID_MM_DEVICE_IGNORE}="1" + +# Xplained Pro SamG55 Device +SUBSYSTEMS=="usb", ATTRS{idVendor}=="03eb", ATTRS{idProduct}=="2111", MODE="0666", GROUP="users", ENV{ID_MM_DEVICE_IGNORE}="1" +SUBSYSTEMS=="tty", ATTRS{idVendor}=="03eb", ATTRS{idProduct}=="2111", MODE="0666", GROUP="users", ENV{ID_MM_DEVICE_IGNORE}="1" + +# TI Stellaris/Tiva-C Launchpad ICDI +SUBSYSTEM=="usb", ATTRS{idVendor}=="1cbe", ATTRS{idProduct}=="00fd", MODE="0666" + +# CMSIS-DAP, vendor = ARM +SUBSYSTEM=="usb", ATTR{idVendor}=="0d28", MODE="0666" + +# wch-link +SUBSYSTEM=="usb", ATTR{idVendor}=="1a86", ATTR{idProduct}=="8010", GROUP="plugdev" +SUBSYSTEM=="usb", ATTR{idVendor}=="4348", ATTR{idProduct}=="55e0", GROUP="plugdev" +SUBSYSTEM=="usb", ATTR{idVendor}=="1a86", ATTR{idProduct}=="8012", GROUP="plugdev" + +# Pxlogic +SUBSYSTEM=="usb", ATTRS{idVendor}=="2a0e", MODE="0666" +SUBSYSTEM=="usb", ATTRS{idVendor}=="1a86", MODE="0666" + +# Arduino Renesas +SUBSYSTEMS=="usb", ATTRS{idVendor}=="2341", MODE="0666" + +# E2/E2 Lite/E1/E20/IE850A emulator +ATTR{idProduct}=="82a1", ATTR{idVendor}=="045b", MODE="0666" +ATTR{idProduct}=="82a0", ATTR{idVendor}=="045b", MODE="0666" +ATTR{idProduct}=="823b", ATTR{idVendor}=="045b", MODE="0666" +ATTR{idProduct}=="823c", ATTR{idVendor}=="045b", MODE="0666" +ATTR{idProduct}=="0250", ATTR{idVendor}=="045b", MODE="0666" +# Prevent E2/E2Lite/E1/E20/IE850A from being captured by modem manager service as E2/E2 Lite/E1/E20/IE850A is not a modem +ATTR{idProduct}=="82a1", ATTR{idVendor}=="045b", ENV{ID_MM_DEVICE_IGNORE}="1" +ATTR{idProduct}=="82a0", ATTR{idVendor}=="045b", ENV{ID_MM_DEVICE_IGNORE}="1" +ATTR{idProduct}=="823b", ATTR{idVendor}=="045b", ENV{ID_MM_DEVICE_IGNORE}="1" +ATTR{idProduct}=="823c", ATTR{idVendor}=="045b", ENV{ID_MM_DEVICE_IGNORE}="1" +ATTR{idProduct}=="0250", ATTR{idVendor}=="045b", ENV{ID_MM_DEVICE_IGNORE}="1" + +#TI MSP430UIF +ATTRS{idVendor}=="2047",ATTRS{idProduct}=="0010",MODE="0666" +ATTRS{idVendor}=="2047",ATTRS{idProduct}=="0013",MODE="0666" +ATTRS{idVendor}=="2047",ATTRS{idProduct}=="0014",MODE="0666" +ATTRS{idVendor}=="2047",ATTRS{idProduct}=="0203",MODE="0666" +ATTRS{idVendor}=="2047",ATTRS{idProduct}=="0204",MODE="0666" +ATTRS{idVendor}=="0451",ATTRS{idProduct}=="f432",MODE="0666" + +# fomu +ATTRS{idVendor}=="1209",ATTRS{idProduct}=="5bf0",MODE="0666" + +# FTDI +ATTRS{idVendor}=="0403", MODE="0660", GROUP="adm" + +# Sipeed Slogic16 +SUBSYSTEM=="usb", ATTRS{idVendor}=="359f", MODE="0666", TAG+="uaccess", ENV{ID_MM_DEVICE_IGNORE}="1" + +# ataradov usb-sniffer (github.com/ataradov/usb-sniffer): programmed unit + blank FX2LP +ATTRS{idVendor}=="6666", ATTRS{idProduct}=="6620", MODE="0666" +ATTRS{idVendor}=="04b4", ATTRS{idProduct}=="8613", MODE="0666" diff --git a/tools/build.py b/tools/build.py index 5eaaeb513..51d3d0f70 100755 --- a/tools/build.py +++ b/tools/build.py @@ -92,6 +92,7 @@ def get_examples(family): if family == 'espressif': all_examples.append('device/board_test') + all_examples.append('device/usbtest') all_examples.append('device/video_capture') all_examples.append('host/device_info') all_examples.sort() diff --git a/tools/check_example_pids.py b/tools/check_example_pids.py new file mode 100644 index 000000000..d8795af34 --- /dev/null +++ b/tools/check_example_pids.py @@ -0,0 +1,55 @@ +#!/usr/bin/env python3 +"""Check that every example enumerates with a unique USB PID. + +Each example's usb_descriptors.c hardcodes its idProduct (0x40xx). Uniqueness is what +guarantees back-to-back re-enumeration on the HIL rig and a fresh host driver match, and +it is easy to break by hand: a new example copying a neighbour's PID, or an arithmetic +PID (dynamic_configuration derives a second one from USB_PID). This collects every +`#define USB_PID 0x....`, every literal `.idProduct = 0x....`, and every `USB_PID + <n>` +derivation across examples/, and fails on any duplicate value. +""" + +import re +import sys +from pathlib import Path + +EXAMPLES = Path(__file__).resolve().parents[1] / 'examples' + +RE_DEFINE = re.compile(r'#define\s+USB_PID\s+\(?(0x[0-9a-fA-F]+)\)?') +RE_LITERAL = re.compile(r'\.idProduct\s*=\s*(0x[0-9a-fA-F]+)') +RE_DERIVED = re.compile(r'\.idProduct\s*=\s*USB_PID\s*\+\s*(0x[0-9a-fA-F]+|\d+)') + + +def main() -> int: + pids: dict[int, list[str]] = {} + for f in sorted(EXAMPLES.glob('*/*/src/usb_descriptors.c')): + text = f.read_text(errors='replace') + rel = f.relative_to(EXAMPLES.parent) + base = None + m = RE_DEFINE.search(text) + if m: + base = int(m.group(1), 16) + for m in RE_LITERAL.finditer(text): + pids.setdefault(int(m.group(1), 16), []).append(str(rel)) + for m in RE_DERIVED.finditer(text): + if base is None: + print(f'{rel}: derived idProduct but no USB_PID define', file=sys.stderr) + return 1 + pids.setdefault(base + int(m.group(1), 0), []).append(f'{rel} (USB_PID + {m.group(1)})') + # examples whose descriptor uses .idProduct = USB_PID pick up the define itself + if base is not None and re.search(r'\.idProduct\s*=\s*USB_PID\s*[,;]', text): + pids.setdefault(base, []).append(str(rel)) + + dups = {pid: users for pid, users in pids.items() if len(users) > 1} + for pid, users in sorted(dups.items()): + print(f'duplicate USB PID 0x{pid:04x}:', file=sys.stderr) + for u in users: + print(f' {u}', file=sys.stderr) + if dups: + return 1 + print(f'{len(pids)} unique example USB PIDs') + return 0 + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/tools/codespell/ignore-words.txt b/tools/codespell/ignore-words.txt index 0b1aa284a..6b301d27b 100644 --- a/tools/codespell/ignore-words.txt +++ b/tools/codespell/ignore-words.txt @@ -5,6 +5,7 @@ endianess fro hsi inout +linke mot ore pris diff --git a/tools/get_deps.py b/tools/get_deps.py index 9d745ee26..baaf3761f 100755 --- a/tools/get_deps.py +++ b/tools/get_deps.py @@ -46,6 +46,9 @@ deps_optional = { 'hw/mcu/gd/nuclei-sdk': ['https://github.com/Nuclei-Software/nuclei-sdk.git', '7eb7bfa9ea4fbeacfafe1d5f77d5a0e6ed3922e7', 'gd32vf103'], + 'hw/mcu/geehy/APM32F0xx_SDK': ['https://github.com/GeehySemi/APM32F0xx_SDK.git', + 'cfc1fe826e1869133de86d4c6b298fc153e6bc32', + 'apm32f0xx'], 'hw/mcu/infineon/mtb-xmclib-cat3': ['https://github.com/Infineon/mtb-xmclib-cat3.git', 'daf5500d03cba23e68c2f241c30af79cd9d63880', 'xmc4000'], @@ -82,6 +85,12 @@ deps_optional = { 'hw/mcu/nxp/mcux-devices-rt': ['https://github.com/nxp-mcuxpresso/mcux-devices-rt', 'dba2b523c9df61f3330bd186242f8210a8e47c45', 'imxrt'], + 'hw/mcu/puya/PY32F071_Firmware': ['https://github.com/OpenPuya/PY32F071_Firmware.git', + '73e384cddce63e3019de41d9b6af17dfc96fa536', + 'py32f0'], + 'hw/mcu/puya/PY32F072_Firmware': ['https://github.com/OpenPuya/PY32F072_Firmware.git', + 'bc3a6cdbece335a27abb8b51e6fc4911f5116185', + 'py32f0'], 'hw/mcu/raspberry_pi/FreeRTOS-Kernel': ['https://github.com/raspberrypi/FreeRTOS-Kernel.git', '4f7299d6ea746b27a9dd19e87af568e34bd65b15', 'rp2040'], diff --git a/tools/metrics_compare_base.py b/tools/metrics_compare_base.py index 799a96800..844130097 100644 --- a/tools/metrics_compare_base.py +++ b/tools/metrics_compare_base.py @@ -81,7 +81,7 @@ def symlink_deps(main_root, worktree_dir): def ci_first_boards(): """Return the first board (alphabetical) of each arm-gcc CI family.""" - matrix_py = os.path.join(TINYUSB_ROOT, '.github', 'workflows', 'ci_set_matrix.py') + matrix_py = os.path.join(TINYUSB_ROOT, '.github', 'scripts', 'ci_set_matrix.py') if not os.path.isfile(matrix_py): return [] ret = run([sys.executable, matrix_py]) @@ -188,7 +188,7 @@ def main(): args.combined = True ci_boards = ci_first_boards() if not ci_boards: - parser.error('--ci: failed to derive boards from .github/workflows/ci_set_matrix.py') + parser.error('--ci: failed to derive boards from .github/scripts/ci_set_matrix.py') # Append, dedup, preserve order seen = set(args.board) for b in ci_boards: diff --git a/tools/usb_drivers/tinyusb_win_usbser.inf b/tools/usb_drivers/tinyusb_win_usbser.inf index 659f048ae..e3875a478 100644 --- a/tools/usb_drivers/tinyusb_win_usbser.inf +++ b/tools/usb_drivers/tinyusb_win_usbser.inf @@ -88,11 +88,11 @@ ServiceBinary=%12%\%DRIVERFILENAME%.sys [SourceDisksNames] [DeviceList] -%DESCRIPTION%=DriverInstall, USB\VID_CAFE&PID_4001&MI_00, USB\VID_CAFE&PID_4003&MI_00, USB\VID_CAFE&PID_4005&MI_00, USB\VID_CAFE&PID_4007&MI_00, USB\VID_CAFE&PID_4009&MI_00, USB\VID_CAFE&PID_400b&MI_00, USB\VID_CAFE&PID_400d&MI_00, USB\VID_CAFE&PID_400f&MI_00, USB\VID_CAFE&PID_4011&MI_00, USB\VID_CAFE&PID_4013&MI_00, USB\VID_CAFE&PID_4015&MI_00, USB\VID_CAFE&PID_4017&MI_00, USB\VID_CAFE&PID_4019&MI_00, USB\VID_CAFE&PID_401b&MI_00, USB\VID_CAFE&PID_401d&MI_00, USB\VID_CAFE&PID_401f&MI_00, USB\VID_CAFE&PID_4021&MI_00, USB\VID_CAFE&PID_4023&MI_00, USB\VID_CAFE&PID_4025&MI_00, USB\VID_CAFE&PID_4027&MI_00, USB\VID_CAFE&PID_4029&MI_00, USB\VID_CAFE&PID_402b&MI_00, USB\VID_CAFE&PID_402d&MI_00, USB\VID_CAFE&PID_402f&MI_00, USB\VID_CAFE&PID_4031&MI_00, USB\VID_CAFE&PID_4033&MI_00, USB\VID_CAFE&PID_4035&MI_00, USB\VID_CAFE&PID_4037&MI_00, USB\VID_CAFE&PID_4039&MI_00, USB\VID_CAFE&PID_403b&MI_00, USB\VID_CAFE&PID_403d&MI_00, USB\VID_CAFE&PID_403f&MI_00 +%DESCRIPTION%=DriverInstall, USB\VID_CAFE&PID_4001&MI_00, USB\VID_CAFE&PID_4003&MI_00, USB\VID_CAFE&PID_4005&MI_00, USB\VID_CAFE&PID_4007&MI_00, USB\VID_CAFE&PID_4009&MI_00, USB\VID_CAFE&PID_400b&MI_00, USB\VID_CAFE&PID_400d&MI_00, USB\VID_CAFE&PID_400f&MI_00, USB\VID_CAFE&PID_4011&MI_00, USB\VID_CAFE&PID_4013&MI_00, USB\VID_CAFE&PID_4015&MI_00, USB\VID_CAFE&PID_4017&MI_00, USB\VID_CAFE&PID_4019&MI_00, USB\VID_CAFE&PID_401b&MI_00, USB\VID_CAFE&PID_401d&MI_00, USB\VID_CAFE&PID_401f&MI_00, USB\VID_CAFE&PID_4021&MI_00, USB\VID_CAFE&PID_4023&MI_00, USB\VID_CAFE&PID_4025&MI_00, USB\VID_CAFE&PID_4027&MI_00, USB\VID_CAFE&PID_4029&MI_00, USB\VID_CAFE&PID_402b&MI_00, USB\VID_CAFE&PID_402d&MI_00, USB\VID_CAFE&PID_402f&MI_00, USB\VID_CAFE&PID_4031&MI_00, USB\VID_CAFE&PID_4033&MI_00, USB\VID_CAFE&PID_4035&MI_00, USB\VID_CAFE&PID_4037&MI_00, USB\VID_CAFE&PID_4039&MI_00, USB\VID_CAFE&PID_403b&MI_00, USB\VID_CAFE&PID_403d&MI_00, USB\VID_CAFE&PID_403f&MI_00, USB\VID_CAFE&PID_4006&MI_00, USB\VID_CAFE&PID_4008&MI_00, USB\VID_CAFE&PID_400a&MI_00, USB\VID_CAFE&PID_4020&MI_00, USB\VID_CAFE&PID_4022&MI_00 [DeviceList.NTamd64] -%DESCRIPTION%=DriverInstall, USB\VID_CAFE&PID_4001&MI_00, USB\VID_CAFE&PID_4003&MI_00, USB\VID_CAFE&PID_4005&MI_00, USB\VID_CAFE&PID_4007&MI_00, USB\VID_CAFE&PID_4009&MI_00, USB\VID_CAFE&PID_400b&MI_00, USB\VID_CAFE&PID_400d&MI_00, USB\VID_CAFE&PID_400f&MI_00, USB\VID_CAFE&PID_4011&MI_00, USB\VID_CAFE&PID_4013&MI_00, USB\VID_CAFE&PID_4015&MI_00, USB\VID_CAFE&PID_4017&MI_00, USB\VID_CAFE&PID_4019&MI_00, USB\VID_CAFE&PID_401b&MI_00, USB\VID_CAFE&PID_401d&MI_00, USB\VID_CAFE&PID_401f&MI_00, USB\VID_CAFE&PID_4021&MI_00, USB\VID_CAFE&PID_4023&MI_00, USB\VID_CAFE&PID_4025&MI_00, USB\VID_CAFE&PID_4027&MI_00, USB\VID_CAFE&PID_4029&MI_00, USB\VID_CAFE&PID_402b&MI_00, USB\VID_CAFE&PID_402d&MI_00, USB\VID_CAFE&PID_402f&MI_00, USB\VID_CAFE&PID_4031&MI_00, USB\VID_CAFE&PID_4033&MI_00, USB\VID_CAFE&PID_4035&MI_00, USB\VID_CAFE&PID_4037&MI_00, USB\VID_CAFE&PID_4039&MI_00, USB\VID_CAFE&PID_403b&MI_00, USB\VID_CAFE&PID_403d&MI_00, USB\VID_CAFE&PID_403f&MI_00 +%DESCRIPTION%=DriverInstall, USB\VID_CAFE&PID_4001&MI_00, USB\VID_CAFE&PID_4003&MI_00, USB\VID_CAFE&PID_4005&MI_00, USB\VID_CAFE&PID_4007&MI_00, USB\VID_CAFE&PID_4009&MI_00, USB\VID_CAFE&PID_400b&MI_00, USB\VID_CAFE&PID_400d&MI_00, USB\VID_CAFE&PID_400f&MI_00, USB\VID_CAFE&PID_4011&MI_00, USB\VID_CAFE&PID_4013&MI_00, USB\VID_CAFE&PID_4015&MI_00, USB\VID_CAFE&PID_4017&MI_00, USB\VID_CAFE&PID_4019&MI_00, USB\VID_CAFE&PID_401b&MI_00, USB\VID_CAFE&PID_401d&MI_00, USB\VID_CAFE&PID_401f&MI_00, USB\VID_CAFE&PID_4021&MI_00, USB\VID_CAFE&PID_4023&MI_00, USB\VID_CAFE&PID_4025&MI_00, USB\VID_CAFE&PID_4027&MI_00, USB\VID_CAFE&PID_4029&MI_00, USB\VID_CAFE&PID_402b&MI_00, USB\VID_CAFE&PID_402d&MI_00, USB\VID_CAFE&PID_402f&MI_00, USB\VID_CAFE&PID_4031&MI_00, USB\VID_CAFE&PID_4033&MI_00, USB\VID_CAFE&PID_4035&MI_00, USB\VID_CAFE&PID_4037&MI_00, USB\VID_CAFE&PID_4039&MI_00, USB\VID_CAFE&PID_403b&MI_00, USB\VID_CAFE&PID_403d&MI_00, USB\VID_CAFE&PID_403f&MI_00, USB\VID_CAFE&PID_4006&MI_00, USB\VID_CAFE&PID_4008&MI_00, USB\VID_CAFE&PID_400a&MI_00, USB\VID_CAFE&PID_4020&MI_00, USB\VID_CAFE&PID_4022&MI_00 ;------------------------------------------------------------------------------ ; String Definitions |