test/hil: usbtest fleet enablement, shuffled scheduling, unique PIDs

Pool/config:
- record real uids (ra8m1_ek), enable usbtest for espressif s3/p4, then
  park ra6m5_ek and ra8m1_ek in boards-skip (ra6m5's usbtest/MSC traffic
  can kill the uPD720201 host on its ROM firmware; ra8m1 USBHS bring-up
  pending); max32666/nrf54lm20 stay enabled - their MosChip flakiness
  never wedges
- re-enable device/usbtest on HS boards (mimxrt1064, ch32v307) now that
  uPD720201 firmware 2.0.2.6 fixes the command-ring death; mimxrt1015
  stays skipped - its HS battery killed the controller on both ROM and
  2.0.2.6 firmware (board-specific); match the moved host-test bundles
  (f723 <-> rt1064); skip never-passing tests on the new
  nrf5340dk/nrf54lm20dk boards and the detached pico host bundle, each
  documented with a comment

Host-controller quirk gating in usbtest.py (auto-skip, self-heals on a
healthy xHCI):
- MosChip MCS9990 EHCI: case 25 (int-OUT never scheduled, FRINDEX bug)
  and case 11 (unlinked reads complete short/EREMOTEIO)
- Renesas uPD720201 xHCI: firmware-gated. The card must run firmware
  >= 2.0.2.6 (RAM-uploaded - it reverts to ROM on every power cycle):
  on older firmware the command ring dies under unlink stress (a
  Configure Endpoint command stops completing; the hub worker deadlocks
  holding the device lock; only a host power cycle recovers; three
  boards reproduced it). usbtest.py reads the FW version register (PCI
  config 0x6c) and refuses to run at all on older firmware - hil_test
  surfaces that as a failed test with the reason. On current firmware
  the full 30-case battery runs (validated FS+HS: metro_m4, f723,
  f723-DMA all 30/30).

Scheduling (hil_test.py):
- Shuffle each (board, variant)'s test order with a seeded RNG
  (HIL_SHUFFLE_SEED to replay) so usbtest batteries and flash churn spread
  across the timeline instead of convoying on one controller.
- Per-controller usbtest + flash semaphores: HIL_USBTEST_PARALLEL
  (default 4) concurrent usbtest batteries and HIL_FLASH_PARALLEL
  (default 8) concurrent flashes per host controller. Profiled on
  uPD720201 firmware 2.0.2.6 across 8/1..12/8: wall time falls
  22.2/14.3/12.5/10.8 min at usbtest width 1/2/3/4 and plateaus there;
  zero controller errors everywhere; first battery case failures
  (leaf-hub bandwidth stretch) appear at 12/8, and flash width 12 only
  amplifies flasher-hub contention flakes - so 8/4 is the optimum. A
  separate battery-window flash throttle was profiled and dropped.
- Give every example a unique hardcoded USB PID (0x4001-0x4022, usbtest
  keeps 0x4010) instead of the PID_MAP interface bitmap: different
  examples now always re-enumerate back-to-back, even on boards whose
  CPU reset does not drop D+ (WCH CH58x), so the EXAMPLE_PID table and
  same-PID adjacency reordering in hil_test.py are gone; only the
  variant-boundary same-example repeat needs a swap.
- Report matrix: stable columns with the metric-bearing tests pinned
  first (usbtest, cdc_msc_throughput, msc_file_explorer[_freertos]),
  the rest alphabetical.

Fail fast:
- enum wait budget 8 s on the first attempt, 4 s on retries; dfu waits
  are deadline-based so dfu-util's own runtime counts against the
  budget.
  A device-absent failure now costs ~3-5x a passing test (20-30 s)
  instead of 10-30x (47-150 s).
- CI runs hil_test with --retry 1 and no in-run second pass: a broken
  fixture fails the job fast instead of holding the self-hosted runner
  for hours and blocking other PRs' HIL jobs. hil_test still writes the
  .skip sidecar, so a manual re-run attempt only retests what failed.

Review fixes (multi-agent adversarial review of this commit):
- tinyusb_win_usbser.inf: the PID rework moved five CDC examples onto
  even PIDs the INF's odd-only DeviceList never matched (legacy-Windows
  usbser binding) - appended 0x4006/4008/400a/4020/4022 to both lists.
- usbtest example: USBTEST_TIER is now overridable and the descriptors
  and pumps are tier-conditional, so a board whose DCD cannot serve a
  tier lowers it instead of skipping the whole example - RA2A1 (RUSB2
  with no isochronous pipe) builds at tier 3 via its BOARD_ define; the
  host battery follows the tier advertised in bcdDevice. Tier-4 output
  verified byte-identical after the refactor.
- dynamic_configuration's second config derived USB_PID + 11 = 0x4018,
  colliding with net_lwip_webserver - now USB_PID + 0x0100, outside the
  per-example space. tools/check_example_pids.py (pre-commit hook)
  enforces PID uniqueness incl. derived and literal idProduct values.
- usbtest.py firmware gate: matched by device ID (uPD720201/720202,
  both use the 0x6c FW register), and an unreadable version (setpci
  missing/denied) now refuses with its own message instead of
  masquerading as "firmware 0x00000000"; noted the gate is necessary
  but not sufficient (board-specific kills stay per-board skips).
- hil_test: deadline waits use time.monotonic(); multiprocessing
  context pinned to fork (raw semaphores in Pool initargs); flash and
  usbtest permits unified into one fail-closed, exception-safe
  ctrl_permit (unknown controller takes every slot and logs a warning
  instead of silently borrowing slot 0); an all-skipped battery
  reports as skip, not "0/0" failure; slow-body polls (mtp, printer,
  disk read) go through a shared deadline-based wait_until so their
  bodies count against the enum budget; throughput's FS detection
  compares serials case-insensitively like every other walk; a missing
  MSC read-speed line now fails the host msc_file_explorer test
  instead of passing with an empty metric.

Hardening:
- fail fast (15 s) when a driver-registry sysfs write blocks: a wedged
  device otherwise turns every subsequent battery into an unkillable
  D-state writer and silently hangs the whole run
- usb-recover skill: a VM reboot is not a reliable cure (MosChip hubs
  latch up across the PCIe reset); full host power cycle is

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HeF2gZ1M7GWkz6Av4BpKPg

1 files changed, 69 insertions, 14 deletions

diff --git a/test/hil/tinyusb.json b/test/hil/tinyusb.json
index be39c9eb5..8f121b6f8 100644
--- a/test/hil/tinyusb.json
+++ b/test/hil/tinyusb.json
@@ -22,6 +22,7 @@
                 { "name": "espressif_p4_function_ev-DMA", "flags": "-DCFG_TUD_DWC2_DMA_ENABLE=1 -DCFG_TUH_DWC2_DMA_ENABLE=1" }
             ],
             "tests": {
+                "comment": "only IDF/FreeRTOS examples are part of the espressif fleet build; device/usbtest builds under IDF but is not built/flashed by the fleet, so it is not listed",
                 "only": [
                     "device/cdc_msc_freertos",
                     "device/hid_composite_freertos",
@@ -152,6 +153,8 @@
             "name": "mimxrt1015_evk",
             "uid": "DC28F865D2111D228D00B0543A70463C",
             "tests": {
+                "skip": ["device/usbtest"],
+                "comment": "this board's HS battery killed the uPD720201 twice (2026-07-11 on ROM fw, 2026-07-13 case 27 on fw 2.0.2.6 - stop-endpoint timeout, HC died); mimxrt1064/ch32v307 batteries pass, so it is board-specific - keep skipped",
                 "device": true,
                 "host": false,
                 "dual": false
@@ -166,19 +169,20 @@
             "name": "mimxrt1064_evk",
             "uid": "BAE96FB95AFA6DBB8F00005002001200",
             "tests": {
+                "skip": ["host/cdc_msc_hid"],
+                "comment-cdc-echo": "CH9102+Lexar bundle (moved here from stm32f723disco) mounts fine but echo returns nothing - TX-RX loopback jumper likely lost in the move; re-check wiring then re-enable",
                 "device": true,
                 "host": true,
                 "dual": true,
                 "dev_attached": [
                     {
-                        "vid_pid": "10c4_ea60",
-                        "serial": "0001",
-                        "is_cdc": true,
-                        "comment": "cp2102"
+                        "vid_pid": "1a86_55d4",
+                        "serial": "52D2003414",
+                        "is_cdc": true
                     },
                     {
                         "vid_pid": "21c4_0cc7",
-                        "serial": "900058874D871F66",
+                        "serial": "90005893730A1A63",
                         "is_msc": true,
                         "block_size": 512,
                         "block_count": 60620800,
@@ -230,6 +234,8 @@
                 "device": true,
                 "host": true,
                 "dual": true,
+                "skip": ["host/cdc_msc_hid", "host/device_info", "host/msc_file_explorer", "host/msc_file_explorer_freertos", "dual/host_info_to_device_cdc"],
+                "comment-skip": "PIO-USB host port enumerates nothing since the board moves (CH340+UDisk bundle unplugged or unpowered) - re-attach the bundle then drop these skips",
                 "dev_attached": [
                     {
                         "vid_pid": "1a86_7523",
@@ -379,13 +385,14 @@
                 "dual": false,
                 "dev_attached": [
                     {
-                        "vid_pid": "1a86_55d4",
-                        "serial": "52D2003414",
-                        "is_cdc": true
+                        "vid_pid": "10c4_ea60",
+                        "serial": "0001",
+                        "is_cdc": true,
+                        "comment": "cp2102"
                     },
                     {
                         "vid_pid": "21c4_0cc7",
-                        "serial": "90005893730A1A63",
+                        "serial": "900058874D871F66",
                         "is_msc": true,
                         "block_size": 512,
                         "block_count": 60620800,
@@ -512,14 +519,11 @@
                 "uid": "BC5DA47360D0",
                 "args": ""
             }
-        }
-    ],
-    "boards-skip": [
+        },
         {
             "name": "ch582m_evt",
             "uid": "D443627B5450",
             "toolchain": "riscv-gcc",
-            "comment": "unplugged: fixture (board + WCH-Link) failed to re-enumerate after the 2026-07-06 rig reboot; replug to re-enable",
             "tests": {
                 "device": true,
                 "host": false,
@@ -532,18 +536,69 @@
             }
         },
         {
+            "name": "nrf5340dk",
+            "uid": "78E60E166B5F88BE",
+            "tests": {
+                "device": true,
+                "host": false,
+                "dual": false,
+                "skip": ["device/cdc_msc_freertos", "device/audio_test_freertos"],
+                "comment": "board new to HIL: FreeRTOS examples hardfault (UFSR=INVPC) at first task launch on the CM33_NTZ port - pre-existing upstream issue, non-FreeRTOS examples and usbtest pass; fix separately"
+            },
+            "flasher": {
+                "name": "jlink",
+                "uid": "001050076405",
+                "args": "-device NRF5340_XXAA_APP"
+            }
+        },
+        {
             "name": "nrf54lm20dk",
             "uid": "899C3DE5B0F4D5CA",
             "tests": {
                 "device": true,
                 "host": false,
-                "dual": false
+                "dual": false,
+                "skip": ["device/audio_test_freertos"],
+                "comment": "board new to HIL: audio_test_freertos never reaches dcd_init (FreeRTOS itself runs; cdc_msc_freertos and usbtest pass) - example-level issue on nRF54L, fix separately"
             },
             "flasher": {
                 "name": "jlink",
                 "uid": "1051856258",
                 "args": "-device NRF54LM20A_M33"
             }
+        }
+    ],
+    "boards-skip": [
+        {
+            "name": "ra6m5_ek",
+            "uid": "8419032D32363657364EF4622D294B4E",
+            "tests": {
+                "device": true,
+                "host": false,
+                "dual": false,
+                "skip": ["device/cdc_msc_throughput", "device/msc_dual_lun"],
+                "comment": "MSC writes wedge the uPD720201 host (URBs queued, zero wire activity, bus-15 ctrl xfers time out until the device's URBs are killed); reproduced identically with master firmware - device side armed+BUF and exonerated. MSC reads and usbtest bulk (15.8 MB/s) are fine"
+            },
+            "flasher": {
+                "name": "jlink",
+                "uid": "000831915224",
+                "args": "-device R7FA6M5BH"
+            }
+        },
+        {
+            "name": "ra8m1_ek",
+            "uid": "797D142D36345030364E1737922E4B4E",
+            "comment": "USBHS bring-up pending (HS chirp completes digitally but terminations never switch; FS-forced build passed usbtest 30/30). Parked until fixed",
+            "tests": {
+                "device": true,
+                "host": false,
+                "dual": false
+            },
+            "flasher": {
+                "name": "jlink",
+                "uid": "001083115236",
+                "args": "-device R7FA8M1AH"
+            }
         },
         {
             "name": "stm32f769disco",