tinyusb.git/test/hil/hil_test.py, branch master

Key HIL report dir by run id so re-runs and other PRs cannot clobber it

2026-07-17T08:39:10Z

A re-run attempt merged into an empty base: another PR's HIL job ran between attempt 1 and the retry and rewrote the shared hil_report.json, so the run-stamp guard (correctly) refused the foreign base but the full-fleet results were lost - the retry report contained only the re-run cells. Give each (run id, job) its own report dir instead: - attempts of the same run share a dir, so the retry always finds its own sidecar and .failed spec intact - interleaved runs of other PRs/jobs write elsewhere and cannot clobber - the run-stamp mechanism (.failed.run file) becomes redundant and is removed - stale per-run dirs are pruned after 2 weeks

hil: controller-aware scheduling of flash and usbtest concurrency

2026-07-17T04:54:14Z

Full-fleet profiling (HIL_PROFILE=1 instrumentation, included) showed each uPD720201 controller's serialized usbtest battery chain dominates wall time, and a board whose marginal device port bounces during concurrent batteries can wedge or kill the controller ("xHCI host not responding to stop endpoint command"). Every such death traced to mimxrt1015's port (its old "kills the uPD720201" reputation) - it is removed from the config until recabled; mimxrt1064's enum-retry stalls were a loose device cable (re-seated). nrf54lm20dk moves to boards-skip until its failing J-Link probe is replugged. With the hardware fixed both cards run width-4 batteries plus full flash churn clean, so scheduling stays simple: two symmetric knobs, flashes and batteries budgeted per controller. - schedule_boards(): dispatch boards round-robin across host controllers from a persisted hint cache (~/.cache/tinyusb-hil/ctrl_cache.json), learned and merge-on-write refreshed each run (concurrent HIL jobs keep each other's entries). Only the cached PCI address is consumed - dispatch order and first-flash budgeting, never battery serialization (batteries resolve live or fail closed to an all-slot permit). - HIL_FLASH_PARALLEL (8) and HIL_USBTEST_PARALLEL (4) are budgeted per controller via lock slots assigned on first sight. - re-runs: a failed run writes /.failed with the exact re-run spec (--accumulate -b -bt :) instead of the inverted --skip-board list of everything that passed; --skip-board is gone, --flasher/--exclude-flasher scope a config across CI jobs by flasher type (no board names hardcoded in workflows), and -a/--accumulate merges a re-run into the existing report. The spec is stamped with GITHUB_RUN_ID and cleared on fresh runs, so a retry can never consume a spec left behind by a different run's dead or skipped attempt. - CI: esp-idf firmware builds move out of hil-build into hil-build-esp, and the esptool-flashed boards run in their own hil-tinyusb-esp job, so the main hil-tinyusb run starts as soon as the fast toolchains finish instead of waiting on the slow esp-idf build (an esp toolchain flake previously skipped the whole rig run). Artifacts are namespaced per toolchain so the esp job downloads only esp-idf binaries. - HIL_PROFILE=1: timestamped log lines, per-flash durations, permit-wait logging, uid->controller map dump for analysis. - hil_report: per-variant test duration as a dedicated trailing column, recorded only by full runs. Validated on the ci rig (fixed seeds 20260716/777, full fleet at 8/4): 738s/780s walls with only known-flake failures and no controller deaths, vs 1134-1211s serialized-battery baseline.

hil: add usb_recover hub-cycle action; drop MosChip skips, gate it as incompatible

2026-07-15T11:20:04Z

The MosChip MCS9990 card is physically removed from the rig: delete its cases-11/25 SKIP workaround (and the now-orphaned SKIP accounting) from usbtest.py and refuse to run outright if a DUT ever sits behind one again. usb_recover.sh gains `hub-cycle `: uhubctl VBUS cycle of the port feeding the device, walking upstream (parent hub -> root port) until it re-enumerates. Verified on the rig: leaf-level recovery (13-4.4 usbtest device) and full walk to the root port on a dead branch. SKILL.md updated for the action and the two-Renesas topology (root-port ppps is real; leaf 1a40:0201 hubs fake their "ganged" switching). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01WxUeX4Yn26KibfjvDg2pN9

Merge remote-tracking branch 'origin/master' into usbtest

2026-07-14T06:40:45Z

# Conflicts: # .claude/skills/hil/SKILL.md # test/hil/hil_test.py

test/hil: usbtest fleet enablement, shuffled scheduling, unique PIDs

2026-07-14T06:22:48Z

Pool/config: - record real uids (ra8m1_ek), enable usbtest for espressif s3/p4, then park ra6m5_ek and ra8m1_ek in boards-skip (ra6m5's usbtest/MSC traffic can kill the uPD720201 host on its ROM firmware; ra8m1 USBHS bring-up pending); max32666/nrf54lm20 stay enabled - their MosChip flakiness never wedges - re-enable device/usbtest on HS boards (mimxrt1064, ch32v307) now that uPD720201 firmware 2.0.2.6 fixes the command-ring death; mimxrt1015 stays skipped - its HS battery killed the controller on both ROM and 2.0.2.6 firmware (board-specific); match the moved host-test bundles (f723 <-> rt1064); skip never-passing tests on the new nrf5340dk/nrf54lm20dk boards and the detached pico host bundle, each documented with a comment Host-controller quirk gating in usbtest.py (auto-skip, self-heals on a healthy xHCI): - MosChip MCS9990 EHCI: case 25 (int-OUT never scheduled, FRINDEX bug) and case 11 (unlinked reads complete short/EREMOTEIO) - Renesas uPD720201 xHCI: firmware-gated. The card must run firmware >= 2.0.2.6 (RAM-uploaded - it reverts to ROM on every power cycle): on older firmware the command ring dies under unlink stress (a Configure Endpoint command stops completing; the hub worker deadlocks holding the device lock; only a host power cycle recovers; three boards reproduced it). usbtest.py reads the FW version register (PCI config 0x6c) and refuses to run at all on older firmware - hil_test surfaces that as a failed test with the reason. On current firmware the full 30-case battery runs (validated FS+HS: metro_m4, f723, f723-DMA all 30/30). Scheduling (hil_test.py): - Shuffle each (board, variant)'s test order with a seeded RNG (HIL_SHUFFLE_SEED to replay) so usbtest batteries and flash churn spread across the timeline instead of convoying on one controller. - Per-controller usbtest + flash semaphores: HIL_USBTEST_PARALLEL (default 4) concurrent usbtest batteries and HIL_FLASH_PARALLEL (default 8) concurrent flashes per host controller. Profiled on uPD720201 firmware 2.0.2.6 across 8/1..12/8: wall time falls 22.2/14.3/12.5/10.8 min at usbtest width 1/2/3/4 and plateaus there; zero controller errors everywhere; first battery case failures (leaf-hub bandwidth stretch) appear at 12/8, and flash width 12 only amplifies flasher-hub contention flakes - so 8/4 is the optimum. A separate battery-window flash throttle was profiled and dropped. - Give every example a unique hardcoded USB PID (0x4001-0x4022, usbtest keeps 0x4010) instead of the PID_MAP interface bitmap: different examples now always re-enumerate back-to-back, even on boards whose CPU reset does not drop D+ (WCH CH58x), so the EXAMPLE_PID table and same-PID adjacency reordering in hil_test.py are gone; only the variant-boundary same-example repeat needs a swap. - Report matrix: stable columns with the metric-bearing tests pinned first (usbtest, cdc_msc_throughput, msc_file_explorer[_freertos]), the rest alphabetical. Fail fast: - enum wait budget 8 s on the first attempt, 4 s on retries; dfu waits are deadline-based so dfu-util's own runtime counts against the budget. A device-absent failure now costs ~3-5x a passing test (20-30 s) instead of 10-30x (47-150 s). - CI runs hil_test with --retry 1 and no in-run second pass: a broken fixture fails the job fast instead of holding the self-hosted runner for hours and blocking other PRs' HIL jobs. hil_test still writes the .skip sidecar, so a manual re-run attempt only retests what failed. Review fixes (multi-agent adversarial review of this commit): - tinyusb_win_usbser.inf: the PID rework moved five CDC examples onto even PIDs the INF's odd-only DeviceList never matched (legacy-Windows usbser binding) - appended 0x4006/4008/400a/4020/4022 to both lists. - usbtest example: USBTEST_TIER is now overridable and the descriptors and pumps are tier-conditional, so a board whose DCD cannot serve a tier lowers it instead of skipping the whole example - RA2A1 (RUSB2 with no isochronous pipe) builds at tier 3 via its BOARD_ define; the host battery follows the tier advertised in bcdDevice. Tier-4 output verified byte-identical after the refactor. - dynamic_configuration's second config derived USB_PID + 11 = 0x4018, colliding with net_lwip_webserver - now USB_PID + 0x0100, outside the per-example space. tools/check_example_pids.py (pre-commit hook) enforces PID uniqueness incl. derived and literal idProduct values. - usbtest.py firmware gate: matched by device ID (uPD720201/720202, both use the 0x6c FW register), and an unreadable version (setpci missing/denied) now refuses with its own message instead of masquerading as "firmware 0x00000000"; noted the gate is necessary but not sufficient (board-specific kills stay per-board skips). - hil_test: deadline waits use time.monotonic(); multiprocessing context pinned to fork (raw semaphores in Pool initargs); flash and usbtest permits unified into one fail-closed, exception-safe ctrl_permit (unknown controller takes every slot and logs a warning instead of silently borrowing slot 0); an all-skipped battery reports as skip, not "0/0" failure; slow-body polls (mtp, printer, disk read) go through a shared deadline-based wait_until so their bodies count against the enum budget; throughput's FS detection compares serials case-insensitively like every other walk; a missing MSC read-speed line now fails the host msc_file_explorer test instead of passing with an empty metric. Hardening: - fail fast (15 s) when a driver-registry sysfs write blocks: a wedged device otherwise turns every subsequent battery into an unkillable D-state writer and silently hangs the whole run - usb-recover skill: a VM reboot is not a reliable cure (MosChip hubs latch up across the PCIe reset); full host power cycle is Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01HeF2gZ1M7GWkz6Av4BpKPg

Fix max-effort review findings in lock protocol, workflows, and docs

2026-07-13T11:01:07Z

Confirmed by a 10-finder / 28-verifier adversarial review pass: board_lock.py — the flock is now the sole authority: drop cmd_hold's pid-liveness pre-gate (a live hil_test.py pool worker's stale record no longer blocks a genuinely free board); cmd_release probes the flock and only signals a verified holder, refuses to kill hil_test.py holders (CI mid-test), handles PermissionError; the holder daemon truncates its lock records on SIGTERM and keeps the success pipe clear of fds 0-2 (closed-stdio hold used to leave an orphan holder while reporting failure); --config default resolves beside the script. hil_test.py — truncate the lock record on per-board release (pool workers outlive their flocks); warn instead of silently failing open when the lock dir is unusable; error out on -b names absent from the config (was a silent zero-test exit 0, readable as a green HIL run); drop an emptied board row in accumulate_report (variant boards left a blank ghost row). workflows — remove the stray positional arg that made the validate size stage exit 2 on every run; wrap JSON.parse(args) in all six scripts; factor pr-babysit's drifted reply recipe into postReplyRecipe and dedup refutation replies across cycles; validate args.pr and maxCycles; driver-review rejects an empty dimensions list; hil-validate drops a dead guard clause and retries diagnostics with -v -r 1. agents/docs — port-dev scopes git clang-format to its own files (concurrent workers reformatted each other in shared checkouts); hil-operator/hil skill wording matches actual fail-fast output; the implementation plan is now a DO-NOT-EXECUTE historical record (banner + checked boxes) so plan-executing agents cannot revert shipped files. Verified: lock storm 1-winner-in-10, stale-record hold, closed-stdio hold, dead-pid cleanup, CI-holder refusal, ghost-row 4-scenario merge, unknown-board exit 1, py_compile + check.sh on all six workflows, pre-commit clean. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Upj4hta5TNoAbidqeC1zZ6

Fix review findings and add static-analyzer agent

2026-07-10T16:28:09Z

Review-fix batch (owner-confirmed) on the multi-agent harness: - board_lock: detach holder stdio so a captured `hold` cannot hang on the daemon's inherited pipe; probe locks by holder-pid liveness instead of a momentary flock, which could spuriously fail a concurrent acquirer (storm-tested: 1 winner in 10, 0/15 acquire failures under probe storm) - hil_test: locked board renders a visible board-locked fail row so the report matches the exit code; stale marker cleared on a real re-run - pr-babysit: autoPush now opt-in (default dry run); resolve recipe paginates reviewThreads; post-push resolve gets issue-comment fallback - validate: size stage honors non-default base via --base-branch; pvs stage delegated to the new agent - new static-analyzer agent (sonnet): PVS-Studio SAST+MISRA for one board, structured findings gated on files changed vs base Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Rn1AN5DsTdFhRwhugfgKZi

test/hil: usbtest.py runner + HIL integration

2026-07-09T16:38:49Z

Binds the kernel usbtest driver (gadget-zero profile), runs the tier-based battery, auto-recovers kernel-side hangs, and skips cases the host controller cannot run (MosChip MCS9990 EHCI int-OUT). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01HeF2gZ1M7GWkz6Av4BpKPg

feat: Claude Code multi-agent dev/test harness for TinyUSB

2026-07-09T16:34:29Z

Add worker agents (builder, port-dev, driver-reviewer, hil-operator, pr-monitor), deterministic workflows (validate, fanout-dev, driver-review, hil-validate, full-check, pr-babysit) and a /pre-pr gate skill, so sessions can fan build/test/review/PR-triage work out to tiered subagents. pr-babysit drives a PR to green: triage CI + bot reviews, fix validated findings, verify, push, and reply-to + resolve each inline review thread (fixed or refuted). Replace the stop-the-runner HIL discipline with per-board flock locks: test/hil/board_lock.py plus a fail-open guard in hil_test.py let CI and dev sessions share the rig per board (locked boards fail fast and re-run; HIL_NO_BOARD_LOCK=1 is a user-authorized bypass). The actions-runner is never stopped. Design spec, implementation plan, and real-rig smoke evidence under docs/superpowers/. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Rn1AN5DsTdFhRwhugfgKZi

test/hil: show pass/fail/skip counts above the HIL report table

2026-06-24T09:42:04Z

Prefix each rig's hil_report.md matrix with a one-line tally (passed / failed / skipped) so the number of failed tests is visible at a glance in the PR comment without scanning the table. A metric-string cell (e.g. throughput) counts as a pass; blank/not-run cells are excluded. Co-Authored-By: Claude Opus 4.8 (1M context)