tinyusb.git/test, branch update-sponsor

ci_set_matrix: fall open when no selected family builds anywhere

2026-08-21T09:08:25Z

family_list maps a family to the toolchains that build it, and seven hw/bsp families are in neither: cxd56, efm32, espressif, f1c100s, pic32mz, py32f0, same7x. Scoping to one of them intersected to nothing, so every toolchain key was [], every cmake leg skipped on `if: inputs.build-args != '[]'`, code-metrics took its no-metrics branch, and the PR went green from a build job that ran no compiler. The only signal was a stderr line nothing greps for. Not a coverage regression - master gave the same diff no compile coverage either, since none of the other families compiles same7x's board.h. What is new is that the gap used to be masked by the full matrix and is now the whole answer, and that green now means "ran no compiler" rather than "compiled 64 families". A selection whose families ALL miss is now unusable rather than empty: it prints UNSCOPED, which build.yml and .circleci/config.yml already grep to drop the build extras with it, and emits the full matrix. The two neighbouring cases keep their own answers - an explicit families: [] is still a legitimate nothing-selected, and a partial miss still scopes to the families that do build, noting the rest. The contract test pinned an exact count of fall-open markers, which this would have broken; it now pins the invariant (every message that emits the full matrix carries the marker) and was checked to still fail when a marker is removed. Also corrects the drift guard's note about espressif: hil-build-esp builds its boards by name, but that job is gated on repository_owner, so on a fork an espressif-only PR builds nowhere.

hil: express a board's always-on defines as a variant, dropping build.args

2026-08-21T07:23:40Z

The roster had two ways to pass a cmake -D to a board's build: `build.args`, applied to every variant, and `variant[].defines`, applied to one. They did the same thing, and only metro_m4_express used the first - for MAX3421_HOST=1, which is what makes it the one rig board that compiles hcd_max3421.c. A board whose define is always on now carries a single variant named after itself, which is exactly the shape `board.get('variant') or [{'name': name, 'flags': ''}]` already synthesises everywhere - so the build dir, the HIL report row and the variant-boundary handling are unchanged. raspberry_pi_pico has used that shape for its flags all along. Removes the BuildCfg type and the parallel code path from all four consumers: hil_test.build_board, hil_pool_check's two builders, hil_ci_set_matrix and ci_select.board_options. Verified: the hil-build matrix entry is byte-identical (`-b metro_m4_express -DMAX3421_HOST=1`), hil_test's build command is unchanged, ci_select still selects the board for a max3421 diff with MAX3421_HOST in its options, and a real build of dual/host_info_to_device_cdc and host/cdc_msc_hid on that board still compiles hcd_max3421.c.

ci: fix nine ways the selection under-selected or mismatched

2026-08-21T05:41:47Z

Every one of these dropped coverage silently - the worst failure mode here, because the PR still goes green. Found by review, each reproduced first. Selection rules: * class_macros derived the config macro from the class DIRECTORY, so a change to src/class/midi/midi2_device.c selected the midi_test examples (which do not compile it) and never examples/device/midi2_device (the only one that enables CFG_TUD_MIDI2, and the only one that does). The file's own macro is unioned in where it differs - union, never replace: over-selecting costs a build, under-selecting merges a break. * the ${FAMILY_MCUS} fallback added for espressif fired on any family whose _family_mcus came back empty, and _cmake_sets is if()-blind and keeps the FIRST definition - so mcx/frdm_mcxn947 answered MCXA15, a token six examples' skip.txt names, dropping 12 firmware images CMake builds. Limited now to families that never spell set(FAMILY_MCUS ...) at all. * lib_examples read only an example's top-level CMakeLists.txt/Makefile; host/msc_file_explorer_freertos names lib/embedded-cli in src/CMakeLists.txt and survived by luck. The whole example tree is scanned. (SEGGER_RTT and rt-thread still resolve to nothing: all three references sit inside a LOGGER=rtt guard no CI build sets - the documented ruling, not a miss.) * get_family_boards applied ci_skip_boards/ci_preferred_boards only under GITHUB_ACTIONS/CIRCLECI, so the selector answered differently on a laptop than on a runner; _prune_buildable forces CI semantics. Its one-board pick also abandoned the whole preferred list when entry one could not build the -e set, and asked skip_example without the build's -D tokens. * _config_enables and lib_examples still read with the locale encoding - under LC_ALL=C the selector tracebacked on three tracked tusb_config.h files. The whole selector and its suite run clean there now. Workflows: * the Membrowse Upload step omitted $EX_ARGS, but --one-first now picks the board from the -e set, so it configured a different, empty build dir and uploaded --identical for a board never compiled. It takes $EX_ARGS for the BOARD; the target stays the aggregate, which has no DEPENDS and still records every example. * blanking FAM_REGEX reset only build_filtered, leaving the build scoped while code-metrics took the UNSCOPED branch and diffed a 1-family run against the full averaged baseline. All three drop together now, as CircleCI's fall-open does. * CircleCI's EX_ARGS had no character screen and is used unquoted, and its code-metrics job still exit 1'd on an empty metrics set - which a scoped build makes a legitimate outcome. * a `ci-full` PR label now turns the scoping off for one PR. A selector bug under-selects silently, and without a label the only ways back to a full matrix are accidental. Performance, since the selector gates every other job: family.cmake texts are read once rather than per changed directory (a 6,000-file dep bump re-read 84 files 99,892 times) and _scrape_mcu is cached: 2.2s -> 0.29s there, 0.8s -> 0.33s on a class diff. Tests: a drift guard for hw/bsp families absent from ci_set_matrix.family_list (they select zero legs now, where they used to ride the full matrix); the rule-4 port test asserted a SUBSET, which set() satisfies, so it could not fail on the empty selection it exists to catch; the GITHUB_ENV guard test counted a SUM of two guards. Drops metrics.py's --only-examples, which nothing called, and applies the TOTAL scrub to the by-example branch that skipped it.

ci: scope the build matrix and the HIL run to what a PR affects

2026-08-21T04:07:27Z

Every PR built all 74 legs (2494 example builds on GHA cmake alone) and flashed all 30 rig boards, whatever it touched. One classifier now walks the PR diff twice and answers three questions: which families to build, which examples per family, and which boards run which tests. Fail-open throughout - anything no rule classifies, any exception, any unusable output falls back to the full matrix, and a master push always builds everything. test/hil/helper/hil_select.py moves to tools/ci_select.py: it is no longer HIL-only, and tools/ is where the build side can import it. test_hil_select.py follows it as test_ci_select.py. Rules (docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md holds the full table): a port selects the families whose family.cmake references it, and its role - a dcd change skips host examples and vice versa; a class selects only the examples whose tusb_config.h enables its CFG_TU[DH]_ macro, following cross-class includes; an example selects itself; hw/bsp selects its family or board; hw/mcu and lib select whoever references them. CMake is the reference for all of it - make follows whatever cmake decides, family.mk is never scanned. Empty means empty (maintainer ruling): a rule that classifies a path to nothing selects nothing. Ports no family references, classes no config enables, libs no example builds and hw/mcu paths that resolve nowhere are all real - nothing compiles them, so nothing can validate them, and the master-push build is the net. Structural tests pin each such case with an explicit allowlist, so the day one stops being empty it fails pre-commit instead of silently narrowing CI. Per-example builds: build.py grows a repeatable -e, resolved against the targets CMake actually registered and batched into one `cmake --build --target a b c`. build_utils mirrors CMake's family_filter (the whole FAMILY_MCUS list, ${...} and string(TOUPPER ...) resolved) for the cmake side, while the make side keeps master's algorithm verbatim - the two build systems answer differently and a shared answer breaks lpc54's make link. hil-build gains this even on a full selection: 1702 example builds become 515. Transport: the selection travels as a file, never an argv or env var - a mass-sweep diff selects 261 KB against a 128 KiB exec limit, and E2BIG would fail the step before its own fallback could run. CircleCI carries the example map inside the generated config (pipeline parameters cap at 512 chars), swapped into the parameter defaults by sentinel match, and drops the scoping wholesale if that rewrite fails. Every PR-derived value written to $GITHUB_ENV/$GITHUB_OUTPUT is character-screened. Code metrics follow the scoping: metrics.py emits per-example totals, and metrics_pair_compare compares the (board, example) pairs present on both sides instead of a scoped run against a full-matrix average. The selector's own suite gates it in both providers: a selector that exits 0 with valid-but-wrong JSON is the one failure fail-open cannot catch, so a red suite means the full matrix.

hil: address Copilot review — loud extraction markers, exit-visible variant warnings

2026-08-20T11:30:45Z

The workflow-logic harness slices hil-validate.js between marker strings (the body is not a module; the runtime wraps it, so markers are the only handle). A renamed marker used to produce a garbage slice and a confusing ReferenceError; it now fails naming the missing marker, proven by mutating the marker and watching the message. The variant-warning loop in hil_ci.sh read variant_names through a process substitution -- the exact exit-status blindness the comment in resolve_build_dirs warns about, two functions earlier in the same file. A plain command-substitution assignment is visible to set -e, so a malformed roster now aborts instead of silently skipping the warnings.

hil, docs: reference toolchains by their official env vars, not one rig's paths

2026-08-20T10:43:44Z

~/code/pico/pico-sdk and $HOME/code/esp-idf/export.sh are the ci rig's private layout; written into instructions they silently stop being true on tusb, a dev PC, or any future rig. The docs now use the variables the vendors define -- PICO_SDK_PATH for the Pico SDK and IDF_PATH for ESP-IDF, activated explicitly as `. "$IDF_PATH/export.sh"` -- and leave where the checkouts live to each host's profile. The variables are only useful if the shells that agents actually get can see them, and `ssh 'cmd'` is non-interactive AND non-login: it reads no profile, and Debian's sshd-sourced ~/.bashrc returns at the interactive guard before most of the file. The ci rig already keeps its exports in the section ABOVE that early-return; IDF_PATH now sits there beside PICO_SDK_PATH, and the whole chain is verified from a plain non-interactive ssh: both variables visible, `. "$IDF_PATH/export.sh"` activates ESP-IDF v5.5.3 with idf.py on PATH -- no login shell, no alias, no hard-coded path. hil-pool-check documents that placement so the next rig is set up the same way.

hil-pool-check: document the probe power-cycle escalation; name the env script directly

2026-08-20T09:45:32Z

A probe whose firmware has wedged reports flash-failed with the probe present and "probe toggle unconfirmed". The tool's own recovery cannot fix that: an authorized toggle re-enumerates but never removes power, and hil_pool_check.py:304 already notes that ST-Link, WCH-Link, CP210x and picoprobe keep their sysfs kobject across one. So the check correctly gives up, and the operator was left to invent the next rung. Write it down, as what this rig's hardware actually does rather than what uhubctl advertises: the Renesas cards list their root hubs as ppps-capable but do not implement it (owner-confirmed - VBUS never drops, only D+/D-), so a root-port cycle is a harder forced re-enumeration that a wedged probe can ride out, worth exactly one attempt; and the AMD 0000:02:00.0, where the WCH-Links live, has no port-power switching at all - nothing to cycle, straight to a physical replug. Which card a probe hangs off decides which case applies, so the procedure starts from readlink. The ordering rules encode the shared-rig protocol: let the full run finish (a bounce re-enumerates siblings and corrupts checks still in flight), hold --all with this host's --config before the cycle (hil_lock.py hold validates nothing against the roster and nothing maps a sysfs busport to a board name, so a narrower hand-listed hold reserves nothing while reporting success - and --all defaults to tinyusb.json, which on the tusb rig would reserve 27 boards that do not exist there), release BEFORE the re-check (hil_pool_check.py self-locks every board it checks, so a hold still in place makes the verification report locked against your own hold and verify nothing), and drive the cycle through usb_recover.sh root-cycle by its full in-repo path - it is on no PATH and sudo's secure_path excludes the checkout. Never a bare `uhubctl -a cycle`: without -S it writes sysfs disable, whose disable_store takes the root hub's lock uninterruptibly and then usb_disconnect()s the wedged child - the one input that turns a probe wedge into a bus-wide wedge. Give the script the wedged probe's own busport, not the hub path: the serial guard and the success check both read the path you pass, and the hub's inode always changes when its own port cycles. Reporting asks for both passes: a final table showing every board healthy hides that a probe needed power-cycling to get there, which is the signal that it will recur. Also: the ESP-IDF env hints name `. $HOME/code/esp-idf/export.sh` instead of the `get-idf` alias, which lives only in interactive shells and fails from scripts.

hil: run every board in one hil_test.py and hand results across as JSON

2026-08-20T09:43:49Z

hil-validate ran one hil-operator per board. That parallelizes at the wrong layer: hil_test.py already schedules boards across host controllers and budgets concurrent flashes and usbtest batteries per controller (FLASH_PARALLEL/USBTEST_PARALLEL), and those permits live in one process - N parallel runs multiply the budget onto the same uPD720201 cards for no wall-clock gain over one run that already parallelizes. The workflow now spawns ONE operator with every board as repeated -b. The operator no longer retypes the report table. Four consecutive max-effort review rounds found ~15 defects in this file and every one was in reconstructing board identity from transcribed prose: report rows are named per VARIANT (nanoch32v203 only ever produces -fsdev/-usbfs rows), a variant need not start with its board's name, lock contention is a `board-locked` cell rather than a phrase, and each fix introduced the next round's bug - including a fake-green test that asserted an invariant with the one input shape that could not break it. The new helper test/hil/helper/hil_summary.py does the join where the roster lives and emits one machine verdict per board ({board, ran, pass, locked, detail}); the operator returns that JSON verbatim plus `wedged`, the only field it authors, and the workflow reads fields, never parses a string. Its cell classifier mirrors hil_test.py's own tally exactly: failures are always marked ('fail' or a ❌ prefix, TestFail's contract), everything unmarked is a pass - a passing test may return a plain metric cell like '13443 KB/s', and the mirrored rule is what keeps a green table from becoming a red verdict. hil_ci.sh kept only the LAST -b, so multi-board remote runs staged one board's binaries and every other board died on the rig after its lock and flash slot were spent. It now parses every -b spelling argparse accepts (with the -bt arms ordered first, longest-match, so the .failed retry form is never read as a board named "t..."), pre-flights roster membership and build dirs for ALL boards before anything is wiped or staged, warns per declared variant with no build dir (which hil_test.py would silently green-skip), forwards HIL_* knobs as export lines in one %q word the remote evals ('; '-joined so it round-trips under dash - an authorized HIL_NO_BOARD_LOCK force must not silently no-op), keeps HIL_REPORT_DIR local because the copy-backs look in REMOTE_DIR, and copies hil_report.json and the .failed re-run spec back beside the markdown, deleting stale local copies first so a green run cannot leave last run's spec looking current. Retries preserve the fleet: the documented path is the .failed spec, which already begins with --accumulate; a fresh scoped re-run would unlink the report and collapse the whole-fleet table to the retried boards alone. The risky logic is executable, not argued about: .claude/workflows/test-hil-validate.mjs pins the lookup/verdict helpers and runs in pre-commit (hil-validate-logic); nine staging tests drive hil_ci.sh through an ssh stub that models the real thing (argv joined into one string the remote re-splits, heredoc on stdin - the naive echo-stub passed while the feature was broken); and deliberate mutations of the verdict logic are all caught. Validated on the rig: a 2-board run (usbtest 30/30 on both; the pre-fix classifier, replayed against that run's real report, fails the fully-green stm32f723disco on its two passing '13443 KB/s' cells), the .failed --accumulate retry (merged report kept every earlier row), and a 10-run soak over random subsets of a 22-board pool - 43 board-slots, every failure signature matched pre-existing CI state or known flake, zero tooling failures, no locks left behind.

bsp(lpc55): run lpcxpresso55s28 as a high-speed device, add it to the ci pool

2026-08-18T15:07:49Z

Flip the board to device-highspeed/host-fullspeed, matching lpcxpresso55s69 and the way it is cabled on the test rig, and add it to the rig pool with the unique id read from its flash PFR. This is the first hardware coverage the ip3511 high-speed device path has ever had, and it immediately exposed the clear-stall type-bit bug fixed separately. The port swap also exposed a build gap: family.mk only linked a host controller for port 1, so make host builds on port 0 failed with undefined references - mirror family.cmake and link the OHCI driver there. The board's rhport defaults now come from family.cmake's guarded ones rather than a duplicate copy, so a -D override on the command line wins.

test/hil, ci: contain a wedged USB stack instead of stranding the runner

2026-08-18T05:19:09Z

A wedged USB device used to take the whole HIL run with it. Every worker that touched the poisoned node blocked uninterruptibly, the pool could not be joined, map_async discarded every board's result, and the job ran to the GitHub ceiling with no report at all -- while the self-hosted runner's single job slot stayed occupied and every queued job waited behind it. Bound the calls a worker makes itself. read_sysfs, bounded_open and run_cmd all answer within a wall clock; read_sysfs distinguishes "absent" from "unknown", because a blocked read is not evidence of absence, and caps stranded readers at four (each costs a thread and an fd for the life of the process) after which the worker declares itself blind. mtype, the gio unmount, the libmtp session and the arecord/iperf reaps go through those bounds; the MTP session runs in a disposable subprocess, since libmtp's ctypes calls block unkillably in D state. Bound the run. A pool guard (HIL_POOL_TIMEOUT, 60 min) fires before any job ceiling and still writes a report. When the pool will not shut down, the sweep kills what the workers spawned -- descendants, not just direct children, since flashers run in their own session -- confirms each kill actually landed, and exits early so the runner is freed. Whatever survived is named in the report. Deliberately shallow past that point. We do not re-scan process groups, prove pid ownership, or escalate through sudo: a root-owned survivor is reported, not force-killed, because signalling a pid we cannot prove is ours is the worse failure, and the job ceiling backstops whatever this misses. A D-state holder was never killable anyway. Recover instead of reporting a wedge. A HUNG usbtest case reflashes its own DUT through its roster flasher, but only where the flasher can reach its probe past a poisoned node -- openocd pinned to a validated vid_pid, or esptool. Where it cannot, the run says so rather than reserving budget for a path that cannot fire. Raise the CI ceilings above the pool guard so the guard fires first and still writes its report, and pin --retry 1 on every HIL leg: the guard is a flat constant and does not scale with max_retry, so argparse's default of 3 would triple the serialized usbtest tail against an unchanged guard. Split the module: execution in hil_test/hil_flash/usbtest, infrastructure in helper/ (locking, health, selection, shared bounded IO), and the two matrix generators into .github/scripts/ -- ci_set_matrix.py sat in workflows/, where GitHub treats every file as a workflow definition. 193 tests cover the bounded paths, the kill ladder, the guard and the selector against synthetic /proc trees and PATH-injected fakes; a real wedge cannot be manufactured on demand.