tinyusb.git/docs/superpowers, branch update-sponsor

ci: fix nine ways the selection under-selected or mismatched

2026-08-21T05:41:47Z

Every one of these dropped coverage silently - the worst failure mode here, because the PR still goes green. Found by review, each reproduced first. Selection rules: * class_macros derived the config macro from the class DIRECTORY, so a change to src/class/midi/midi2_device.c selected the midi_test examples (which do not compile it) and never examples/device/midi2_device (the only one that enables CFG_TUD_MIDI2, and the only one that does). The file's own macro is unioned in where it differs - union, never replace: over-selecting costs a build, under-selecting merges a break. * the ${FAMILY_MCUS} fallback added for espressif fired on any family whose _family_mcus came back empty, and _cmake_sets is if()-blind and keeps the FIRST definition - so mcx/frdm_mcxn947 answered MCXA15, a token six examples' skip.txt names, dropping 12 firmware images CMake builds. Limited now to families that never spell set(FAMILY_MCUS ...) at all. * lib_examples read only an example's top-level CMakeLists.txt/Makefile; host/msc_file_explorer_freertos names lib/embedded-cli in src/CMakeLists.txt and survived by luck. The whole example tree is scanned. (SEGGER_RTT and rt-thread still resolve to nothing: all three references sit inside a LOGGER=rtt guard no CI build sets - the documented ruling, not a miss.) * get_family_boards applied ci_skip_boards/ci_preferred_boards only under GITHUB_ACTIONS/CIRCLECI, so the selector answered differently on a laptop than on a runner; _prune_buildable forces CI semantics. Its one-board pick also abandoned the whole preferred list when entry one could not build the -e set, and asked skip_example without the build's -D tokens. * _config_enables and lib_examples still read with the locale encoding - under LC_ALL=C the selector tracebacked on three tracked tusb_config.h files. The whole selector and its suite run clean there now. Workflows: * the Membrowse Upload step omitted $EX_ARGS, but --one-first now picks the board from the -e set, so it configured a different, empty build dir and uploaded --identical for a board never compiled. It takes $EX_ARGS for the BOARD; the target stays the aggregate, which has no DEPENDS and still records every example. * blanking FAM_REGEX reset only build_filtered, leaving the build scoped while code-metrics took the UNSCOPED branch and diffed a 1-family run against the full averaged baseline. All three drop together now, as CircleCI's fall-open does. * CircleCI's EX_ARGS had no character screen and is used unquoted, and its code-metrics job still exit 1'd on an empty metrics set - which a scoped build makes a legitimate outcome. * a `ci-full` PR label now turns the scoping off for one PR. A selector bug under-selects silently, and without a label the only ways back to a full matrix are accidental. Performance, since the selector gates every other job: family.cmake texts are read once rather than per changed directory (a 6,000-file dep bump re-read 84 files 99,892 times) and _scrape_mcu is cached: 2.2s -> 0.29s there, 0.8s -> 0.33s on a class diff. Tests: a drift guard for hw/bsp families absent from ci_set_matrix.family_list (they select zero legs now, where they used to ride the full matrix); the rule-4 port test asserted a SUBSET, which set() satisfies, so it could not fail on the empty selection it exists to catch; the GITHUB_ENV guard test counted a SUM of two guards. Drops metrics.py's --only-examples, which nothing called, and applies the TOTAL scrub to the by-example branch that skipped it.

docs: record the CI selection design and its plan

2026-08-21T04:09:03Z

The binding rule table (17 rows x 3 answer columns), the measured effect per PR shape, and the reasoning behind the parts that look surprising: why empty means empty, why hw/mcu and lib are rules rather than full-matrix paths, why get_deps.py is diffed as data, and which build system is the reference. The plan is the task-by-task record of how it was built, kept as the origin trail.

ci: scope the build matrix and the HIL run to what a PR affects

2026-08-21T04:07:27Z

Every PR built all 74 legs (2494 example builds on GHA cmake alone) and flashed all 30 rig boards, whatever it touched. One classifier now walks the PR diff twice and answers three questions: which families to build, which examples per family, and which boards run which tests. Fail-open throughout - anything no rule classifies, any exception, any unusable output falls back to the full matrix, and a master push always builds everything. test/hil/helper/hil_select.py moves to tools/ci_select.py: it is no longer HIL-only, and tools/ is where the build side can import it. test_hil_select.py follows it as test_ci_select.py. Rules (docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md holds the full table): a port selects the families whose family.cmake references it, and its role - a dcd change skips host examples and vice versa; a class selects only the examples whose tusb_config.h enables its CFG_TU[DH]_ macro, following cross-class includes; an example selects itself; hw/bsp selects its family or board; hw/mcu and lib select whoever references them. CMake is the reference for all of it - make follows whatever cmake decides, family.mk is never scanned. Empty means empty (maintainer ruling): a rule that classifies a path to nothing selects nothing. Ports no family references, classes no config enables, libs no example builds and hw/mcu paths that resolve nowhere are all real - nothing compiles them, so nothing can validate them, and the master-push build is the net. Structural tests pin each such case with an explicit allowlist, so the day one stops being empty it fails pre-commit instead of silently narrowing CI. Per-example builds: build.py grows a repeatable -e, resolved against the targets CMake actually registered and batched into one `cmake --build --target a b c`. build_utils mirrors CMake's family_filter (the whole FAMILY_MCUS list, ${...} and string(TOUPPER ...) resolved) for the cmake side, while the make side keeps master's algorithm verbatim - the two build systems answer differently and a shared answer breaks lpc54's make link. hil-build gains this even on a full selection: 1702 example builds become 515. Transport: the selection travels as a file, never an argv or env var - a mass-sweep diff selects 261 KB against a 128 KiB exec limit, and E2BIG would fail the step before its own fallback could run. CircleCI carries the example map inside the generated config (pipeline parameters cap at 512 chars), swapped into the parameter defaults by sentinel match, and drops the scoping wholesale if that rewrite fails. Every PR-derived value written to $GITHUB_ENV/$GITHUB_OUTPUT is character-screened. Code metrics follow the scoping: metrics.py emits per-example totals, and metrics_pair_compare compares the (board, example) pairs present on both sides instead of a scoped run against a full-matrix average. The selector's own suite gates it in both providers: a selector that exits 0 with valid-but-wrong JSON is the one failure fail-open cannot catch, so a red suite means the full matrix.

docs: name the report-unification handoff after its PR

2026-08-20T10:50:46Z

The doc carried its own rename instruction for when the branch gained a number; the branch is PR #3836.

docs: hand off unifying the HIL report's two artifacts

2026-08-20T09:47:23Z

Four writers produce hil_report.md and three of them write no JSON - the no-boards exit, the pool-guard fallback and _abandon_exit's text prepend. Those are exactly the runs that failed, so hil_summary.py, which builds an agent's per-board verdicts from the sidecar, sees nothing while a human reading the markdown sees the real story. The scope note is markdown-only too, so a three-board PR run and a full run that lost 24 boards are indistinguishable in JSON. Five tasks: put scope in the sidecar, render the markdown from the document, give the two early-exit paths a document, make _abandon_exit set a caveat field instead of prepending to a file it did not write, then pin the invariant that re-rendering the JSON reproduces the markdown byte for byte. Split out because it is a hil_test.py reporting refactor, and the abandon path runs while the interpreter is being torn down - it deserves its own review.

docs: spec, plan and outcome of the .claude instruction-surface audit

2026-08-20T09:47:23Z

Nothing checked the agents, workflows and skills against the code they describe, and the surface had drifted into stating incompatible rules. This records the protocol that found the defects and what it cost. Method: parallel subagents extract every falsifiable claim into JSONL ledgers; a validator re-reads each cited line and rejects any ledger whose quoted text is not there, so an extractor that paraphrases or hallucinates fails a script rather than reaching the verification queue - 1,387 claims, zero such failures. Verification runs only in the main session, and the same gate pointed at `citation` then checks the verifier's own work. Hard-earned evidence is source of truth: code is authoritative about code, experience about hardware, so claims get a fourth verdict, EARNED, and "no backing found" is never grounds for deletion. All 1,387 claims carry a verdict; the behavior sweep deliberately never emits CONFIRMED from a token match, because finding a claim's vocabulary proves presence, not truth. Every real defect came from cross-document comparison - none from any mechanical pass. A path-existence lint was built, measured (11 flags on the audited tree, all false positives, and the target defect is lexically identical to correct text elsewhere), and rejected; recorded so nobody rebuilds it.

examples: document and work around the i.MX RT and LPC55 USB errata

2026-08-18T15:07:49Z

ERR050101: while an isochronous IN endpoint is active, an IN token addressed to that same endpoint number on ANOTHER device sharing the host can silently unprime one of this device's OUT endpoints - control, bulk, interrupt or isochronous alike. NXP states it cannot be detected by software and raises no interrupt, so the endpoint simply stops answering and the transfer never completes. The workaround is a uniqueness requirement rather than a particular number: the isochronous IN endpoint must not share its number with any IN endpoint in use on the bus. One family-wide constant therefore defeats it, since two affected boards on the same hub then pick the same number and each becomes the other's aggressor. CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101 is set only for the parts whose errata list it - RT1015, RT1020, RT1024 and RT1050, where it is marked no fix scheduled, plus RT1060 and RT1064 rev A - so RT1010 and the RT11xx family keep the ordinary number and cannot collide with an affected board beside them. Several affected boards on one hub can still be given distinct numbers with -DEPNUM_ISO_IN. The guard covers every example that has an isochronous IN endpoint: audio_test, audio_4_channel_mic, uac2_headset, cdc_uac2, usbtest, video_capture and video_capture_2ch. The video examples move the endpoint only when streaming isochronously, since the bulk configuration is unaffected, and video_capture_2ch takes two numbers because it has two streams. The macro name follows CFG_TUSB_RP2_ERRATA_E2/E4/E15 already in tree, and its is fixed, and which cannot be told apart at compile time - a way to define it to 0. device_issues.rst records ERR050101 against every affected part with a link to each errata sheet, and adds the LPC55S2x USB.3 speed-detection and USB.5 isochronous IN entries, neither of which TinyUSB works around. The branch's design notes are included under docs/superpowers. Verified: 340 wedge-free runs on mimxrt1064_evk, which previously wedged within hours, and the macro resolving to endpoint 0x87 on mimxrt1064_evk against 0x83 on mimxrt1010_evk and stm32f407disco.

docs: hand off follow-up work as per-PR plans

2026-08-18T05:19:09Z

Records the convention in CLAUDE.md -- deferred work is a SEPARATE scope that deserves its own PR, written by another session, so it is handed off as a writing-plans doc in docs/superpowers/followup/pr-.md rather than accumulated in the PR that found it. Five handoffs from #3803: flasher_recover (convoy-safe recovery for J-Link boards, seven validated on the rig), the blindness reporting gaps, the usbtest recovery reserve, the IAR re-run spec, and the pci-rebind stranding question. Each carries what is already established with its citations and measurements, what remains, and why it was split out. One doc per follow-up, not one per PR: a per-PR file invites unrelated work into the same document and rots as a unit.

skills, docs: rewrite USB recovery from the live incidents

2026-08-18T05:19:09Z

Two things the rig taught us that the old guidance got wrong. A usbfs ioctl wedged in D state cannot be freed on a running kernel. It holds the device lock, so usb_disconnect() blocks behind it; reboot(2) walks device_shutdown() and takes the same lock, so every userspace reboot stalls too. Only sysrq b (emergency_restart, which skips device_shutdown) or hypervisor action clears it -- all cited to the kernel source. The recovery ladder is generic across rigs now (ci.lan, hifiphile, a bench PC) and ends at hypervisor escalation only where host access exists. Two claims are corrected outright: JLinkExe is NOT convoy-safe, and a park-flash cannot free a device-lock owner. The hil skill's banner list is what an operator agent matches a report against, so it enumerates the banners that actually exist, including the D-state note -- which is explicitly NOT a wedge, since a healthy in-flight testusb is uninterruptible for most of every case and a concurrent CI battery would otherwise turn a clean run red.

test/hil, ci: contain a wedged USB stack instead of stranding the runner

2026-08-18T05:19:09Z

A wedged USB device used to take the whole HIL run with it. Every worker that touched the poisoned node blocked uninterruptibly, the pool could not be joined, map_async discarded every board's result, and the job ran to the GitHub ceiling with no report at all -- while the self-hosted runner's single job slot stayed occupied and every queued job waited behind it. Bound the calls a worker makes itself. read_sysfs, bounded_open and run_cmd all answer within a wall clock; read_sysfs distinguishes "absent" from "unknown", because a blocked read is not evidence of absence, and caps stranded readers at four (each costs a thread and an fd for the life of the process) after which the worker declares itself blind. mtype, the gio unmount, the libmtp session and the arecord/iperf reaps go through those bounds; the MTP session runs in a disposable subprocess, since libmtp's ctypes calls block unkillably in D state. Bound the run. A pool guard (HIL_POOL_TIMEOUT, 60 min) fires before any job ceiling and still writes a report. When the pool will not shut down, the sweep kills what the workers spawned -- descendants, not just direct children, since flashers run in their own session -- confirms each kill actually landed, and exits early so the runner is freed. Whatever survived is named in the report. Deliberately shallow past that point. We do not re-scan process groups, prove pid ownership, or escalate through sudo: a root-owned survivor is reported, not force-killed, because signalling a pid we cannot prove is ours is the worse failure, and the job ceiling backstops whatever this misses. A D-state holder was never killable anyway. Recover instead of reporting a wedge. A HUNG usbtest case reflashes its own DUT through its roster flasher, but only where the flasher can reach its probe past a poisoned node -- openocd pinned to a validated vid_pid, or esptool. Where it cannot, the run says so rather than reserving budget for a path that cannot fire. Raise the CI ceilings above the pool guard so the guard fires first and still writes its report, and pin --retry 1 on every HIL leg: the guard is a flat constant and does not scale with max_retry, so argparse's default of 3 would triple the serialized usbtest tail against an unchanged guard. Split the module: execution in hil_test/hil_flash/usbtest, infrastructure in helper/ (locking, health, selection, shared bounded IO), and the two matrix generators into .github/scripts/ -- ci_set_matrix.py sat in workflows/, where GitHub treats every file as a workflow definition. 193 tests cover the bounded paths, the kill ladder, the guard and the selector against synthetic /proc trees and PATH-injected fakes; a real wedge cannot be manufactured on demand.