tinyusb.git/.github/workflows/build_util.yml, branch rework-metrics

ci: disable SonarQube analysis

2026-09-06T13:12:43Z

ci: address Sonar security findings

2026-09-06T09:08:01Z

ci-pinned-boards: curate the board set, esp upload via hil-build-esp, rp2040 fixes

2026-09-06T08:57:00Z

The cmake job builds and uploads every pinned board to membrowse on every run, PRs and pushes alike (--identical when code did not change) - it is the single owner of that upload, as before this branch touched it. Curate the board set itself. cxd56/spresense, da1469x/da1469x_dk_pro, fomu/fomu and ft9xx/mm900evxb are no longer pinned CI boards: none sit on either HIL roster and no other CI board carries their driver, so their size tracking was build-only with no hardware behind it - documented with honest `uncovered` reasons instead of silently dropping coverage. Swap lpc15/lpcxpresso1549 -> lpc11/lpcxpresso11u68 for dcd_lpc_ip3511, lpc17/lpcxpresso1769 -> lpc40/ea4088_quickstart for dcd_lpc17_40+ohci, and samd5x_e5x/same54_xplained -> metro_m4_express; adafruit_fruit_jam now also claims dcd_rp2040. All four changed boards build-verified (device/cdc_msc, cmake), and the rp2040 group's (family, board) sort order restored. Two stale notes the swap left behind are fixed: lpcxpresso11u68 is on neither roster, so its "HIL (htpc)" claim (copy-pasted from the entry it replaced) was false, and ea4088_quickstart is a maintainer bench board noted the same way frdm_kl25z and lpcxpresso1549 were. Rename the board file to .github/ci-pinned-boards.json, with matching --ci-pinned-boards/--ci-pinned-boards-only flags: it pins the board that represents each family in CI, and membrowse history is keyed on each pinned board's / target names, so the set is explicitly held rather than derived. Content and target names are unchanged; the per-entry "family" key is dropped - board names are unique across hw/bsp/*/boards, so it was redundant data that could drift from the tree. build.py's resolve_ci_boards() and the coverage checker derive the family from the board dir, and entries are sorted by board name, the only key left to sort on. espressif builds have no leg in the cmake job (esp-idf is by far the slowest toolchain, and its boards are already enumerated by name for the rig) - hil-build-esp keeps its own upload-membrowse + secrets: inherit instead, and hil-build-esp-identical covers the --identical row for a no-code-change push, the same gap hil-build-esp's own code-changed gate would otherwise leave. build_util.yml's Membrowse Upload step skips any leg whose args carry --build-name, an extra -D or --cflag - a HIL variant build - since it configures with the same -DBOARD= as the plain build and would otherwise fight it for the same upload target name; this protects hil-build-esp's DMA variants specifically. The step also does not scope by $EX_ARGS (kept on the Build step): scoping the upload by the PR's example filter would leave examples outside that filter with no upload attempt at all - a silent history gap rather than the --identical row the design intends. Fork PRs: GHA withholds secrets, so MEMBROWSE_API_KEY reaches membrowse_report.py empty. Master's CMake-expanded bare --api-key fell through to membrowse's GitHub tokenless auth; the wrapper now does the same by omitting --api-key entirely when the env var is empty, rather than exiting - the previous hard-exit was silently hidden behind the workflow's continue-on-error, so every fork PR lost its upload with no visible failure. Retiring ci_skip_boards let CircleCI's plain family sweep reach five rp2040 boards for the first time; two didn't build. pico_sdk fails dual/host_hid_to_device_cdc and dual/host_info_to_device_cdc - a meta-board with no board.h pin defines of its own has no PICO_DEFAULT_PIO_USB_* macros, which a dual-role rp2040 example needs for its PIO-USB host side - so each example gained a skip.txt entry naming the failure. feather_rp2040_max3421 failed host/bare_api on -Wmissing-prototypes for max3421_int_handler in hw/bsp/rp2040/family.c; it is only ever used as a same-file GPIO IRQ callback (no other caller), so it is marked static rather than skip-listing a zero-risk one-line fix. All 8 rp2040 boards now build clean, `-b ` and the bare `rp2040` family sweep alike. drivers_coverage_check.py type-checks `drivers` before iterating: a string previously iterated character-by-character into nonsense "X matches no driver source file" errors, and null raised an unhandled TypeError. Not fixed, flagged for a follow-up: MEMBROWSE_API_KEY reaches membrowse_report.py as a child-process argv (visible in /proc//cmdline on the runner, even though the logged command line is redacted) rather than via the environment. Final-review fixes: the espressif Membrowse Upload docker run only forwarded MEMBROWSE_API_KEY and CI into the container, so membrowse's --github metadata detection (GITHUB_EVENT_NAME/GITHUB_SHA/...) and its fork-PR tokenless auth never saw the actual event - add GITHUB_ACTIONS/GITHUB_EVENT_NAME/GITHUB_SHA/GITHUB_REF_NAME plus a read-only bind mount of GITHUB_EVENT_PATH (containers don't inherit runner env by default; the esp Build step needs none of this, since its `--target all` never builds the *-membrowse/-upload custom targets, so its now-unused `pip install membrowse` is also dropped). check-paths' code filter also gains tools/membrowse_report.py, so a PR touching only that file exercises the real build+upload matrix (ci_select rule 2d) instead of 74 no-op --identical legs. Review fixes: generate_membrowse_sizes()'s all-examples scope (-b with no -e) now averages each file's size across the elfs it appears in instead of summing, matching metrics.py's compute_avg() semantics from the linkermap engine this branch replaced - a file linked by N examples no longer reports ~N times its real size, and the Flash/RAM columns are a real binary's size again; single-example (-e) stays byte-identical (sum, same as before). The Membrowse Upload step (build_util.yml) and hil-build-esp-identical's loop (build.yml) now strip any -e pairs hil_ci_set_matrix.py bakes into a PR-scoped hil-build-esp leg's args before invoking, so a PR-scoped run still uploads every example of the pinned board rather than only the PR-selected subset - closing exactly the silent history gap the step's own comment already warned against. Also: build.py's espressif branch gates the idf.py build path on 'all' being present in build_targets rather than being exactly ['all'], so a combined --target invocation still runs the build instead of silently skipping on a fresh dir; membrowse_compare.py's layout-based bucketing unions the actual flash/ram buckets of a multi-region section's regions instead of assuming any 2+-region split is a flash+ram pair (e.g. RAM_D1/RAM_D2 now stays ram-only); membrowse_report.py's extract_defsyms() dedupes to match its own docstring. Post-validate fixes, folded in. The Membrowse Upload step threads the PR's -e filter through for BOARD ELIGIBILITY only: without it, resolve_ci_boards() ran with examples=None and still resolved the pinned board even when the Build step above had fallen back to a substitute, uploading --identical for a board that run never built. Under --ci-pinned-boards-only tools/build.py now nulls the example filter after board selection, so the upload still touches every example of whichever board got resolved - real rows for what was built, --identical for the rest. same7x earns its pin instead of assuming it: a full family sweep showed five *_freertos examples failing (no FreeRTOSConfig.h wired for same7x - skip.txt'd as family:same7x, naming the reason) and same70_qmtech's board.cmake/mk pointing at a same70n19b_flash.ld that never existed in the tree (the SDK ships none) - the board now carries its own, Q21B's script with the N19B's MEMORY sizes (512K flash / 256K ram per same70n19b.h), like the other boards with a board-local .ld. Both same7x boards now build their full example sets green under the real budget. cxd56 stays OUT of ci_set_matrix's family_list and its BSP reverts to master untouched: the same sweep showed NuttX-header failures across three examples, spresense is unpinned anyway, and a -Wno-error suppression for a family no CI leg builds silences a real diagnostic class for nothing. drivers_coverage_check.py gains two guards: a pinned family that no CI toolchain actually compiles is now a validity error (the dead-pin case the same7x episode demonstrated), and a non-dict entry in "boards" reports one clear error instead of a traceback. The spec and plan docs are updated to the shipped names (membrowse-targets.json / --board-pins / --pins-only never shipped; 71 occurrences), and stale test fixtures naming the deleted tinyusb_metrics target are repointed at examples-membrowse-upload. Validate-loop fixes, folded in. membrowse_compare.py's report_for_elf() now passes the elf's own linker scripts and --defsym values (read from its ninja build graph, exactly as the CI upload path does) to `membrowse report`: with none given membrowse falls back to its default Code/Data regions, which the layout bucketing cannot map, so `.data` lost its flash-side load image and was counted RAM-only (verified on stm32h743eval: region 'Data' without the script, ['FLASH','RAM_D1'] with it). _bucket_from_layout() also unions in the flash side for a BOTH_SECTIONS section whose layout lists only its RAM region - pico-sdk's script never lists `.data` under FLASH, undercounting raspberry_pi_pico's flash by 11.2K. membrowse_onboard.py: `membrowse onboard` checks out and `git clean -fdx`s every historical commit in whatever directory it runs, which would detach HEAD and wipe the deps symlinks in the repo root - so the wrapper runs it in a disposable worktree under cmake-metrics/, and the composed build script relinks deps and reconfigures the (also wiped) build dir before every historical build, not just the first. --build-dirs now includes the example's own directory, so a change to its sources no longer backfills as --identical. Not fixable in the wrapper: onboard applies one commit's linker scripts to every commit in the range (no per-commit override in its CLI).

ci-boards: one curated board list for CI, membrowse and HIL coverage

2026-09-06T08:57:00Z

The pinned-target file becomes .github/ci-boards.json - "pin" reads as GPIO pin in this repo - and with it the flags (--ci-boards, --ci-boards-only), the resolver (resolve_ci_boards()) and the JSON's own key (boards). It is now the only board-curation mechanism: tools/build.py's ci_skip_boards/ci_preferred_boards are gone, with samd2x_l2x and stm32h7 pinned to preserve today's CI picks, and resolve_ci_boards() applies the -e buildability filter so a scoped PR cannot pick a board that builds none of the selected examples and go green having compiled nothing. Curate the set itself: RP2350 boards (raspberry_pi_pico2, adafruit_fruit_jam for hcd_pio_usb) replace feather_rp2040_max3421, add max32666fthr, align the lpc55/nrf pins with rig boards, and drop the nuvoton trio, msp432e4, broadcom_64bit, f1c100s, gd32vf103 and mm32 - dwc2 stays covered, the rest move to uncovered with reasons. The checker moves to tools/drivers_coverage_check.py and reports coverage for both consumers without failing on gaps: membrowse boards (documented uncovered -> INFO, undocumented -> WARNING) and the HIL rosters (INFO), mapping boards to drivers through ci_select's rule-3/4 machinery including its role filter, so an hcd driver only counts host-role rig boards. Validity errors - unknown names, a driver both pinned and uncovered, an hcd claim on a board with no host example - still exit 1. Its chdir now scopes both cwd-relative calls, so the checker works from any directory, and the hook watches the inputs it actually reads.

metrics: replace the linkermap CI pipeline with membrowse

2026-09-06T08:57:00Z

Membrowse becomes TinyUSB's size-analytics system, in CI and locally. .github/membrowse-targets.json names the boards CI builds per family and the drivers each covers; a checker enforced by pre-commit fails when a dcd/hcd driver (plus ehci/ohci) is neither claimed nor documented in `uncovered`, validates board and family names against hw/bsp, and requires a board claiming an hcd driver to build at least one host/dual example. tools/build.py resolves that file into the build matrix and gates uploads on it. Locally, tools/membrowse_compare.py diffs `membrowse report --json --all-symbols` per source file, keyed on object_file (membrowse 1.2.9 truncates source_file to a basename), and metrics_compare_base.py uses it as the default engine, with `--engine linkermap` as the legacy map.json fallback (still required for --combined/--ci). A guard errors instead of writing a silent all-zero table when the filters match nothing. linkermap stops running on every build: the POST_BUILD hook goes, the explicit -linkermap target and a new examples-linkermap aggregate stay, and family_add_linkermap() no-ops when the dep is not fetched. CI loses the whole linkermap pipeline - the code-metrics job, the sticky PR size comment, release metrics assets, per-family artifacts, metrics_pair_compare.py, the membrowse-onboard workflow and CircleCI's dead tinyusb_metrics lane. The cmake job builds the named boards (esp-idf included, membrowse wired into the IDF docker path via family_initialize_project and MEMBROWSE_LD_OVERRIDE) and uploads only those, keeping the --identical path for code-unchanged runs. ci_select follows: the size tooling no longer runs in any CI build, while the board list decides what a family's legs compile and so forces a full matrix. Adds the espressif, same7x, cxd56 and f1c100s families the board list needs, each build-verified. The upload itself moves out of cmake: family_add_membrowse()'s ~90-line inline bash becomes tools/membrowse_report.py, carrying ninja linker-script extraction with recursive INCLUDE resolution, --defsym collection, map-file detection, --identical fallback and upload composition. The API key is read from the environment at run time, passed via argv and redacted in logs - nothing is baked into build.ninja at configure time, which the CMake-expanded $ENV{MEMBROWSE_API_KEY} used to do. tools/membrowse_onboard.py wraps `membrowse onboard` for history backfill, deriving the build script, elf path and the CI-matching target name (/) so a backfill cannot land under a name CI never uploads to; it refuses to start on a dirty worktree, since onboard checks out past commits in place. A membrowse skill documents local reports, size diffs, board-list maintenance, uploads and backfill; the code-size skill records the new default engine and its linkermap fallback. Three follow-up handoffs capture what was deliberately split out: full linkermap removal, membrowse --combined support, and the dead build_filtered plumbing.

ci: an empty selection must build nothing, plus selector follow-ups (#3845)

2026-08-25T02:46:42Z

ci: an empty selection must build nothing, plus selector follow-ups A PR whose build axis legitimately selected nothing rebuilt everything. build.yml reads .build.families twice - as a |-joined regex, and implicitly as "is anything selected" - but tested only -z "$FAMILY_REGEX", which an empty list and a charset-rejected one both satisfy while meaning opposite things. ci_set_matrix had already returned the correct all-empty matrix; the fall-open branch discarded it. #3842 and #3840 each spent 74 cmake legs on it. Branch on the two cases instead, rename FAM_* to FAMILY_*, and cover the block with a test that extracts it from build.yml and executes it - it had no test at all, which is how this shipped through two merges. Follow-ups to the same machinery: glob.escape the repo root at five sites, so a checkout path containing [ or * stops failing closed; drop the ci-full label, read after the matrix was already computed and so never functional; delete 13 mcu:MKL25ZXX / mcu:SAME5X skip tokens matching no board; carry the rule table in the module docstring, guarded against drift; and pin six selection behaviours a mutation pass proved untested. Cut the selector's cost 1.8x (26.0s -> 14.6s) with 0 divergences over 260 paths, and stop scoping the membrowse upload by the PR example filter.

ci: fix nine ways the selection under-selected or mismatched

2026-08-21T05:41:47Z

Every one of these dropped coverage silently - the worst failure mode here, because the PR still goes green. Found by review, each reproduced first. Selection rules: * class_macros derived the config macro from the class DIRECTORY, so a change to src/class/midi/midi2_device.c selected the midi_test examples (which do not compile it) and never examples/device/midi2_device (the only one that enables CFG_TUD_MIDI2, and the only one that does). The file's own macro is unioned in where it differs - union, never replace: over-selecting costs a build, under-selecting merges a break. * the ${FAMILY_MCUS} fallback added for espressif fired on any family whose _family_mcus came back empty, and _cmake_sets is if()-blind and keeps the FIRST definition - so mcx/frdm_mcxn947 answered MCXA15, a token six examples' skip.txt names, dropping 12 firmware images CMake builds. Limited now to families that never spell set(FAMILY_MCUS ...) at all. * lib_examples read only an example's top-level CMakeLists.txt/Makefile; host/msc_file_explorer_freertos names lib/embedded-cli in src/CMakeLists.txt and survived by luck. The whole example tree is scanned. (SEGGER_RTT and rt-thread still resolve to nothing: all three references sit inside a LOGGER=rtt guard no CI build sets - the documented ruling, not a miss.) * get_family_boards applied ci_skip_boards/ci_preferred_boards only under GITHUB_ACTIONS/CIRCLECI, so the selector answered differently on a laptop than on a runner; _prune_buildable forces CI semantics. Its one-board pick also abandoned the whole preferred list when entry one could not build the -e set, and asked skip_example without the build's -D tokens. * _config_enables and lib_examples still read with the locale encoding - under LC_ALL=C the selector tracebacked on three tracked tusb_config.h files. The whole selector and its suite run clean there now. Workflows: * the Membrowse Upload step omitted $EX_ARGS, but --one-first now picks the board from the -e set, so it configured a different, empty build dir and uploaded --identical for a board never compiled. It takes $EX_ARGS for the BOARD; the target stays the aggregate, which has no DEPENDS and still records every example. * blanking FAM_REGEX reset only build_filtered, leaving the build scoped while code-metrics took the UNSCOPED branch and diffed a 1-family run against the full averaged baseline. All three drop together now, as CircleCI's fall-open does. * CircleCI's EX_ARGS had no character screen and is used unquoted, and its code-metrics job still exit 1'd on an empty metrics set - which a scoped build makes a legitimate outcome. * a `ci-full` PR label now turns the scoping off for one PR. A selector bug under-selects silently, and without a label the only ways back to a full matrix are accidental. Performance, since the selector gates every other job: family.cmake texts are read once rather than per changed directory (a 6,000-file dep bump re-read 84 files 99,892 times) and _scrape_mcu is cached: 2.2s -> 0.29s there, 0.8s -> 0.33s on a class diff. Tests: a drift guard for hw/bsp families absent from ci_set_matrix.family_list (they select zero legs now, where they used to ride the full matrix); the rule-4 port test asserted a SUBSET, which set() satisfies, so it could not fail on the empty selection it exists to catch; the GITHUB_ENV guard test counted a SUM of two guards. Drops metrics.py's --only-examples, which nothing called, and applies the TOTAL scrub to the by-example branch that skipped it.

ci: scope the build matrix and the HIL run to what a PR affects

2026-08-21T04:07:27Z

Every PR built all 74 legs (2494 example builds on GHA cmake alone) and flashed all 30 rig boards, whatever it touched. One classifier now walks the PR diff twice and answers three questions: which families to build, which examples per family, and which boards run which tests. Fail-open throughout - anything no rule classifies, any exception, any unusable output falls back to the full matrix, and a master push always builds everything. test/hil/helper/hil_select.py moves to tools/ci_select.py: it is no longer HIL-only, and tools/ is where the build side can import it. test_hil_select.py follows it as test_ci_select.py. Rules (docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md holds the full table): a port selects the families whose family.cmake references it, and its role - a dcd change skips host examples and vice versa; a class selects only the examples whose tusb_config.h enables its CFG_TU[DH]_ macro, following cross-class includes; an example selects itself; hw/bsp selects its family or board; hw/mcu and lib select whoever references them. CMake is the reference for all of it - make follows whatever cmake decides, family.mk is never scanned. Empty means empty (maintainer ruling): a rule that classifies a path to nothing selects nothing. Ports no family references, classes no config enables, libs no example builds and hw/mcu paths that resolve nowhere are all real - nothing compiles them, so nothing can validate them, and the master-push build is the net. Structural tests pin each such case with an explicit allowlist, so the day one stops being empty it fails pre-commit instead of silently narrowing CI. Per-example builds: build.py grows a repeatable -e, resolved against the targets CMake actually registered and batched into one `cmake --build --target a b c`. build_utils mirrors CMake's family_filter (the whole FAMILY_MCUS list, ${...} and string(TOUPPER ...) resolved) for the cmake side, while the make side keeps master's algorithm verbatim - the two build systems answer differently and a shared answer breaks lpc54's make link. hil-build gains this even on a full selection: 1702 example builds become 515. Transport: the selection travels as a file, never an argv or env var - a mass-sweep diff selects 261 KB against a 128 KiB exec limit, and E2BIG would fail the step before its own fallback could run. CircleCI carries the example map inside the generated config (pipeline parameters cap at 512 chars), swapped into the parameter defaults by sentinel match, and drops the scoping wholesale if that rewrite fails. Every PR-derived value written to $GITHUB_ENV/$GITHUB_OUTPUT is character-screened. Code metrics follow the scoping: metrics.py emits per-example totals, and metrics_pair_compare compares the (board, example) pairs present on both sides instead of a scoped run against a full-matrix average. The selector's own suite gates it in both providers: a selector that exits 0 with valid-but-wrong JSON is the one failure fail-open cannot catch, so a red suite means the full matrix.

hil, ci: scope HIL builds and tests to the boards a PR affects (#3797)

2026-07-29T19:29:32Z

hil, ci: scope HIL builds and tests to the boards a PR affects Add test/hil/hil_select.py, a stdlib-only selector that maps a PR diff to the rig boards, tests and BSP families a change can affect, and wire it into CI so pull requests build and run only those. A port change picks its families' boards, a class change picks the examples enabling that class, and device/host changes prune the other role. Anything unclassified — infra, an unmapped port, a selector error — falls back to the full matrix, and push/schedule runs are untouched. Move the shared example lists to hil_examples.py; 54 hardware-free tests cover the rules.

hil: controller-aware scheduling of flash and usbtest concurrency

2026-07-17T04:54:14Z

Full-fleet profiling (HIL_PROFILE=1 instrumentation, included) showed each uPD720201 controller's serialized usbtest battery chain dominates wall time, and a board whose marginal device port bounces during concurrent batteries can wedge or kill the controller ("xHCI host not responding to stop endpoint command"). Every such death traced to mimxrt1015's port (its old "kills the uPD720201" reputation) - it is removed from the config until recabled; mimxrt1064's enum-retry stalls were a loose device cable (re-seated). nrf54lm20dk moves to boards-skip until its failing J-Link probe is replugged. With the hardware fixed both cards run width-4 batteries plus full flash churn clean, so scheduling stays simple: two symmetric knobs, flashes and batteries budgeted per controller. - schedule_boards(): dispatch boards round-robin across host controllers from a persisted hint cache (~/.cache/tinyusb-hil/ctrl_cache.json), learned and merge-on-write refreshed each run (concurrent HIL jobs keep each other's entries). Only the cached PCI address is consumed - dispatch order and first-flash budgeting, never battery serialization (batteries resolve live or fail closed to an all-slot permit). - HIL_FLASH_PARALLEL (8) and HIL_USBTEST_PARALLEL (4) are budgeted per controller via lock slots assigned on first sight. - re-runs: a failed run writes /.failed with the exact re-run spec (--accumulate -b -bt :) instead of the inverted --skip-board list of everything that passed; --skip-board is gone, --flasher/--exclude-flasher scope a config across CI jobs by flasher type (no board names hardcoded in workflows), and -a/--accumulate merges a re-run into the existing report. The spec is stamped with GITHUB_RUN_ID and cleared on fresh runs, so a retry can never consume a spec left behind by a different run's dead or skipped attempt. - CI: esp-idf firmware builds move out of hil-build into hil-build-esp, and the esptool-flashed boards run in their own hil-tinyusb-esp job, so the main hil-tinyusb run starts as soon as the fast toolchains finish instead of waiting on the slow esp-idf build (an esp toolchain flake previously skipped the whole rig run). Artifacts are namespaced per toolchain so the esp job downloads only esp-idf binaries. - HIL_PROFILE=1: timestamped log lines, per-flash durations, permit-wait logging, uid->controller map dump for analysis. - hil_report: per-variant test duration as a dedicated trailing column, recorded only by full runs. Validated on the ci rig (fixed seeds 20260716/777, full fleet at 8/4): 738s/780s walls with only known-flake failures and no controller deaths, vs 1134-1211s serialized-battery baseline.