tinyusb.git/.github/workflows, branch rework-metrics

ci: upload ESP history from full roster

2026-09-06T15:03:42Z

ci: disable SonarQube analysis

2026-09-06T13:12:43Z

ci: address Sonar security findings

2026-09-06T09:08:01Z

ci: remove dead metrics scaffolding

2026-09-06T08:57:00Z

ci-pinned-boards: curate the board set, esp upload via hil-build-esp, rp2040 fixes

2026-09-06T08:57:00Z

The cmake job builds and uploads every pinned board to membrowse on every run, PRs and pushes alike (--identical when code did not change) - it is the single owner of that upload, as before this branch touched it. Curate the board set itself. cxd56/spresense, da1469x/da1469x_dk_pro, fomu/fomu and ft9xx/mm900evxb are no longer pinned CI boards: none sit on either HIL roster and no other CI board carries their driver, so their size tracking was build-only with no hardware behind it - documented with honest `uncovered` reasons instead of silently dropping coverage. Swap lpc15/lpcxpresso1549 -> lpc11/lpcxpresso11u68 for dcd_lpc_ip3511, lpc17/lpcxpresso1769 -> lpc40/ea4088_quickstart for dcd_lpc17_40+ohci, and samd5x_e5x/same54_xplained -> metro_m4_express; adafruit_fruit_jam now also claims dcd_rp2040. All four changed boards build-verified (device/cdc_msc, cmake), and the rp2040 group's (family, board) sort order restored. Two stale notes the swap left behind are fixed: lpcxpresso11u68 is on neither roster, so its "HIL (htpc)" claim (copy-pasted from the entry it replaced) was false, and ea4088_quickstart is a maintainer bench board noted the same way frdm_kl25z and lpcxpresso1549 were. Rename the board file to .github/ci-pinned-boards.json, with matching --ci-pinned-boards/--ci-pinned-boards-only flags: it pins the board that represents each family in CI, and membrowse history is keyed on each pinned board's / target names, so the set is explicitly held rather than derived. Content and target names are unchanged; the per-entry "family" key is dropped - board names are unique across hw/bsp/*/boards, so it was redundant data that could drift from the tree. build.py's resolve_ci_boards() and the coverage checker derive the family from the board dir, and entries are sorted by board name, the only key left to sort on. espressif builds have no leg in the cmake job (esp-idf is by far the slowest toolchain, and its boards are already enumerated by name for the rig) - hil-build-esp keeps its own upload-membrowse + secrets: inherit instead, and hil-build-esp-identical covers the --identical row for a no-code-change push, the same gap hil-build-esp's own code-changed gate would otherwise leave. build_util.yml's Membrowse Upload step skips any leg whose args carry --build-name, an extra -D or --cflag - a HIL variant build - since it configures with the same -DBOARD= as the plain build and would otherwise fight it for the same upload target name; this protects hil-build-esp's DMA variants specifically. The step also does not scope by $EX_ARGS (kept on the Build step): scoping the upload by the PR's example filter would leave examples outside that filter with no upload attempt at all - a silent history gap rather than the --identical row the design intends. Fork PRs: GHA withholds secrets, so MEMBROWSE_API_KEY reaches membrowse_report.py empty. Master's CMake-expanded bare --api-key fell through to membrowse's GitHub tokenless auth; the wrapper now does the same by omitting --api-key entirely when the env var is empty, rather than exiting - the previous hard-exit was silently hidden behind the workflow's continue-on-error, so every fork PR lost its upload with no visible failure. Retiring ci_skip_boards let CircleCI's plain family sweep reach five rp2040 boards for the first time; two didn't build. pico_sdk fails dual/host_hid_to_device_cdc and dual/host_info_to_device_cdc - a meta-board with no board.h pin defines of its own has no PICO_DEFAULT_PIO_USB_* macros, which a dual-role rp2040 example needs for its PIO-USB host side - so each example gained a skip.txt entry naming the failure. feather_rp2040_max3421 failed host/bare_api on -Wmissing-prototypes for max3421_int_handler in hw/bsp/rp2040/family.c; it is only ever used as a same-file GPIO IRQ callback (no other caller), so it is marked static rather than skip-listing a zero-risk one-line fix. All 8 rp2040 boards now build clean, `-b ` and the bare `rp2040` family sweep alike. drivers_coverage_check.py type-checks `drivers` before iterating: a string previously iterated character-by-character into nonsense "X matches no driver source file" errors, and null raised an unhandled TypeError. Not fixed, flagged for a follow-up: MEMBROWSE_API_KEY reaches membrowse_report.py as a child-process argv (visible in /proc//cmdline on the runner, even though the logged command line is redacted) rather than via the environment. Final-review fixes: the espressif Membrowse Upload docker run only forwarded MEMBROWSE_API_KEY and CI into the container, so membrowse's --github metadata detection (GITHUB_EVENT_NAME/GITHUB_SHA/...) and its fork-PR tokenless auth never saw the actual event - add GITHUB_ACTIONS/GITHUB_EVENT_NAME/GITHUB_SHA/GITHUB_REF_NAME plus a read-only bind mount of GITHUB_EVENT_PATH (containers don't inherit runner env by default; the esp Build step needs none of this, since its `--target all` never builds the *-membrowse/-upload custom targets, so its now-unused `pip install membrowse` is also dropped). check-paths' code filter also gains tools/membrowse_report.py, so a PR touching only that file exercises the real build+upload matrix (ci_select rule 2d) instead of 74 no-op --identical legs. Review fixes: generate_membrowse_sizes()'s all-examples scope (-b with no -e) now averages each file's size across the elfs it appears in instead of summing, matching metrics.py's compute_avg() semantics from the linkermap engine this branch replaced - a file linked by N examples no longer reports ~N times its real size, and the Flash/RAM columns are a real binary's size again; single-example (-e) stays byte-identical (sum, same as before). The Membrowse Upload step (build_util.yml) and hil-build-esp-identical's loop (build.yml) now strip any -e pairs hil_ci_set_matrix.py bakes into a PR-scoped hil-build-esp leg's args before invoking, so a PR-scoped run still uploads every example of the pinned board rather than only the PR-selected subset - closing exactly the silent history gap the step's own comment already warned against. Also: build.py's espressif branch gates the idf.py build path on 'all' being present in build_targets rather than being exactly ['all'], so a combined --target invocation still runs the build instead of silently skipping on a fresh dir; membrowse_compare.py's layout-based bucketing unions the actual flash/ram buckets of a multi-region section's regions instead of assuming any 2+-region split is a flash+ram pair (e.g. RAM_D1/RAM_D2 now stays ram-only); membrowse_report.py's extract_defsyms() dedupes to match its own docstring. Post-validate fixes, folded in. The Membrowse Upload step threads the PR's -e filter through for BOARD ELIGIBILITY only: without it, resolve_ci_boards() ran with examples=None and still resolved the pinned board even when the Build step above had fallen back to a substitute, uploading --identical for a board that run never built. Under --ci-pinned-boards-only tools/build.py now nulls the example filter after board selection, so the upload still touches every example of whichever board got resolved - real rows for what was built, --identical for the rest. same7x earns its pin instead of assuming it: a full family sweep showed five *_freertos examples failing (no FreeRTOSConfig.h wired for same7x - skip.txt'd as family:same7x, naming the reason) and same70_qmtech's board.cmake/mk pointing at a same70n19b_flash.ld that never existed in the tree (the SDK ships none) - the board now carries its own, Q21B's script with the N19B's MEMORY sizes (512K flash / 256K ram per same70n19b.h), like the other boards with a board-local .ld. Both same7x boards now build their full example sets green under the real budget. cxd56 stays OUT of ci_set_matrix's family_list and its BSP reverts to master untouched: the same sweep showed NuttX-header failures across three examples, spresense is unpinned anyway, and a -Wno-error suppression for a family no CI leg builds silences a real diagnostic class for nothing. drivers_coverage_check.py gains two guards: a pinned family that no CI toolchain actually compiles is now a validity error (the dead-pin case the same7x episode demonstrated), and a non-dict entry in "boards" reports one clear error instead of a traceback. The spec and plan docs are updated to the shipped names (membrowse-targets.json / --board-pins / --pins-only never shipped; 71 occurrences), and stale test fixtures naming the deleted tinyusb_metrics target are repointed at examples-membrowse-upload. Validate-loop fixes, folded in. membrowse_compare.py's report_for_elf() now passes the elf's own linker scripts and --defsym values (read from its ninja build graph, exactly as the CI upload path does) to `membrowse report`: with none given membrowse falls back to its default Code/Data regions, which the layout bucketing cannot map, so `.data` lost its flash-side load image and was counted RAM-only (verified on stm32h743eval: region 'Data' without the script, ['FLASH','RAM_D1'] with it). _bucket_from_layout() also unions in the flash side for a BOTH_SECTIONS section whose layout lists only its RAM region - pico-sdk's script never lists `.data` under FLASH, undercounting raspberry_pi_pico's flash by 11.2K. membrowse_onboard.py: `membrowse onboard` checks out and `git clean -fdx`s every historical commit in whatever directory it runs, which would detach HEAD and wipe the deps symlinks in the repo root - so the wrapper runs it in a disposable worktree under cmake-metrics/, and the composed build script relinks deps and reconfigures the (also wiped) build dir before every historical build, not just the first. --build-dirs now includes the example's own directory, so a change to its sources no longer backfills as --identical. Not fixable in the wrapper: onboard applies one commit's linker scripts to every commit in the range (no per-commit override in its CLI).

membrowse tooling: fail loudly, dedup espressif, bucket by memory layout

2026-09-06T08:57:00Z

metrics_compare_base.py caught nothing when the `membrowse` CLI was absent, so the default engine died with a bare traceback after both builds had already run; it now points at `pip install membrowse` or --engine linkermap, documented in the skills. membrowse_report.py's ninja query returned '' on failure, starving the linker-script and --defsym extraction so membrowse silently reported default Code/Data regions and exited 0; it now exits with the failing invocation, and likewise when a successful query yields no linker script without --ld. membrowse_onboard.py passes the extracted --ld-scripts/--def so a backfill matches what CI uploads. Stale bare --ci/--combined examples in the tool and in CLAUDE.md are corrected. espressif was compiled twice per run - once by hil-build-esp for the rig, once by the cmake job's esp-idf leg - so it leaves ci-boards.json, the cmake matrix and ci_set_matrix's family_list (which also removes a never-run CircleCI leg that would have built all 15 espressif boards), and hil-build-esp carries the membrowse upload instead. A separate job, gated on the opposite condition and with no downstream dependents, does the --identical upload on no-code-change pushes without letting the HIL test jobs reach the rig. Flash-vs-RAM bucketing stops guessing from section names. TinyUSB places code and buffers in RAM under names no prefix list can enumerate - .time_critical.tinyusb, NonCacheable, .fast, .ccmram - so RAM deltas read as 0 on exactly the boards that do it. Cross-reference each symbol's section against every memory_layout region's sections list instead: present in two regions means a flash load copy and a RAM run copy (both budgets), one region is classified by region name or, if that is unrecognized, by the ELF section type membrowse already derived from sh_flags. The name table survives as a documented fallback, extended with the vendor flash and RAM section names verified in this repo's linker scripts. Note membrowse 1.2.9 reports every linker-script-derived region's type as UNKNOWN, so the region name is the only region-level signal it surfaces.

ci-boards: one curated board list for CI, membrowse and HIL coverage

2026-09-06T08:57:00Z

The pinned-target file becomes .github/ci-boards.json - "pin" reads as GPIO pin in this repo - and with it the flags (--ci-boards, --ci-boards-only), the resolver (resolve_ci_boards()) and the JSON's own key (boards). It is now the only board-curation mechanism: tools/build.py's ci_skip_boards/ci_preferred_boards are gone, with samd2x_l2x and stm32h7 pinned to preserve today's CI picks, and resolve_ci_boards() applies the -e buildability filter so a scoped PR cannot pick a board that builds none of the selected examples and go green having compiled nothing. Curate the set itself: RP2350 boards (raspberry_pi_pico2, adafruit_fruit_jam for hcd_pio_usb) replace feather_rp2040_max3421, add max32666fthr, align the lpc55/nrf pins with rig boards, and drop the nuvoton trio, msp432e4, broadcom_64bit, f1c100s, gd32vf103 and mm32 - dwc2 stays covered, the rest move to uncovered with reasons. The checker moves to tools/drivers_coverage_check.py and reports coverage for both consumers without failing on gaps: membrowse boards (documented uncovered -> INFO, undocumented -> WARNING) and the HIL rosters (INFO), mapping boards to drivers through ci_select's rule-3/4 machinery including its role filter, so an hcd driver only counts host-role rig boards. Validity errors - unknown names, a driver both pinned and uncovered, an hcd claim on a board with no host example - still exit 1. Its chdir now scopes both cwd-relative calls, so the checker works from any directory, and the hook watches the inputs it actually reads.

metrics: replace the linkermap CI pipeline with membrowse

2026-09-06T08:57:00Z

Membrowse becomes TinyUSB's size-analytics system, in CI and locally. .github/membrowse-targets.json names the boards CI builds per family and the drivers each covers; a checker enforced by pre-commit fails when a dcd/hcd driver (plus ehci/ohci) is neither claimed nor documented in `uncovered`, validates board and family names against hw/bsp, and requires a board claiming an hcd driver to build at least one host/dual example. tools/build.py resolves that file into the build matrix and gates uploads on it. Locally, tools/membrowse_compare.py diffs `membrowse report --json --all-symbols` per source file, keyed on object_file (membrowse 1.2.9 truncates source_file to a basename), and metrics_compare_base.py uses it as the default engine, with `--engine linkermap` as the legacy map.json fallback (still required for --combined/--ci). A guard errors instead of writing a silent all-zero table when the filters match nothing. linkermap stops running on every build: the POST_BUILD hook goes, the explicit -linkermap target and a new examples-linkermap aggregate stay, and family_add_linkermap() no-ops when the dep is not fetched. CI loses the whole linkermap pipeline - the code-metrics job, the sticky PR size comment, release metrics assets, per-family artifacts, metrics_pair_compare.py, the membrowse-onboard workflow and CircleCI's dead tinyusb_metrics lane. The cmake job builds the named boards (esp-idf included, membrowse wired into the IDF docker path via family_initialize_project and MEMBROWSE_LD_OVERRIDE) and uploads only those, keeping the --identical path for code-unchanged runs. ci_select follows: the size tooling no longer runs in any CI build, while the board list decides what a family's legs compile and so forces a full matrix. Adds the espressif, same7x, cxd56 and f1c100s families the board list needs, each build-verified. The upload itself moves out of cmake: family_add_membrowse()'s ~90-line inline bash becomes tools/membrowse_report.py, carrying ninja linker-script extraction with recursive INCLUDE resolution, --defsym collection, map-file detection, --identical fallback and upload composition. The API key is read from the environment at run time, passed via argv and redacted in logs - nothing is baked into build.ninja at configure time, which the CMake-expanded $ENV{MEMBROWSE_API_KEY} used to do. tools/membrowse_onboard.py wraps `membrowse onboard` for history backfill, deriving the build script, elf path and the CI-matching target name (/) so a backfill cannot land under a name CI never uploads to; it refuses to start on a dirty worktree, since onboard checks out past commits in place. A membrowse skill documents local reports, size diffs, board-list maintenance, uploads and backfill; the code-size skill records the new default engine and its linkermap fallback. Three follow-up handoffs capture what was deliberately split out: full linkermap removal, membrowse --combined support, and the dead build_filtered plumbing.

ci: run the Claude review and @claude jobs on Opus 5

2026-09-03T21:42:29Z

ci: one Claude review per PR at a time, at effort high

2026-09-03T21:38:49Z

A new push now cancels the stale in-flight review instead of stacking a second run on the same PR. Effort max on every synchronize is the only place the routine review needs the top setting; high is the documented default. Drop the transcript dump that was marked temporary.