| Age | Commit message (Collapse) | Author |
|---|
|
fixes
The cmake job builds and uploads every pinned board to membrowse on
every run, PRs and pushes alike (--identical when code did not
change) - it is the single owner of that upload, as before this
branch touched it.
Curate the board set itself. cxd56/spresense, da1469x/da1469x_dk_pro,
fomu/fomu and ft9xx/mm900evxb are no longer pinned CI boards: none sit
on either HIL roster and no other CI board carries their driver, so
their size tracking was build-only with no hardware behind it -
documented with honest `uncovered` reasons instead of silently
dropping coverage. Swap lpc15/lpcxpresso1549 -> lpc11/lpcxpresso11u68
for dcd_lpc_ip3511, lpc17/lpcxpresso1769 -> lpc40/ea4088_quickstart
for dcd_lpc17_40+ohci, and samd5x_e5x/same54_xplained ->
metro_m4_express; adafruit_fruit_jam now also claims dcd_rp2040. All
four changed boards build-verified (device/cdc_msc, cmake), and the
rp2040 group's (family, board) sort order restored. Two stale notes
the swap left behind are fixed: lpcxpresso11u68 is on neither roster,
so its "HIL (htpc)" claim (copy-pasted from the entry it replaced) was
false, and ea4088_quickstart is a maintainer bench board noted the
same way frdm_kl25z and lpcxpresso1549 were.
Rename the board file to .github/ci-pinned-boards.json, with matching
--ci-pinned-boards/--ci-pinned-boards-only flags: it pins the board
that represents each family in CI, and membrowse history is keyed on
each pinned board's <board>/<example> target names, so the set is
explicitly held rather than derived. Content and target names are
unchanged; the per-entry "family" key is dropped - board names are
unique across hw/bsp/*/boards, so it was redundant data that could
drift from the tree. build.py's resolve_ci_boards() and the coverage
checker derive the family from the board dir, and entries are sorted
by board name, the only key left to sort on.
espressif builds have no leg in the cmake job (esp-idf is by far the
slowest toolchain, and its boards are already enumerated by name for
the rig) - hil-build-esp keeps its own upload-membrowse + secrets:
inherit instead, and hil-build-esp-identical covers the --identical
row for a no-code-change push, the same gap hil-build-esp's own
code-changed gate would otherwise leave. build_util.yml's Membrowse
Upload step skips any leg whose args carry --build-name, an extra -D
or --cflag - a HIL variant build - since it configures with the same
-DBOARD=<name> as the plain build and would otherwise fight it for
the same upload target name; this protects hil-build-esp's DMA
variants specifically. The step also does not scope by $EX_ARGS
(kept on the Build step): scoping the upload by the PR's example
filter would leave examples outside that filter with no upload
attempt at all - a silent history gap rather than the --identical row
the design intends.
Fork PRs: GHA withholds secrets, so MEMBROWSE_API_KEY reaches
membrowse_report.py empty. Master's CMake-expanded bare --api-key
fell through to membrowse's GitHub tokenless auth; the wrapper now
does the same by omitting --api-key entirely when the env var is
empty, rather than exiting - the previous hard-exit was silently
hidden behind the workflow's continue-on-error, so every fork PR
lost its upload with no visible failure.
Retiring ci_skip_boards let CircleCI's plain family sweep reach five
rp2040 boards for the first time; two didn't build. pico_sdk fails
dual/host_hid_to_device_cdc and dual/host_info_to_device_cdc - a
meta-board with no board.h pin defines of its own has no
PICO_DEFAULT_PIO_USB_* macros, which a dual-role rp2040 example needs
for its PIO-USB host side - so each example gained a skip.txt entry
naming the failure. feather_rp2040_max3421 failed host/bare_api on
-Wmissing-prototypes for max3421_int_handler in hw/bsp/rp2040/family.c;
it is only ever used as a same-file GPIO IRQ callback (no other caller),
so it is marked static rather than skip-listing a zero-risk one-line
fix. All 8 rp2040 boards now build clean, `-b <board>` and the bare
`rp2040` family sweep alike.
drivers_coverage_check.py type-checks `drivers` before iterating: a
string previously iterated character-by-character into nonsense "X
matches no driver source file" errors, and null raised an unhandled
TypeError.
Not fixed, flagged for a follow-up: MEMBROWSE_API_KEY reaches
membrowse_report.py as a child-process argv (visible in
/proc/<pid>/cmdline on the runner, even though the logged command line
is redacted) rather than via the environment.
Final-review fixes: the espressif Membrowse Upload docker run only
forwarded MEMBROWSE_API_KEY and CI into the container, so membrowse's
--github metadata detection (GITHUB_EVENT_NAME/GITHUB_SHA/...) and its
fork-PR tokenless auth never saw the actual event - add
GITHUB_ACTIONS/GITHUB_EVENT_NAME/GITHUB_SHA/GITHUB_REF_NAME plus a
read-only bind mount of GITHUB_EVENT_PATH (containers don't inherit
runner env by default; the esp Build step needs none of this, since
its `--target all` never builds the *-membrowse/-upload custom
targets, so its now-unused `pip install membrowse` is also dropped).
check-paths' code filter also gains tools/membrowse_report.py, so a PR
touching only that file exercises the real build+upload matrix
(ci_select rule 2d) instead of 74 no-op --identical legs.
Review fixes: generate_membrowse_sizes()'s all-examples scope (-b with no -e)
now averages each file's size across the elfs it appears in instead of
summing, matching metrics.py's compute_avg() semantics from the linkermap
engine this branch replaced - a file linked by N examples no longer reports
~N times its real size, and the Flash/RAM columns are a real binary's size
again; single-example (-e) stays byte-identical (sum, same as before). The
Membrowse Upload step (build_util.yml) and hil-build-esp-identical's loop
(build.yml) now strip any -e <example> pairs hil_ci_set_matrix.py bakes into
a PR-scoped hil-build-esp leg's args before invoking, so a PR-scoped run
still uploads every example of the pinned board rather than only the
PR-selected subset - closing exactly the silent history gap the step's own
comment already warned against.
Also: build.py's espressif branch gates the idf.py build path on 'all' being
present in build_targets rather than being exactly ['all'], so a combined
--target invocation still runs the build instead of silently skipping on a
fresh dir; membrowse_compare.py's layout-based bucketing unions the actual
flash/ram buckets of a multi-region section's regions instead of assuming
any 2+-region split is a flash+ram pair (e.g. RAM_D1/RAM_D2 now stays
ram-only); membrowse_report.py's extract_defsyms() dedupes to match its own
docstring.
Post-validate fixes, folded in. The Membrowse Upload step threads the
PR's -e filter through for BOARD ELIGIBILITY only: without it,
resolve_ci_boards() ran with examples=None and still resolved the
pinned board even when the Build step above had fallen back to a
substitute, uploading --identical for a board that run never built.
Under --ci-pinned-boards-only tools/build.py now nulls the example
filter after board selection, so the upload still touches every
example of whichever board got resolved - real rows for what was
built, --identical for the rest.
same7x earns its pin instead of assuming it: a full family sweep
showed five *_freertos examples failing (no FreeRTOSConfig.h wired
for same7x - skip.txt'd as family:same7x, naming the reason) and
same70_qmtech's board.cmake/mk pointing at a same70n19b_flash.ld
that never existed in the tree (the SDK ships none) - the board now
carries its own, Q21B's script with the N19B's MEMORY sizes (512K
flash / 256K ram per same70n19b.h), like the other boards with a
board-local .ld. Both same7x boards now build their full example sets
green under the real budget. cxd56 stays OUT of
ci_set_matrix's family_list and its BSP reverts to master untouched:
the same sweep showed NuttX-header failures across three examples,
spresense is unpinned anyway, and a -Wno-error suppression for a
family no CI leg builds silences a real diagnostic class for nothing.
drivers_coverage_check.py gains two guards: a pinned family that no
CI toolchain actually compiles is now a validity error (the dead-pin
case the same7x episode demonstrated), and a non-dict entry in
"boards" reports one clear error instead of a traceback. The spec and
plan docs are updated to the shipped names (membrowse-targets.json /
--board-pins / --pins-only never shipped; 71 occurrences), and stale
test fixtures naming the deleted tinyusb_metrics target are repointed
at examples-membrowse-upload.
Validate-loop fixes, folded in. membrowse_compare.py's report_for_elf()
now passes the elf's own linker scripts and --defsym values (read from
its ninja build graph, exactly as the CI upload path does) to
`membrowse report`: with none given membrowse falls back to its default
Code/Data regions, which the layout bucketing cannot map, so `.data`
lost its flash-side load image and was counted RAM-only (verified on
stm32h743eval: region 'Data' without the script, ['FLASH','RAM_D1']
with it). _bucket_from_layout() also unions in the flash side for a
BOTH_SECTIONS section whose layout lists only its RAM region -
pico-sdk's script never lists `.data` under FLASH, undercounting
raspberry_pi_pico's flash by 11.2K.
membrowse_onboard.py: `membrowse onboard` checks out and `git clean
-fdx`s every historical commit in whatever directory it runs, which
would detach HEAD and wipe the deps symlinks in the repo root - so the
wrapper runs it in a disposable worktree under cmake-metrics/, and the
composed build script relinks deps and reconfigures the (also wiped)
build dir before every historical build, not just the first.
--build-dirs now includes the example's own directory, so a change to
its sources no longer backfills as --identical. Not fixable in the
wrapper: onboard applies one commit's linker scripts to every commit in
the range (no per-commit override in its CLI).
|
|
metrics_compare_base.py caught nothing when the `membrowse` CLI was
absent, so the default engine died with a bare traceback after both
builds had already run; it now points at `pip install membrowse` or
--engine linkermap, documented in the skills. membrowse_report.py's
ninja query returned '' on failure, starving the linker-script and
--defsym extraction so membrowse silently reported default Code/Data
regions and exited 0; it now exits with the failing invocation, and
likewise when a successful query yields no linker script without --ld.
membrowse_onboard.py passes the extracted --ld-scripts/--def so a
backfill matches what CI uploads. Stale bare --ci/--combined examples
in the tool and in CLAUDE.md are corrected.
espressif was compiled twice per run - once by hil-build-esp for the
rig, once by the cmake job's esp-idf leg - so it leaves ci-boards.json,
the cmake matrix and ci_set_matrix's family_list (which also removes a
never-run CircleCI leg that would have built all 15 espressif boards),
and hil-build-esp carries the membrowse upload instead. A separate job,
gated on the opposite condition and with no downstream dependents, does
the --identical upload on no-code-change pushes without letting the HIL
test jobs reach the rig.
Flash-vs-RAM bucketing stops guessing from section names. TinyUSB
places code and buffers in RAM under names no prefix list can
enumerate - .time_critical.tinyusb, NonCacheable, .fast, .ccmram - so
RAM deltas read as 0 on exactly the boards that do it. Cross-reference
each symbol's section against every memory_layout region's sections
list instead: present in two regions means a flash load copy and a RAM
run copy (both budgets), one region is classified by region name or, if
that is unrecognized, by the ELF section type membrowse already derived
from sh_flags. The name table survives as a documented fallback,
extended with the vendor flash and RAM section names verified in this
repo's linker scripts. Note membrowse 1.2.9 reports every
linker-script-derived region's type as UNKNOWN, so the region name is
the only region-level signal it surfaces.
|
|
The pinned-target file becomes .github/ci-boards.json - "pin" reads as
GPIO pin in this repo - and with it the flags (--ci-boards,
--ci-boards-only), the resolver (resolve_ci_boards()) and the JSON's
own key (boards). It is now the only board-curation mechanism:
tools/build.py's ci_skip_boards/ci_preferred_boards are gone, with
samd2x_l2x and stm32h7 pinned to preserve today's CI picks, and
resolve_ci_boards() applies the -e buildability filter so a scoped PR
cannot pick a board that builds none of the selected examples and go
green having compiled nothing.
Curate the set itself: RP2350 boards (raspberry_pi_pico2,
adafruit_fruit_jam for hcd_pio_usb) replace feather_rp2040_max3421, add
max32666fthr, align the lpc55/nrf pins with rig boards, and drop the
nuvoton trio, msp432e4, broadcom_64bit, f1c100s, gd32vf103 and mm32 -
dwc2 stays covered, the rest move to uncovered with reasons.
The checker moves to tools/drivers_coverage_check.py and reports
coverage for both consumers without failing on gaps: membrowse boards
(documented uncovered -> INFO, undocumented -> WARNING) and the HIL
rosters (INFO), mapping boards to drivers through ci_select's rule-3/4
machinery including its role filter, so an hcd driver only counts
host-role rig boards. Validity errors - unknown names, a driver both
pinned and uncovered, an hcd claim on a board with no host example -
still exit 1. Its chdir now scopes both cwd-relative calls, so the
checker works from any directory, and the hook watches the inputs it
actually reads.
|
|
Membrowse becomes TinyUSB's size-analytics system, in CI and locally.
.github/membrowse-targets.json names the boards CI builds per family
and the drivers each covers; a checker enforced by pre-commit fails
when a dcd/hcd driver (plus ehci/ohci) is neither claimed nor
documented in `uncovered`, validates board and family names against
hw/bsp, and requires a board claiming an hcd driver to build at least
one host/dual example. tools/build.py resolves that file into the
build matrix and gates uploads on it.
Locally, tools/membrowse_compare.py diffs `membrowse report --json
--all-symbols` per source file, keyed on object_file (membrowse 1.2.9
truncates source_file to a basename), and metrics_compare_base.py uses
it as the default engine, with `--engine linkermap` as the legacy
map.json fallback (still required for --combined/--ci). A guard errors
instead of writing a silent all-zero table when the filters match
nothing. linkermap stops running on every build: the POST_BUILD hook
goes, the explicit <target>-linkermap target and a new
examples-linkermap aggregate stay, and family_add_linkermap() no-ops
when the dep is not fetched.
CI loses the whole linkermap pipeline - the code-metrics job, the
sticky PR size comment, release metrics assets, per-family artifacts,
metrics_pair_compare.py, the membrowse-onboard workflow and CircleCI's
dead tinyusb_metrics lane. The cmake job builds the named boards
(esp-idf included, membrowse wired into the IDF docker path via
family_initialize_project and MEMBROWSE_LD_OVERRIDE) and uploads only
those, keeping the --identical path for code-unchanged runs. ci_select
follows: the size tooling no longer runs in any CI build, while the
board list decides what a family's legs compile and so forces a full
matrix. Adds the espressif, same7x, cxd56 and f1c100s families the
board list needs, each build-verified.
The upload itself moves out of cmake: family_add_membrowse()'s ~90-line
inline bash becomes tools/membrowse_report.py, carrying ninja
linker-script extraction with recursive INCLUDE resolution, --defsym
collection, map-file detection, --identical fallback and upload
composition. The API key is read from the environment at run time,
passed via argv and redacted in logs - nothing is baked into
build.ninja at configure time, which the CMake-expanded
$ENV{MEMBROWSE_API_KEY} used to do.
tools/membrowse_onboard.py wraps `membrowse onboard` for history
backfill, deriving the build script, elf path and the CI-matching
target name (<board>/<basename>) so a backfill cannot land under a name
CI never uploads to; it refuses to start on a dirty worktree, since
onboard checks out past commits in place.
A membrowse skill documents local reports, size diffs, board-list
maintenance, uploads and backfill; the code-size skill records the new
default engine and its linkermap fallback. Three follow-up handoffs
capture what was deliberately split out: full linkermap removal,
membrowse --combined support, and the dead build_filtered plumbing.
|
|
support (#3853)
Promote SEGGER RTT from an inline debugging technique to a standalone skill
backed by one stdlib-only implementation in tools/rtt.py: a CLI and importable
module for console/capture over J-Link (RTTTelnetPort) and OpenOCD (rtt server)
probes, with probe selection by serial or VID:PID, control-block address via
--elf or --addr, bidirectional console, post-mortem ring dump, and
--reset-before-attach for boot-time capture. The HIL harness reads a board's
console over RTT when its probe has no VCOM ("logger": "rtt" plus a LOGGER=rtt
variant define), covering device_info, pool-check aliveness, and CI wiring.
Validated on 22 boards across both backends; 26 unit tests run in pre-commit.
|
|
ci: an empty selection must build nothing, plus selector follow-ups
A PR whose build axis legitimately selected nothing rebuilt everything.
build.yml reads .build.families twice - as a |-joined regex, and implicitly
as "is anything selected" - but tested only -z "$FAMILY_REGEX", which an
empty list and a charset-rejected one both satisfy while meaning opposite
things. ci_set_matrix had already returned the correct all-empty matrix;
the fall-open branch discarded it. #3842 and #3840 each spent 74 cmake legs
on it. Branch on the two cases instead, rename FAM_* to FAMILY_*, and cover
the block with a test that extracts it from build.yml and executes it - it
had no test at all, which is how this shipped through two merges.
Follow-ups to the same machinery: glob.escape the repo root at five sites,
so a checkout path containing [ or * stops failing closed; drop the ci-full
label, read after the matrix was already computed and so never functional;
delete 13 mcu:MKL25ZXX / mcu:SAME5X skip tokens matching no board; carry the
rule table in the module docstring, guarded against drift; and pin six
selection behaviours a mutation pass proved untested.
Cut the selector's cost 1.8x (26.0s -> 14.6s) with 0 divergences over 260
paths, and stop scoping the membrowse upload by the PR example filter.
|
|
family_list maps a family to the toolchains that build it, and seven hw/bsp
families are in neither: cxd56, efm32, espressif, f1c100s, pic32mz, py32f0,
same7x. Scoping to one of them intersected to nothing, so every toolchain key was
[], every cmake leg skipped on `if: inputs.build-args != '[]'`, code-metrics took
its no-metrics branch, and the PR went green from a build job that ran no
compiler. The only signal was a stderr line nothing greps for.
Not a coverage regression - master gave the same diff no compile coverage either,
since none of the other families compiles same7x's board.h. What is new is that
the gap used to be masked by the full matrix and is now the whole answer, and
that green now means "ran no compiler" rather than "compiled 64 families".
A selection whose families ALL miss is now unusable rather than empty: it prints
UNSCOPED, which build.yml and .circleci/config.yml already grep to drop the build
extras with it, and emits the full matrix. The two neighbouring cases keep their
own answers - an explicit families: [] is still a legitimate nothing-selected, and
a partial miss still scopes to the families that do build, noting the rest.
The contract test pinned an exact count of fall-open markers, which this would
have broken; it now pins the invariant (every message that emits the full matrix
carries the marker) and was checked to still fail when a marker is removed.
Also corrects the drift guard's note about espressif: hil-build-esp builds its
boards by name, but that job is gated on repository_owner, so on a fork an
espressif-only PR builds nowhere.
|
|
The roster had two ways to pass a cmake -D to a board's build: `build.args`,
applied to every variant, and `variant[].defines`, applied to one. They did the
same thing, and only metro_m4_express used the first - for MAX3421_HOST=1, which
is what makes it the one rig board that compiles hcd_max3421.c.
A board whose define is always on now carries a single variant named after itself,
which is exactly the shape `board.get('variant') or [{'name': name, 'flags': ''}]`
already synthesises everywhere - so the build dir, the HIL report row and the
variant-boundary handling are unchanged. raspberry_pi_pico has used that shape
for its flags all along.
Removes the BuildCfg type and the parallel code path from all four consumers:
hil_test.build_board, hil_pool_check's two builders, hil_ci_set_matrix and
ci_select.board_options.
Verified: the hil-build matrix entry is byte-identical
(`-b metro_m4_express -DMAX3421_HOST=1`), hil_test's build command is unchanged,
ci_select still selects the board for a max3421 diff with MAX3421_HOST in its
options, and a real build of dual/host_info_to_device_cdc and host/cdc_msc_hid on
that board still compiles hcd_max3421.c.
|
|
Every PR built all 74 legs (2494 example builds on GHA cmake alone) and flashed
all 30 rig boards, whatever it touched. One classifier now walks the PR diff twice
and answers three questions: which families to build, which examples per family,
and which boards run which tests. Fail-open throughout - anything no rule
classifies, any exception, any unusable output falls back to the full matrix, and
a master push always builds everything.
test/hil/helper/hil_select.py moves to tools/ci_select.py: it is no longer HIL-only,
and tools/ is where the build side can import it. test_hil_select.py follows it as
test_ci_select.py.
Rules (docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md holds the
full table): a port selects the families whose family.cmake references it, and its
role - a dcd change skips host examples and vice versa; a class selects only the
examples whose tusb_config.h enables its CFG_TU[DH]_ macro, following cross-class
includes; an example selects itself; hw/bsp selects its family or board; hw/mcu and
lib select whoever references them. CMake is the reference for all of it - make
follows whatever cmake decides, family.mk is never scanned.
Empty means empty (maintainer ruling): a rule that classifies a path to nothing
selects nothing. Ports no family references, classes no config enables, libs no
example builds and hw/mcu paths that resolve nowhere are all real - nothing
compiles them, so nothing can validate them, and the master-push build is the net.
Structural tests pin each such case with an explicit allowlist, so the day one
stops being empty it fails pre-commit instead of silently narrowing CI.
Per-example builds: build.py grows a repeatable -e, resolved against the targets
CMake actually registered and batched into one `cmake --build --target a b c`.
build_utils mirrors CMake's family_filter (the whole FAMILY_MCUS list, ${...} and
string(TOUPPER ...) resolved) for the cmake side, while the make side keeps
master's algorithm verbatim - the two build systems answer differently and a shared
answer breaks lpc54's make link. hil-build gains this even on a full selection:
1702 example builds become 515.
Transport: the selection travels as a file, never an argv or env var - a mass-sweep
diff selects 261 KB against a 128 KiB exec limit, and E2BIG would fail the step
before its own fallback could run. CircleCI carries the example map inside the
generated config (pipeline parameters cap at 512 chars), swapped into the parameter
defaults by sentinel match, and drops the scoping wholesale if that rewrite fails.
Every PR-derived value written to $GITHUB_ENV/$GITHUB_OUTPUT is character-screened.
Code metrics follow the scoping: metrics.py emits per-example totals, and
metrics_pair_compare compares the (board, example) pairs present on both sides
instead of a scoped run against a full-matrix average.
The selector's own suite gates it in both providers: a selector that exits 0 with
valid-but-wrong JSON is the one failure fail-open cannot catch, so a red suite
means the full matrix.
|
|
A wedged USB device used to take the whole HIL run with it. Every worker that
touched the poisoned node blocked uninterruptibly, the pool could not be joined,
map_async discarded every board's result, and the job ran to the GitHub ceiling
with no report at all -- while the self-hosted runner's single job slot stayed
occupied and every queued job waited behind it.
Bound the calls a worker makes itself. read_sysfs, bounded_open and run_cmd all
answer within a wall clock; read_sysfs distinguishes "absent" from "unknown",
because a blocked read is not evidence of absence, and caps stranded readers at
four (each costs a thread and an fd for the life of the process) after which the
worker declares itself blind. mtype, the gio unmount, the libmtp session and the
arecord/iperf reaps go through those bounds; the MTP session runs in a disposable
subprocess, since libmtp's ctypes calls block unkillably in D state.
Bound the run. A pool guard (HIL_POOL_TIMEOUT, 60 min) fires before any job
ceiling and still writes a report. When the pool will not shut down, the sweep
kills what the workers spawned -- descendants, not just direct children, since
flashers run in their own session -- confirms each kill actually landed, and
exits early so the runner is freed. Whatever survived is named in the report.
Deliberately shallow past that point. We do not re-scan process groups, prove
pid ownership, or escalate through sudo: a root-owned survivor is reported, not
force-killed, because signalling a pid we cannot prove is ours is the worse
failure, and the job ceiling backstops whatever this misses. A D-state holder
was never killable anyway.
Recover instead of reporting a wedge. A HUNG usbtest case reflashes its own DUT
through its roster flasher, but only where the flasher can reach its probe past
a poisoned node -- openocd pinned to a validated vid_pid, or esptool. Where it
cannot, the run says so rather than reserving budget for a path that cannot fire.
Raise the CI ceilings above the pool guard so the guard fires first and still
writes its report, and pin --retry 1 on every HIL leg: the guard is a flat
constant and does not scale with max_retry, so argparse's default of 3 would
triple the serialized usbtest tail against an unchanged guard.
Split the module: execution in hil_test/hil_flash/usbtest, infrastructure in
helper/ (locking, health, selection, shared bounded IO), and the two matrix
generators into .github/scripts/ -- ci_set_matrix.py sat in workflows/, where
GitHub treats every file as a workflow definition. 193 tests cover the bounded
paths, the kill ladder, the guard and the selector against synthetic /proc trees
and PATH-injected fakes; a real wedge cannot be manufactured on demand.
|
