tinyusb.git, branch hil-report

docs: design, plan and follow-ups for the hil_report consolidation

2026-08-21T15:25:13Z

Records why the report code was spread across three modules and what the consolidation buys, so the next reader does not have to re-derive it from the diff. Names the new functions explicitly rather than presenting the change as pure code motion -- that framing points reviewers away from the code that carried the defects. Three findings this PR deliberately does not close get one handoff each, per CLAUDE.md: the worker-result tuple hil_report still unpacks positionally, SKILL.md's no-boards rule drifting from the code, and write_report's two non-atomic writes. Drops the pr3836 handoff, which this branch implements.

hil_ci: upload the sidecar so a remote --accumulate has a merge base

2026-08-21T15:25:13Z

hil_ci.sh rm -rf's REMOTE_DIR at the start of every run, and accumulate_report merges onto the sidecar in the run's cwd -- so a remote retry found nothing, silently started fresh, and its one-row table REPLACED the full-fleet one it was meant to extend. The copy-back at the end of the script has always existed; this is the other half of it. Verified on the rig end to end, which is the only place this shows up: a fresh run of stm32f407disco, then a retry spelled `-av`, and the merged sidecar holds all three rows where it previously held only nanoch32v203's two. Gated on --accumulate rather than unconditional: a fresh run unlinks the sidecar anyway. argparse decides whether the flag is present rather than a case arm -- hil_test.py declares `-a, --accumulate`, so `-av`, `-va`, `--accum` and `--acc` all mean it, and hil-validate.js tells operators to retry "adding -v". A missing merge base warns loudly instead of failing silently, since the run succeeds either way and a mistyped retry quietly shrinking the published table is the actual damage. The upload also checks provenance: hil_report.json is not namespaced by CONFIG or REMOTE, so a run against another rig leaves a sidecar behind that would otherwise merge in, publishing boards that never ran here. The copy-back fetches both halves to temps and commits them as a pair, or neither. It used to delete both before scps that are allowed to fail silently, so an ssh drop at the end of a 60-minute run destroyed the report and the next retry's merge base; copying them independently was no better, since a markdown that arrived beside a sidecar that did not left the local pair failing the rendering invariant. .failed is not uploaded: hil_test.py only ever writes it, never reads it. The retry spec already reaches the rig as the -b/-bt arguments the caller expanded from it.

hil_report: one module owns hil_report.json and hil_report.md

2026-08-21T15:25:13Z

The markdown IS a rendering of the sidecar now: every writer goes through render_report(), so a table can never contain something the JSON does not. Four writers previously composed it independently and three wrote no JSON at all -- and those three are exactly the paths where the run died, so a JSON consumer saw nothing on the runs that mattered most. The per-board verdicts an agent hands back reported the whole fleet as "no report row" while a human reading the markdown saw the real story. The document gains the two fields the markdown carried but the JSON did not: `scope` (a three-board PR run and a full run that lost 24 boards were indistinguishable) and `caveat` (how the run ENDED -- abandoned, aborted, no boards). `banner` keeps its existing meaning: the rig-health conditions the cells were collected under. The distinction is load-bearing, because banner carries across an --accumulate retry and caveat must not; conflating them made a clean retry publish an abandonment that never happened, and let a stale notice from an earlier attempt silence a genuinely new one. Consolidating into helper/hil_report.py is what makes that complete. The renderer, the writers, the merge and the fold to per-board verdicts live in one module that hil_test.py and hil_health.py both import. That dissolves the import cycle which forced write_timeout_report to compose its own markdown -- the pool-guard fallback renders like everyone else, so all five writers are byte-identical -- and removes the second copy of the cell classifier, which hil_summary.py's docstring described as "the EXACT classifier hil_test.py's own tally uses". Two copies of one rule, kept in sync by hand against re-typed emoji literals: change REPORT_CELL and the human's table and the agent's verdict silently disagree. hil_summary.py is deleted; its CLI moves here and the two harness docs that invoke it by name follow. `caveat` gates the workflow verdict and is required by its schema, because on the abandon path every row can legitimately pass while hil_test.py exits non-zero, and an operator omitting the field would silently disable the gate. NOT purely code motion, and worth reading as new code: measured against master, hil_test.py held only render_matrix and accumulate_report. render_report, write_report, mark_report_abandoned, mark_report_no_boards, _load, cell_state and the scope/caveat plumbing are new, and three rounds of review found their defects there. Each was reproduced before being fixed and is pinned by a test: a stuck board that already had a row got no pool-timeout cell and summarized as a pass; a stale board-locked cell masked it, so a board that wedged the rig was published as LOCKED and re-run; mark_report_abandoned republished the markdown even when it declined to stamp, inspected the wrong field, gave up on a missing or torn sidecar, and called the table "partial" against SKILL.md's contract that it IS this run's; write_report swallowed OSError, making two layers of fallback dead code, and committed the JSON before rendering; a sidecar with a null banner or a non-list rows killed a fully successful run with no artifact at all; an unhashable cell value raised on the normal accumulate path; the no-boards exit republished a previous run's rows, and its guard blocked even a fresh run. _load is the trust boundary for all of it, since hil_ci.sh uploads a sidecar as the --accumulate merge base and it is therefore untrusted input. A corrupt cell drops rather than being coerced to str, which would classify it as a pass. Verified on the rig: a 25-board fleet run, ten randomized passes mixing fresh and --accumulate over different board sets, and every containment path exercised against the module actually staged there.

test/hil: stop a machine-wide pgrep matching an unrelated sleep

2026-08-21T15:14:19Z

test_the_child_is_reaped_even_when_work_raises asserted that no 'sleep 20' survives, but pgrep -f searches every process on the host -- another agent session's retry loop was sleeping 20s between attempts, so the test failed 3/3 in isolation while that ran. It is a pre-commit hook, so it blocked every commit. A distinctive duration scopes it back to our own child. Unrelated to the report work.

Merge pull request #3841 from hathach/build-filter

2026-08-21T10:24:35Z

ci: scope the build matrix and HIL run to what a PR affects

examples: keep CFG_TUH_VENDOR 0 in tusb_config.h

2026-08-21T10:04:34Z

Removing the obsolete host vendor driver also dropped the `#define CFG_TUH_VENDOR 0` line from the six example configs that carried it. Put it back: host vendor is coming, and the configs are where a reader looks for the set of host classes an example can turn on. Restored byte-identical to the pre-removal state, each file keeping its own column alignment. The define is inert today - nothing under src/, hw/ or tools/ reads CFG_TUH_VENDOR - and it is 0 everywhere, so ci_select still reads the vendor class as enabled by no example and a change to it still selects nothing. Note the option's default in src/tusb_option.h is still gone; implementing the driver will need that back alongside the usbh driver-table entry.

ci_set_matrix: fall open when no selected family builds anywhere

2026-08-21T09:08:25Z

family_list maps a family to the toolchains that build it, and seven hw/bsp families are in neither: cxd56, efm32, espressif, f1c100s, pic32mz, py32f0, same7x. Scoping to one of them intersected to nothing, so every toolchain key was [], every cmake leg skipped on `if: inputs.build-args != '[]'`, code-metrics took its no-metrics branch, and the PR went green from a build job that ran no compiler. The only signal was a stderr line nothing greps for. Not a coverage regression - master gave the same diff no compile coverage either, since none of the other families compiles same7x's board.h. What is new is that the gap used to be masked by the full matrix and is now the whole answer, and that green now means "ran no compiler" rather than "compiled 64 families". A selection whose families ALL miss is now unusable rather than empty: it prints UNSCOPED, which build.yml and .circleci/config.yml already grep to drop the build extras with it, and emits the full matrix. The two neighbouring cases keep their own answers - an explicit families: [] is still a legitimate nothing-selected, and a partial miss still scopes to the families that do build, noting the rest. The contract test pinned an exact count of fall-open markers, which this would have broken; it now pins the invariant (every message that emits the full matrix carries the marker) and was checked to still fail when a marker is removed. Also corrects the drift guard's note about espressif: hil-build-esp builds its boards by name, but that job is gated on repository_owner, so on a fork an espressif-only PR builds nowhere.

hil: express a board's always-on defines as a variant, dropping build.args

2026-08-21T07:23:40Z

The roster had two ways to pass a cmake -D to a board's build: `build.args`, applied to every variant, and `variant[].defines`, applied to one. They did the same thing, and only metro_m4_express used the first - for MAX3421_HOST=1, which is what makes it the one rig board that compiles hcd_max3421.c. A board whose define is always on now carries a single variant named after itself, which is exactly the shape `board.get('variant') or [{'name': name, 'flags': ''}]` already synthesises everywhere - so the build dir, the HIL report row and the variant-boundary handling are unchanged. raspberry_pi_pico has used that shape for its flags all along. Removes the BuildCfg type and the parallel code path from all four consumers: hil_test.build_board, hil_pool_check's two builders, hil_ci_set_matrix and ci_select.board_options. Verified: the hil-build matrix entry is byte-identical (`-b metro_m4_express -DMAX3421_HOST=1`), hil_test's build command is unchanged, ci_select still selects the board for a max3421 diff with MAX3421_HOST in its options, and a real build of dual/host_info_to_device_cdc and host/cdc_msc_hid on that board still compiles hcd_max3421.c.

ci: fix nine ways the selection under-selected or mismatched

2026-08-21T05:41:47Z

Every one of these dropped coverage silently - the worst failure mode here, because the PR still goes green. Found by review, each reproduced first. Selection rules: * class_macros derived the config macro from the class DIRECTORY, so a change to src/class/midi/midi2_device.c selected the midi_test examples (which do not compile it) and never examples/device/midi2_device (the only one that enables CFG_TUD_MIDI2, and the only one that does). The file's own macro is unioned in where it differs - union, never replace: over-selecting costs a build, under-selecting merges a break. * the ${FAMILY_MCUS} fallback added for espressif fired on any family whose _family_mcus came back empty, and _cmake_sets is if()-blind and keeps the FIRST definition - so mcx/frdm_mcxn947 answered MCXA15, a token six examples' skip.txt names, dropping 12 firmware images CMake builds. Limited now to families that never spell set(FAMILY_MCUS ...) at all. * lib_examples read only an example's top-level CMakeLists.txt/Makefile; host/msc_file_explorer_freertos names lib/embedded-cli in src/CMakeLists.txt and survived by luck. The whole example tree is scanned. (SEGGER_RTT and rt-thread still resolve to nothing: all three references sit inside a LOGGER=rtt guard no CI build sets - the documented ruling, not a miss.) * get_family_boards applied ci_skip_boards/ci_preferred_boards only under GITHUB_ACTIONS/CIRCLECI, so the selector answered differently on a laptop than on a runner; _prune_buildable forces CI semantics. Its one-board pick also abandoned the whole preferred list when entry one could not build the -e set, and asked skip_example without the build's -D tokens. * _config_enables and lib_examples still read with the locale encoding - under LC_ALL=C the selector tracebacked on three tracked tusb_config.h files. The whole selector and its suite run clean there now. Workflows: * the Membrowse Upload step omitted $EX_ARGS, but --one-first now picks the board from the -e set, so it configured a different, empty build dir and uploaded --identical for a board never compiled. It takes $EX_ARGS for the BOARD; the target stays the aggregate, which has no DEPENDS and still records every example. * blanking FAM_REGEX reset only build_filtered, leaving the build scoped while code-metrics took the UNSCOPED branch and diffed a 1-family run against the full averaged baseline. All three drop together now, as CircleCI's fall-open does. * CircleCI's EX_ARGS had no character screen and is used unquoted, and its code-metrics job still exit 1'd on an empty metrics set - which a scoped build makes a legitimate outcome. * a `ci-full` PR label now turns the scoping off for one PR. A selector bug under-selects silently, and without a label the only ways back to a full matrix are accidental. Performance, since the selector gates every other job: family.cmake texts are read once rather than per changed directory (a 6,000-file dep bump re-read 84 files 99,892 times) and _scrape_mcu is cached: 2.2s -> 0.29s there, 0.8s -> 0.33s on a class diff. Tests: a drift guard for hw/bsp families absent from ci_set_matrix.family_list (they select zero legs now, where they used to ride the full matrix); the rule-4 port test asserted a SUBSET, which set() satisfies, so it could not fail on the empty selection it exists to catch; the GITHUB_ENV guard test counted a SUM of two guards. Drops metrics.py's --only-examples, which nothing called, and applies the TOTAL scrub to the by-example branch that skipped it.

docs: record the CI selection design and its plan

2026-08-21T04:09:03Z

The binding rule table (17 rows x 3 answer columns), the measured effect per PR shape, and the reasoning behind the parts that look surprising: why empty means empty, why hw/mcu and lib are rules rather than full-matrix paths, why get_deps.py is diffed as data, and which build system is the reference. The plan is the task-by-task record of how it was built, kept as the origin trail.