tinyusb.git/tools/build.py, branch hil-report

ci: fix nine ways the selection under-selected or mismatched

2026-08-21T05:41:47Z

Every one of these dropped coverage silently - the worst failure mode here, because the PR still goes green. Found by review, each reproduced first. Selection rules: * class_macros derived the config macro from the class DIRECTORY, so a change to src/class/midi/midi2_device.c selected the midi_test examples (which do not compile it) and never examples/device/midi2_device (the only one that enables CFG_TUD_MIDI2, and the only one that does). The file's own macro is unioned in where it differs - union, never replace: over-selecting costs a build, under-selecting merges a break. * the ${FAMILY_MCUS} fallback added for espressif fired on any family whose _family_mcus came back empty, and _cmake_sets is if()-blind and keeps the FIRST definition - so mcx/frdm_mcxn947 answered MCXA15, a token six examples' skip.txt names, dropping 12 firmware images CMake builds. Limited now to families that never spell set(FAMILY_MCUS ...) at all. * lib_examples read only an example's top-level CMakeLists.txt/Makefile; host/msc_file_explorer_freertos names lib/embedded-cli in src/CMakeLists.txt and survived by luck. The whole example tree is scanned. (SEGGER_RTT and rt-thread still resolve to nothing: all three references sit inside a LOGGER=rtt guard no CI build sets - the documented ruling, not a miss.) * get_family_boards applied ci_skip_boards/ci_preferred_boards only under GITHUB_ACTIONS/CIRCLECI, so the selector answered differently on a laptop than on a runner; _prune_buildable forces CI semantics. Its one-board pick also abandoned the whole preferred list when entry one could not build the -e set, and asked skip_example without the build's -D tokens. * _config_enables and lib_examples still read with the locale encoding - under LC_ALL=C the selector tracebacked on three tracked tusb_config.h files. The whole selector and its suite run clean there now. Workflows: * the Membrowse Upload step omitted $EX_ARGS, but --one-first now picks the board from the -e set, so it configured a different, empty build dir and uploaded --identical for a board never compiled. It takes $EX_ARGS for the BOARD; the target stays the aggregate, which has no DEPENDS and still records every example. * blanking FAM_REGEX reset only build_filtered, leaving the build scoped while code-metrics took the UNSCOPED branch and diffed a 1-family run against the full averaged baseline. All three drop together now, as CircleCI's fall-open does. * CircleCI's EX_ARGS had no character screen and is used unquoted, and its code-metrics job still exit 1'd on an empty metrics set - which a scoped build makes a legitimate outcome. * a `ci-full` PR label now turns the scoping off for one PR. A selector bug under-selects silently, and without a label the only ways back to a full matrix are accidental. Performance, since the selector gates every other job: family.cmake texts are read once rather than per changed directory (a 6,000-file dep bump re-read 84 files 99,892 times) and _scrape_mcu is cached: 2.2s -> 0.29s there, 0.8s -> 0.33s on a class diff. Tests: a drift guard for hw/bsp families absent from ci_set_matrix.family_list (they select zero legs now, where they used to ride the full matrix); the rule-4 port test asserted a SUBSET, which set() satisfies, so it could not fail on the empty selection it exists to catch; the GITHUB_ENV guard test counted a SUM of two guards. Drops metrics.py's --only-examples, which nothing called, and applies the TOTAL scrub to the by-example branch that skipped it.

ci: scope the build matrix and the HIL run to what a PR affects

2026-08-21T04:07:27Z

Every PR built all 74 legs (2494 example builds on GHA cmake alone) and flashed all 30 rig boards, whatever it touched. One classifier now walks the PR diff twice and answers three questions: which families to build, which examples per family, and which boards run which tests. Fail-open throughout - anything no rule classifies, any exception, any unusable output falls back to the full matrix, and a master push always builds everything. test/hil/helper/hil_select.py moves to tools/ci_select.py: it is no longer HIL-only, and tools/ is where the build side can import it. test_hil_select.py follows it as test_ci_select.py. Rules (docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md holds the full table): a port selects the families whose family.cmake references it, and its role - a dcd change skips host examples and vice versa; a class selects only the examples whose tusb_config.h enables its CFG_TU[DH]_ macro, following cross-class includes; an example selects itself; hw/bsp selects its family or board; hw/mcu and lib select whoever references them. CMake is the reference for all of it - make follows whatever cmake decides, family.mk is never scanned. Empty means empty (maintainer ruling): a rule that classifies a path to nothing selects nothing. Ports no family references, classes no config enables, libs no example builds and hw/mcu paths that resolve nowhere are all real - nothing compiles them, so nothing can validate them, and the master-push build is the net. Structural tests pin each such case with an explicit allowlist, so the day one stops being empty it fails pre-commit instead of silently narrowing CI. Per-example builds: build.py grows a repeatable -e, resolved against the targets CMake actually registered and batched into one `cmake --build --target a b c`. build_utils mirrors CMake's family_filter (the whole FAMILY_MCUS list, ${...} and string(TOUPPER ...) resolved) for the cmake side, while the make side keeps master's algorithm verbatim - the two build systems answer differently and a shared answer breaks lpc54's make link. hil-build gains this even on a full selection: 1702 example builds become 515. Transport: the selection travels as a file, never an argv or env var - a mass-sweep diff selects 261 KB against a 128 KiB exec limit, and E2BIG would fail the step before its own fallback could run. CircleCI carries the example map inside the generated config (pipeline parameters cap at 512 chars), swapped into the parameter defaults by sentinel match, and drops the scoping wholesale if that rewrite fails. Every PR-derived value written to $GITHUB_ENV/$GITHUB_OUTPUT is character-screened. Code metrics follow the scoping: metrics.py emits per-example totals, and metrics_pair_compare compares the (board, example) pairs present on both sides instead of a scoped run against a full-matrix average. The selector's own suite gates it in both providers: a selector that exits 0 with valid-but-wrong JSON is the one failure fail-open cannot catch, so a red suite means the full matrix.

dwc2: fix EP0 OUT dcache invalidate range; run usbtest on espressif s3/p4 and mimxrt1015

2026-07-14T19:31:14Z

edpt_schedule_packets() advanced xfer->buffer past each armed EP0 chunk, so the OUT-complete handler invalidated the cache at the ADVANCED pointer: one line past the received data. The CPU then read stale cached bytes instead of the DMA'd packet, and the misplaced invalidate discarded a dirty line of whatever variable follows the buffer - random neighbor corruption on every control-OUT data stage. Found by usbtest ctrl_out (cases 14/21) on espressif_p4_function_ev with DMA enabled, the first DWC2 target combining buffer DMA with a data cache: usbd control state wedged after the first control write (every later request stalled), and one build layout panicked in the usbd memcpy with a wild pointer. Rework the EP0 chunk bookkeeping so xfer->buffer always points at the un-consumed position: the arm no longer advances it; instead the EP0 re-arm paths advance past each completed (full) chunk, invalidating it first on the OUT side. The final OUT completion invalidates exactly the received bytes of its last chunk, taken from DOEPDMA ("incremented on every AHB transaction", databook 7.1.83 - the same semantics the SETUP path relies on) before dma_setup_prepare() re-targets it. EP0 chunking state (ep0_pending) is now also dropped on bus reset and on a new SETUP, so a stale latched completion can no longer re-arm EP0 DMA from dead state. No behavior change for targets without dcache. While root-causing, the FIFO layout was cross-checked against the DWC2 databook/programming guide v4.20a: the existing GDFIFOCFG programming (EPInfoBaseAddr = otg_dfifo_depth - 2*ep_count, one SPRAM word per endpoint direction for buffer DMA) is conformant and needs no change; the P4 HS instance's reset GDFIFOCFG (0x03800400) merely reflects a scatter/gather-sized EP_LOC_CNT of 128 that buffer DMA does not need. With the fix in place, enable the usbtest battery on the espressif fleet: tools/build.py allowlists device/usbtest (a plain IDF component like board_test/video_capture) and both espressif boards' only-lists gain device/usbtest. Also re-enable device/usbtest on mimxrt1015_evk: its skip predated the dcd_ci_hs stale-ACTIVE-overlay fix (already on this branch), which cured the battery that previously killed the uPD720201 host controller twice (2026-07-11 ROM fw, 2026-07-13 case 27 on fw 2.0.2.6); rig-validated 30/30 three consecutive runs. Validated on rig (all 30/30): espressif_p4_function_ev(-DMA) (was 22/30 under DMA), espressif_s3_devkitm(-DMA), stm32f723disco(-DMA), mimxrt1015_evk; p4/s3 slave-mode unaffected (DMA-only code path); compile-checked stm32h743nucleo +TUD DMA, stm32f407disco, stm32l476disco (device ports currently on the dead hub). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_017TQZrFfU3K4Y198aLsUpBC

build.py: keep only raspberry_pi_pico in rp2040 ci_preferred_boards

2026-06-30T06:37:29Z

The one-board selector uses preferred_list[0] anyway, so pico is the board built. Co-Authored-By: Claude Opus 4.8 (1M context)

build.py: prefer pico/pico2 for rp2040 CI; skill: drop one-time 0.20.0 backfill note

2026-06-30T06:35:35Z

- ci_preferred_boards: rp2040 -> raspberry_pi_pico, raspberry_pi_pico2 so the one-board CI/metrics build uses a Pico (the first-alphabetical board failed). - make-release skill: the metrics backfill was a one-time 0.20.0 task, not a recurring release step; keep only the CI auto-compare note. Co-Authored-By: Claude Opus 4.8 (1M context)

HIL: replace build.flags_on with named build variants (#3687)

2026-06-11T01:16:43Z

* test/hil: replace build.flags_on with named variant schema Boards declare build variants as `variant: [{name, flags}]` instead of `build.flags_on`. The variant `name` is the build dir (cmake-build-) and the HIL report row; `flags` is the raw CFLAGS string (-D...=1) injected via CFLAGS_CLI. No `variant` => a single build named after the board. - build.py: --build-name (dir) + --cflag= (raw CFLAGS, repeatable, =form survives the matrix's shell word-splitting); drop -f1/CFLAGS wrapping. - hil_ci_set_matrix.py: emit one build arg per variant. - hil_test.py: iterate variants; report row + build dir = variant name. - hil_ci.sh: copy all cmake-build-* dirs for -b runs. - get_deps.py: accept (ignore) --build-name/--cflag from matrix args. - tinyusb.json: migrate all 6 flags_on boards to variant. * board_test: park CI build with busy spin instead of wfe

refactor build scripts to support multiple build targets and improve argument handling

2026-02-12T06:32:00Z

add build target argument to improve flexibility of build scripts and workflows

2026-02-11T09:46:27Z

membrowse-upload upload with --identical if elf file does not exist

add membrowse-upload target and use it in ci after build

2026-02-10T16:51:55Z

apply copilot suggestion

2026-01-09T04:01:58Z