| Age | Commit message (Collapse) | Author |
|---|
|
taking the rig (#3846)
test_the_upload_board_can_diverge_from_the_built_board called
get_family_boards without ci=True, so it pinned the developer's set, not the
runner's: the CI skip lists move the one-first pick on three families. It
held locally and went red on its first CI run. Pass ci=True, as
_prune_buildable already does, and pin the runner's twelve.
Rule 2 is a bare test/hil/ prefix, so the harness's own unit tests booked the
full 27-board rig for diffs that cannot reach it. Carve test/hil/test/** out
to rule 1b, beside test/{fuzz,unit-test}/**; the harness itself is untouched.
A test pins that directory's file list, so anything added there that the rig
does read fails rather than silently skipping hardware. Rule table updated in
the spec and its carbon in the docstring.
|
|
ci: an empty selection must build nothing, plus selector follow-ups
A PR whose build axis legitimately selected nothing rebuilt everything.
build.yml reads .build.families twice - as a |-joined regex, and implicitly
as "is anything selected" - but tested only -z "$FAMILY_REGEX", which an
empty list and a charset-rejected one both satisfy while meaning opposite
things. ci_set_matrix had already returned the correct all-empty matrix;
the fall-open branch discarded it. #3842 and #3840 each spent 74 cmake legs
on it. Branch on the two cases instead, rename FAM_* to FAMILY_*, and cover
the block with a test that extracts it from build.yml and executes it - it
had no test at all, which is how this shipped through two merges.
Follow-ups to the same machinery: glob.escape the repo root at five sites,
so a checkout path containing [ or * stops failing closed; drop the ci-full
label, read after the matrix was already computed and so never functional;
delete 13 mcu:MKL25ZXX / mcu:SAME5X skip tokens matching no board; carry the
rule table in the module docstring, guarded against drift; and pin six
selection behaviours a mutation pass proved untested.
Cut the selector's cost 1.8x (26.0s -> 14.6s) with 0 divergences over 260
paths, and stop scoping the membrowse upload by the PR example filter.
|
|
Anchor the .github file alternatives. FUNDING.yml, labeler.yml and
membrowse_pr_message.j2 sat inside a group whose only `$` belonged to the
workflows/ branch, so they matched as prefixes: .github/labeler.yml.bak and
.github/FUNDING.yml.old were classified as metadata and would have selected
nothing. No such file exists today - the workflows/ alternative was already
anchored and ISSUE_TEMPLATE/ is a directory prefix on purpose.
Rule 12b had no test of its own: TestNoTrackedFileIsUnclassified only proved
src/typec no longer reaches rule 17, not that the answer is right. TestTypecRule
pins it - non-full, every selected example under typec/, all four src/typec files
answering alike, no rig board, and the set derived from CFG_TUC_ENABLED rather
than hardcoded, so it follows a new typec example on its own. Verified all four
fail with rule 12b removed.
|
|
The eight lru_cache layers take repo-RELATIVE paths - 'hw/bsp/<fam>',
'examples/<ex>/skip.txt', the literal 'hw/bsp' glob - while ci_select._in_repo()
chdirs around every call so one process can classify more than one tree. With no
cwd in the key the second tree gets the first tree's answers.
Reproduced: skip_example('host/bare_api','metro_m0_express') is False at the repo
root and STILL False after chdir into a tree where that board does not exist; only
cache_clear() gave the right answer. It bites the code-size skill's base-vs-branch
worktree compare, /pre-pr, and the first test that points classify_build at a
fixture tree. Master had no caching here, so the hazard arrived with it.
_cwd_cache puts os.getcwd() in the key. The 199-test suite passed before only
because every test happens to pass the real REPO; the new
TestCachesAreKeyedOnTheTree crosses trees deliberately.
Also adds the drift guard the class rule was missing. Ports, hw/mcu, get_deps
tokens and bsp families each have one; the class rule had only a comment claiming
vendor_host.c was the sole "enabled by no example config" case until its removal -
which src/class/bth falsifies today. TestClassesWithNoEnablingExample pins the set
to {bth}, so a class added before its first example, or an example config flipped
to 0, fails here instead of silently selecting nothing on both axes. Verified it
fires by adding a class dir nothing enables.
|
|
Rule 17 (unclassified -> full on both axes) is the fail-open net for paths nobody
anticipated, and it must stay that way: a wrong `full` costs runner minutes and is
visible in the run, a wrong `empty` costs a merged regression and is invisible.
But nothing in the tree should REACH it, and 254 tracked files did.
The cost was real. PR #3842 changed a skill, a README and .gitignore; .gitignore
matched no rule, so both axes went full and 74 cmake legs span up runners to do
checkout + toolchain + get_deps before skipping the build, plus the whole 30-board
rig. Three changes, none of which touch rule 17 itself:
1. _META_RE - repo metadata and tooling no Build step reads: .gitignore,
.gitattributes, .clang-format, .codespellrc, .pre-commit-config.yaml,
.readthedocs.yaml, .PVS-Studio/, .idea/, sonar-project.properties, the
packaging manifests, CMakePresets, udev rules, test/{fuzz,unit-test} (their own
jobs build those), the non-build .github/ files, and the tools/*.py scripts no
build invokes. Deliberately NOT included, and still full: .circleci/**,
.github/workflows/build*.yml, .github/actions/**, .github/scripts/**. The line
is "does a Build step read this", not "is it source".
2. Rules 15 and 16 now match what they already claimed. Row 15 names
examples/<role>/CMakeLists.txt and the regex never had it; row 16 says
tools/build*.py but anchored tools/build\.py$. Both got the right answer only
because rule 17 caught them on the way past. Also names their siblings -
family_support.mk, family_rules.mk, src/CMakeLists.txt, src/tinyusb.mk - and
.circleci/**, which generates the whole CircleCI matrix and was in no row at all.
3. src/typec/** gets row 12b. It is listed unconditionally by both build systems
but its body is `#if CFG_TUC_ENABLED`, which only examples/typec/power_delivery
sets - the same shape as the class rule, so the same answer: the examples that
enable it (stm32g4 and stm32u5 after the buildability prune), and nothing on the
rig, which runs no typec test. It was force-fulling 82 families and all 30 boards.
TestNoTrackedFileIsUnclassified walks every tracked file and asserts none reaches
rule 17, on both axes - 254 -> 0. Verified it fails when a new unclassified path
appears. That turns 17 into what it should be: unreachable for anything in the
tree, so it fires only for genuinely new shapes, and the author is told to write
the row rather than letting the fall-through pick an answer for them.
test_full_paths used sonar-project.properties as its stand-in for "unclassified";
that is now metadata, so the case moved to the new
test_repo_metadata_is_not_a_build_input, with test_the_build_machinery_is_still_full
pinning the other side of the line.
|
|
The roster had two ways to pass a cmake -D to a board's build: `build.args`,
applied to every variant, and `variant[].defines`, applied to one. They did the
same thing, and only metro_m4_express used the first - for MAX3421_HOST=1, which
is what makes it the one rig board that compiles hcd_max3421.c.
A board whose define is always on now carries a single variant named after itself,
which is exactly the shape `board.get('variant') or [{'name': name, 'flags': ''}]`
already synthesises everywhere - so the build dir, the HIL report row and the
variant-boundary handling are unchanged. raspberry_pi_pico has used that shape
for its flags all along.
Removes the BuildCfg type and the parallel code path from all four consumers:
hil_test.build_board, hil_pool_check's two builders, hil_ci_set_matrix and
ci_select.board_options.
Verified: the hil-build matrix entry is byte-identical
(`-b metro_m4_express -DMAX3421_HOST=1`), hil_test's build command is unchanged,
ci_select still selects the board for a max3421 diff with MAX3421_HOST in its
options, and a real build of dual/host_info_to_device_cdc and host/cdc_msc_hid on
that board still compiles hcd_max3421.c.
|
|
Every one of these dropped coverage silently - the worst failure mode here,
because the PR still goes green. Found by review, each reproduced first.
Selection rules:
* class_macros derived the config macro from the class DIRECTORY, so a change to
src/class/midi/midi2_device.c selected the midi_test examples (which do not
compile it) and never examples/device/midi2_device (the only one that enables
CFG_TUD_MIDI2, and the only one that does). The file's own macro is unioned in
where it differs - union, never replace: over-selecting costs a build,
under-selecting merges a break.
* the ${FAMILY_MCUS} fallback added for espressif fired on any family whose
_family_mcus came back empty, and _cmake_sets is if()-blind and keeps the FIRST
definition - so mcx/frdm_mcxn947 answered MCXA15, a token six examples' skip.txt
names, dropping 12 firmware images CMake builds. Limited now to families that
never spell set(FAMILY_MCUS ...) at all.
* lib_examples read only an example's top-level CMakeLists.txt/Makefile;
host/msc_file_explorer_freertos names lib/embedded-cli in src/CMakeLists.txt and
survived by luck. The whole example tree is scanned. (SEGGER_RTT and rt-thread
still resolve to nothing: all three references sit inside a LOGGER=rtt guard no
CI build sets - the documented ruling, not a miss.)
* get_family_boards applied ci_skip_boards/ci_preferred_boards only under
GITHUB_ACTIONS/CIRCLECI, so the selector answered differently on a laptop than
on a runner; _prune_buildable forces CI semantics. Its one-board pick also
abandoned the whole preferred list when entry one could not build the -e set,
and asked skip_example without the build's -D tokens.
* _config_enables and lib_examples still read with the locale encoding - under
LC_ALL=C the selector tracebacked on three tracked tusb_config.h files. The
whole selector and its suite run clean there now.
Workflows:
* the Membrowse Upload step omitted $EX_ARGS, but --one-first now picks the board
from the -e set, so it configured a different, empty build dir and uploaded
--identical for a board never compiled. It takes $EX_ARGS for the BOARD; the
target stays the aggregate, which has no DEPENDS and still records every example.
* blanking FAM_REGEX reset only build_filtered, leaving the build scoped while
code-metrics took the UNSCOPED branch and diffed a 1-family run against the full
averaged baseline. All three drop together now, as CircleCI's fall-open does.
* CircleCI's EX_ARGS had no character screen and is used unquoted, and its
code-metrics job still exit 1'd on an empty metrics set - which a scoped build
makes a legitimate outcome.
* a `ci-full` PR label now turns the scoping off for one PR. A selector bug
under-selects silently, and without a label the only ways back to a full matrix
are accidental.
Performance, since the selector gates every other job: family.cmake texts are read
once rather than per changed directory (a 6,000-file dep bump re-read 84 files
99,892 times) and _scrape_mcu is cached: 2.2s -> 0.29s there, 0.8s -> 0.33s on a
class diff.
Tests: a drift guard for hw/bsp families absent from ci_set_matrix.family_list
(they select zero legs now, where they used to ride the full matrix); the rule-4
port test asserted a SUBSET, which set() satisfies, so it could not fail on the
empty selection it exists to catch; the GITHUB_ENV guard test counted a SUM of two
guards. Drops metrics.py's --only-examples, which nothing called, and applies the
TOTAL scrub to the by-example branch that skipped it.
|
|
Every PR built all 74 legs (2494 example builds on GHA cmake alone) and flashed
all 30 rig boards, whatever it touched. One classifier now walks the PR diff twice
and answers three questions: which families to build, which examples per family,
and which boards run which tests. Fail-open throughout - anything no rule
classifies, any exception, any unusable output falls back to the full matrix, and
a master push always builds everything.
test/hil/helper/hil_select.py moves to tools/ci_select.py: it is no longer HIL-only,
and tools/ is where the build side can import it. test_hil_select.py follows it as
test_ci_select.py.
Rules (docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md holds the
full table): a port selects the families whose family.cmake references it, and its
role - a dcd change skips host examples and vice versa; a class selects only the
examples whose tusb_config.h enables its CFG_TU[DH]_ macro, following cross-class
includes; an example selects itself; hw/bsp selects its family or board; hw/mcu and
lib select whoever references them. CMake is the reference for all of it - make
follows whatever cmake decides, family.mk is never scanned.
Empty means empty (maintainer ruling): a rule that classifies a path to nothing
selects nothing. Ports no family references, classes no config enables, libs no
example builds and hw/mcu paths that resolve nowhere are all real - nothing
compiles them, so nothing can validate them, and the master-push build is the net.
Structural tests pin each such case with an explicit allowlist, so the day one
stops being empty it fails pre-commit instead of silently narrowing CI.
Per-example builds: build.py grows a repeatable -e, resolved against the targets
CMake actually registered and batched into one `cmake --build --target a b c`.
build_utils mirrors CMake's family_filter (the whole FAMILY_MCUS list, ${...} and
string(TOUPPER ...) resolved) for the cmake side, while the make side keeps
master's algorithm verbatim - the two build systems answer differently and a shared
answer breaks lpc54's make link. hil-build gains this even on a full selection:
1702 example builds become 515.
Transport: the selection travels as a file, never an argv or env var - a mass-sweep
diff selects 261 KB against a 128 KiB exec limit, and E2BIG would fail the step
before its own fallback could run. CircleCI carries the example map inside the
generated config (pipeline parameters cap at 512 chars), swapped into the parameter
defaults by sentinel match, and drops the scoping wholesale if that rewrite fails.
Every PR-derived value written to $GITHUB_ENV/$GITHUB_OUTPUT is character-screened.
Code metrics follow the scoping: metrics.py emits per-example totals, and
metrics_pair_compare compares the (board, example) pairs present on both sides
instead of a scoped run against a full-matrix average.
The selector's own suite gates it in both providers: a selector that exits 0 with
valid-but-wrong JSON is the one failure fail-open cannot catch, so a red suite
means the full matrix.
|
|
get_deps matches a family token against a requested family name verbatim
(`f in deps_optional[d][2].split()`), so a token naming no hw/bsp directory makes
its entry unreachable:
hw/mcu/allwinner said 'fc100s'; the family is hw/bsp/f1c100s, and
f1c100s/family.cmake sets SDK_DIR to ${TOP}/hw/mcu/allwinner/f1c100s
hw/mcu/sony/cxd56/spresense-exported-sdk said 'spresense' (the SDK's name);
the family is hw/bsp/cxd56, whose family.cmake points SDK_DIR at it
`python3 tools/get_deps.py f1c100s` and `... cxd56` now fetch the SDK each of those
families builds against; before, both printed "no additional dependencies found".
docs/reference/dependencies.rst is generated from deps_all by tools/gen_doc.py, so
it is updated to match - column widths are unchanged (the widest cell is
lib/CMSIS_5's, untouched) and every row was cross-checked against deps_all.
|
|
vendor_host.c/.h implemented a CFG_TUH_VENDOR class driver that no example, board
or test ever enabled: usbh's driver table entry was compiled out everywhere, and
the six tusb_config.h files that mentioned the macro all set it to 0. Maintainer
call - dead code, not a shrinking of supported classes.
Removes the sources, the usbh driver-table entry, the CFG_TUH_VENDOR default in
tusb_option.h, the tusb.h include, both build-system source lists, the rp2040
family.cmake entry and the IAR project template rows.
|
|
Document the ci and hfp HIL rigs in enough detail to reproduce one: bill of
materials with photos, BIOS/IOMMU and vfio-pci passthrough on the Proxmox
host, the Renesas uPD720201 firmware install, the guest software and
permissions, the one-hub-per-root-port USB topology rule and the per-box
split of probe and DUT hubs, how CI drives the rigs, and the operational
gotchas.
The attached-board table is generated from test/hil/tinyusb.json and
test/hil/hfp.json by tools/gen_doc.py into docs/reference/hil_boards.md,
which the page includes. Sphinx excludes that partial so it is not also
built as an orphan document.
Also exclude docs/superpowers/ from the Sphinx build: it holds internal
plans, specs and handoffs rather than published documentation, and since
nothing references them from a toctree each emitted "document isn't
included in any toctree" -- 26 warnings in total, so build_doc.py -W could
never pass. It now does.
|
|
A wedged USB device used to take the whole HIL run with it. Every worker that
touched the poisoned node blocked uninterruptibly, the pool could not be joined,
map_async discarded every board's result, and the job ran to the GitHub ceiling
with no report at all -- while the self-hosted runner's single job slot stayed
occupied and every queued job waited behind it.
Bound the calls a worker makes itself. read_sysfs, bounded_open and run_cmd all
answer within a wall clock; read_sysfs distinguishes "absent" from "unknown",
because a blocked read is not evidence of absence, and caps stranded readers at
four (each costs a thread and an fd for the life of the process) after which the
worker declares itself blind. mtype, the gio unmount, the libmtp session and the
arecord/iperf reaps go through those bounds; the MTP session runs in a disposable
subprocess, since libmtp's ctypes calls block unkillably in D state.
Bound the run. A pool guard (HIL_POOL_TIMEOUT, 60 min) fires before any job
ceiling and still writes a report. When the pool will not shut down, the sweep
kills what the workers spawned -- descendants, not just direct children, since
flashers run in their own session -- confirms each kill actually landed, and
exits early so the runner is freed. Whatever survived is named in the report.
Deliberately shallow past that point. We do not re-scan process groups, prove
pid ownership, or escalate through sudo: a root-owned survivor is reported, not
force-killed, because signalling a pid we cannot prove is ours is the worse
failure, and the job ceiling backstops whatever this misses. A D-state holder
was never killable anyway.
Recover instead of reporting a wedge. A HUNG usbtest case reflashes its own DUT
through its roster flasher, but only where the flasher can reach its probe past
a poisoned node -- openocd pinned to a validated vid_pid, or esptool. Where it
cannot, the run says so rather than reserving budget for a path that cannot fire.
Raise the CI ceilings above the pool guard so the guard fires first and still
writes its report, and pin --retry 1 on every HIL leg: the guard is a flat
constant and does not scale with max_retry, so argparse's default of 3 would
triple the serialized usbtest tail against an unchanged guard.
Split the module: execution in hil_test/hil_flash/usbtest, infrastructure in
helper/ (locking, health, selection, shared bounded IO), and the two matrix
generators into .github/scripts/ -- ci_set_matrix.py sat in workflows/, where
GitHub treats every file as a workflow definition. 193 tests cover the bounded
paths, the kill ladder, the guard and the selector against synthetic /proc trees
and PATH-injected fakes; a real wedge cannot be manufactured on demand.
|
|
The plan doc records why the fork exists and how each vendor source was
ported; the interim handoff it superseded is dropped.
CLAUDE.md: a new worktree should symlink the dependency dirs to the
primary checkout rather than re-fetching them, replacing a single
symlink only when the branch needs a different dep revision. Also allow
'linke' in codespell - WCH-LinkE is a product name.
|
|
|
|
|
|
- getting_started.rst: the reST inline-link markup rendered literally
inside the code-block (not a runnable command) and lacked sudo — use a
plain `sudo cp examples/device/99-tinyusb-examples.rules ...`.
- tools/88-tinyusb.rules: normalize the six MODE="666" entries to the
4-digit octal MODE="0666" used everywhere else in the file.
|
|
Fourth view in the USB debugging toolset (usbmon = host URBs, usb-debug =
host reasoning, usb-target-debug = device firmware, usb-sniffer = what
actually crossed D+/D-). Covers the ataradov/usb-sniffer analyzer:
headless pcapng capture (--speed ls/fs/hs, --fold, --limit self-exit),
Wireshark/tshark analysis recipes, and the wire realities that bite:
downstream broadcast, sniffer self-capture noise, xHCI devnum != wire
address, tap-point-dependent reset visibility (hub choreography anchors),
FS-behind-HS-hub splits. Every recipe hardware-validated on the rig,
including the capture-window floor (a 3 s window provably misses the
enumeration ladder; 3M packets minimum).
Two udev files with distinct audiences, not one:
- examples/device/99-tinyusb-examples.rules (renamed from 99-tinyusb.rules):
the user-facing rules the examples need — cafe VID access, hidraw, the
ModemManager blacklist, a couple of board probes. getting_started.rst,
the webusb_serial README and its source comment point here.
- tools/88-tinyusb.rules: the HIL rig's private probe/analyzer allowlist,
now with the sniffer (6666:6620 + blank FX2LP 04b4:8613). Installed on
the rig only; the usb-sniffer skill references it.
|
|
and mimxrt1015
edpt_schedule_packets() advanced xfer->buffer past each armed EP0
chunk, so the OUT-complete handler invalidated the cache at the
ADVANCED pointer: one line past the received data. The CPU then read
stale cached bytes instead of the DMA'd packet, and the misplaced
invalidate discarded a dirty line of whatever variable follows the
buffer - random neighbor corruption on every control-OUT data stage.
Found by usbtest ctrl_out (cases 14/21) on espressif_p4_function_ev
with DMA enabled, the first DWC2 target combining buffer DMA with a
data cache: usbd control state wedged after the first control write
(every later request stalled), and one build layout panicked in the
usbd memcpy with a wild pointer.
Rework the EP0 chunk bookkeeping so xfer->buffer always points at the
un-consumed position: the arm no longer advances it; instead the EP0
re-arm paths advance past each completed (full) chunk, invalidating it
first on the OUT side. The final OUT completion invalidates exactly
the received bytes of its last chunk, taken from DOEPDMA ("incremented
on every AHB transaction", databook 7.1.83 - the same semantics the
SETUP path relies on) before dma_setup_prepare() re-targets it. EP0
chunking state (ep0_pending) is now also dropped on bus reset and on
a new SETUP, so a stale latched completion can no longer re-arm EP0
DMA from dead state. No behavior change for targets without dcache.
While root-causing, the FIFO layout was cross-checked against the
DWC2 databook/programming guide v4.20a: the existing GDFIFOCFG
programming (EPInfoBaseAddr = otg_dfifo_depth - 2*ep_count, one SPRAM
word per endpoint direction for buffer DMA) is conformant and needs
no change; the P4 HS instance's reset GDFIFOCFG (0x03800400) merely
reflects a scatter/gather-sized EP_LOC_CNT of 128 that buffer DMA
does not need.
With the fix in place, enable the usbtest battery on the espressif
fleet: tools/build.py allowlists device/usbtest (a plain IDF component
like board_test/video_capture) and both espressif boards' only-lists
gain device/usbtest. Also re-enable device/usbtest on mimxrt1015_evk:
its skip predated the dcd_ci_hs stale-ACTIVE-overlay fix (already on
this branch), which cured the battery that previously killed the
uPD720201 host controller twice (2026-07-11 ROM fw, 2026-07-13 case 27
on fw 2.0.2.6); rig-validated 30/30 three consecutive runs.
Validated on rig (all 30/30): espressif_p4_function_ev(-DMA) (was
22/30 under DMA), espressif_s3_devkitm(-DMA), stm32f723disco(-DMA),
mimxrt1015_evk; p4/s3 slave-mode unaffected (DMA-only code path);
compile-checked stm32h743nucleo +TUD DMA, stm32f407disco,
stm32l476disco (device ports currently on the dead hub).
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017TQZrFfU3K4Y198aLsUpBC
|
|
Pool/config:
- record real uids (ra8m1_ek), enable usbtest for espressif s3/p4, then
park ra6m5_ek and ra8m1_ek in boards-skip (ra6m5's usbtest/MSC traffic
can kill the uPD720201 host on its ROM firmware; ra8m1 USBHS bring-up
pending); max32666/nrf54lm20 stay enabled - their MosChip flakiness
never wedges
- re-enable device/usbtest on HS boards (mimxrt1064, ch32v307) now that
uPD720201 firmware 2.0.2.6 fixes the command-ring death; mimxrt1015
stays skipped - its HS battery killed the controller on both ROM and
2.0.2.6 firmware (board-specific); match the moved host-test bundles
(f723 <-> rt1064); skip never-passing tests on the new
nrf5340dk/nrf54lm20dk boards and the detached pico host bundle, each
documented with a comment
Host-controller quirk gating in usbtest.py (auto-skip, self-heals on a
healthy xHCI):
- MosChip MCS9990 EHCI: case 25 (int-OUT never scheduled, FRINDEX bug)
and case 11 (unlinked reads complete short/EREMOTEIO)
- Renesas uPD720201 xHCI: firmware-gated. The card must run firmware
>= 2.0.2.6 (RAM-uploaded - it reverts to ROM on every power cycle):
on older firmware the command ring dies under unlink stress (a
Configure Endpoint command stops completing; the hub worker deadlocks
holding the device lock; only a host power cycle recovers; three
boards reproduced it). usbtest.py reads the FW version register (PCI
config 0x6c) and refuses to run at all on older firmware - hil_test
surfaces that as a failed test with the reason. On current firmware
the full 30-case battery runs (validated FS+HS: metro_m4, f723,
f723-DMA all 30/30).
Scheduling (hil_test.py):
- Shuffle each (board, variant)'s test order with a seeded RNG
(HIL_SHUFFLE_SEED to replay) so usbtest batteries and flash churn spread
across the timeline instead of convoying on one controller.
- Per-controller usbtest + flash semaphores: HIL_USBTEST_PARALLEL
(default 4) concurrent usbtest batteries and HIL_FLASH_PARALLEL
(default 8) concurrent flashes per host controller. Profiled on
uPD720201 firmware 2.0.2.6 across 8/1..12/8: wall time falls
22.2/14.3/12.5/10.8 min at usbtest width 1/2/3/4 and plateaus there;
zero controller errors everywhere; first battery case failures
(leaf-hub bandwidth stretch) appear at 12/8, and flash width 12 only
amplifies flasher-hub contention flakes - so 8/4 is the optimum. A
separate battery-window flash throttle was profiled and dropped.
- Give every example a unique hardcoded USB PID (0x4001-0x4022, usbtest
keeps 0x4010) instead of the PID_MAP interface bitmap: different
examples now always re-enumerate back-to-back, even on boards whose
CPU reset does not drop D+ (WCH CH58x), so the EXAMPLE_PID table and
same-PID adjacency reordering in hil_test.py are gone; only the
variant-boundary same-example repeat needs a swap.
- Report matrix: stable columns with the metric-bearing tests pinned
first (usbtest, cdc_msc_throughput, msc_file_explorer[_freertos]),
the rest alphabetical.
Fail fast:
- enum wait budget 8 s on the first attempt, 4 s on retries; dfu waits
are deadline-based so dfu-util's own runtime counts against the
budget.
A device-absent failure now costs ~3-5x a passing test (20-30 s)
instead of 10-30x (47-150 s).
- CI runs hil_test with --retry 1 and no in-run second pass: a broken
fixture fails the job fast instead of holding the self-hosted runner
for hours and blocking other PRs' HIL jobs. hil_test still writes the
.skip sidecar, so a manual re-run attempt only retests what failed.
Review fixes (multi-agent adversarial review of this commit):
- tinyusb_win_usbser.inf: the PID rework moved five CDC examples onto
even PIDs the INF's odd-only DeviceList never matched (legacy-Windows
usbser binding) - appended 0x4006/4008/400a/4020/4022 to both lists.
- usbtest example: USBTEST_TIER is now overridable and the descriptors
and pumps are tier-conditional, so a board whose DCD cannot serve a
tier lowers it instead of skipping the whole example - RA2A1 (RUSB2
with no isochronous pipe) builds at tier 3 via its BOARD_ define; the
host battery follows the tier advertised in bcdDevice. Tier-4 output
verified byte-identical after the refactor.
- dynamic_configuration's second config derived USB_PID + 11 = 0x4018,
colliding with net_lwip_webserver - now USB_PID + 0x0100, outside the
per-example space. tools/check_example_pids.py (pre-commit hook)
enforces PID uniqueness incl. derived and literal idProduct values.
- usbtest.py firmware gate: matched by device ID (uPD720201/720202,
both use the 0x6c FW register), and an unreadable version (setpci
missing/denied) now refuses with its own message instead of
masquerading as "firmware 0x00000000"; noted the gate is necessary
but not sufficient (board-specific kills stay per-board skips).
- hil_test: deadline waits use time.monotonic(); multiprocessing
context pinned to fork (raw semaphores in Pool initargs); flash and
usbtest permits unified into one fail-closed, exception-safe
ctrl_permit (unknown controller takes every slot and logs a warning
instead of silently borrowing slot 0); an all-skipped battery
reports as skip, not "0/0" failure; slow-body polls (mtp, printer,
disk read) go through a shared deadline-based wait_until so their
bodies count against the enum budget; throughput's FS detection
compares serials case-insensitively like every other walk; a missing
MSC read-speed line now fails the host msc_file_explorer test
instead of passing with an empty metric.
Hardening:
- fail fast (15 s) when a driver-registry sysfs write blocks: a wedged
device otherwise turns every subsequent battery into an unkillable
D-state writer and silently hangs the whole run
- usb-recover skill: a VM reboot is not a reliable cure (MosChip hubs
latch up across the PCIe reset); full host power cycle is
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HeF2gZ1M7GWkz6Av4BpKPg
|
|
* docs: convert changelog release notes from RST to Markdown
|
|
Relocate docs/info/changelog/ -> docs/changelog/ (docs/info held only the
changelog) and update all references: the toctree, README + MAINTAINERS links,
make_release.py, and the make-release skill.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
|
The one-board selector uses preferred_list[0] anyway, so pico is the board built.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
|
backfill note
- ci_preferred_boards: rp2040 -> raspberry_pi_pico, raspberry_pi_pico2 so the
one-board CI/metrics build uses a Pico (the first-alphabetical board failed).
- make-release skill: the metrics backfill was a one-time 0.20.0 task, not a
recurring release step; keep only the CI auto-compare note.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
|
- changelog: fix 3 silent RST mis-renders carried over from the old monolith
— Markdown link in 0.7.0, mismatched/single backticks in 0.18.0 / 0.20.0
- make-release skill contributors one-liner: parallelize the gh fetch, anchor
the bot filter (was a substring that dropped handles like "abbott"), and join
with ", " (paste -sd cycles the delimiter -> "@a,@b @c"); finalize now stages
only the reviewed set (git add -A -- ':!.idea'); drop the obsolete CRLF gotcha
- make_release.py: emit LF not CRLF in the repository.yml insertion (fixes the
gotcha at the source)
- docs sidebar: drop the hardcoded furo component list; override brand.html to
include furo's own template + the sponsor button (decoupled from furo
internals, no upper-bound pin needed); move sponsor styles into custom.css
- .pre-commit-config.yaml: drop stale end-of-file-fixer excludes for the
deleted contributors/CoC include shims
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
|
Release:
- Bump version to 0.21.0 and regenerate tusb_option.h, library.json,
repository.yml, sonar-project.properties, boards.rst, dependencies.rst
- Add 0.21.0 release notes and split the changelog into per-release files
under docs/info/changelog/ (date out of title, driver/class groups as
sub-headings, DCD & HCD section after Device/Host stacks)
Contributors:
- Credit each release's PR authors in a Contributors section
- Drop the curated contributors page; add MAINTAINERS.rst
Docs:
- Update Code of Conduct to Contributor Covenant 3.0 (keep it in the repo,
remove it from the built docs)
- Sidebar: add a home entry, rename the group to "Documentation", move
Changelog into it, add a GitHub Sponsor button, merge external links,
rename FAQ; hide the inline toctree on the landing page
- Add the make-release skill and update the AGENTS.md release process
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
|
docs: add build-doc tooling and a README for every example
|
|
Documentation tooling:
- Add the `build-doc` skill and `tools/build_doc.py` wrapper for local
Sphinx builds (clean / -W / open).
- Enable Markdown (MyST) in conf.py and auto-collect
examples/{device,host,dual}/*/README.md into a 3-level Examples nav
(Examples > Device/Host/Dual > example), noting each page's source
location and normalizing headings to a single H1.
- Remove the stale `.claude/commands/build-doc.md`; point the AGENTS.md
Documentation section at the skill.
Example docs:
- Add a README.md for every device/host/dual example: what it does, USB
interface table, notable tusb_config.h settings, generic CMake + Make
build steps, and how to try it.
- Fold each *_freertos variant into its base README, noting the FreeRTOS
source path and any RTOS-specific behavior.
Generated docs/examples/ output is git-ignored. Builds clean with
`sphinx-build -W`.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
|
Signed-off-by: HiFiPhile <[email protected]>
|
|
|
|
The BSP family and MCU option were named "ch58x"/"CH58X", but the supported part is
the CH583/CH582 (and the SDK repo is openwch/ch583); CH585 is a separate MCU family,
so the CH58x umbrella was misleading. Rename to the specific family:
- hw/bsp/ch58x -> hw/bsp/ch583 (dir), and the BSP-local files ch58x_it.* ->
ch583_it.*, system_ch58x.* -> system_ch583.* (include guards/refs updated). The
vendor SDK files (CH58x_common.h, CH58x_*.c in hw/mcu/wch/ch583) keep their names.
- OPT_MCU_CH58X -> OPT_MCU_CH583 in tusb_option.h, tusb_mcu.h, and the shared WCH
USBFS driver (ch32_usbfs_reg.h, dcd_ch32_usbfs.c). OPT_MCU_CH582 is kept as an
alias (same value), so either name selects the same code.
- FAMILY_MCUS CH58X -> CH583, CFG_TUSB_MCU=OPT_MCU_CH583, mcu:CH58X -> mcu:CH583 in
the example skip lists, the CI build matrix (ci_set_matrix.py), the get_deps family
tag, and docs/reference/boards.rst.
Board names (ch582m_evt, yd-ch582m) are unchanged. Verified: make + cmake build for
ch582m_evt, and ci.lan HIL (all device examples pass).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
|
The dependency is fetched from https://github.com/openwch/ch583.git but lived at
hw/mcu/wch/ch58x. Rename the local path to hw/mcu/wch/ch583 so it matches the
upstream repo name. Updates the get_deps.py path key and the ch58x BSP SDK_DIR
(family.mk + family.cmake); the BSP family stays "ch58x" (covers CH582 and CH583).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
|
When no base metrics are available to compare against, the PR comment
falls back to the combined "TinyUSB Average Code Size Metrics" report
(build.yml copies metrics.md to metrics_compare.md). That posted the
full per-example size table inline, cluttering the comment.
Wrap the table in a <details><summary>Size table</summary> block so the
heading stays visible but the detail is collapsed by default, matching
the Size Difference Report's collapsible sections.
Co-Authored-By: Claude Fable 5 <[email protected]>
|
|
|
|
Wrap the "Changes <1% in size" section in a <details> block like the
"No changes" section, so the report comment only expands changes >1%.
Co-Authored-By: Claude Fable 5 <[email protected]>
|
|
# Conflicts:
# README.rst
|
|
split-IN NAK storm (#3677)
Fix stm32f723disco host HIL: UART RX starvation + DWC2 split bulk NAK/XactErr handling (#3677)
stm32f7 BSP — UART RX starvation
- The host console USART shared interrupt priority with the USB OTG ISR, so a long
OTG interrupt could starve RXNE and drop received bytes. Raise the USART RX IRQ
above OTG_FS/OTG_HS in both the bare-metal and FreeRTOS init paths, guarded by
#ifdef UART_ID so boards without a UART console keep the default OTG priority.
dwc2 host — split NAK/XactErr handling
- Slave mode: a persistently-NAKing split bulk/control IN poll re-armed the
start-split immediately, storming the ISR and starving task context. Throttle by
disabling the channel and re-arming on the resulting halt (no frame deferral).
- Buffer-DMA mode: a pure split bulk-OUT NAK was unhandled, leaving the channel
halted and stalling the transfer — the dominant cause of CDC echo truncation.
Handle it by rewinding the buffer pointers and retrying the start-split
(Programming Guide v4.20a 5.1.4.2).
- Buffer-DMA mode: a split bulk-OUT XactErr was retried immediately, exhausting
HCD_XFER_ERROR_MAX before the transient cleared. Throttle via channel_disable +
re-arm to give the hub TT a recovery gap, mirroring slave mode.
- All three are scoped to split transfers (hcsplt.split_en); non-split NAK/XactErr
keep the core-handled / immediate-retry behavior. The OUT XactErr throttle also
excludes periodic split, where channel_disable() is a no-op and would wedge the
channel. The nak_disabled flag is generalized to retry_disabled and honors
xfer->closing so an endpoint close during a throttled retry tears down cleanly.
Verified on stm32f723disco HIL (slave + CFG_TUH_DWC2_DMA_ENABLE): host/cdc_msc_hid,
msc_file_explorer, and device_info all pass on both variants; DMA CDC echo went
from ~15-25% raw failure to 10/10 clean.
|
|
* test/hil: replace build.flags_on with named variant schema
Boards declare build variants as `variant: [{name, flags}]` instead of
`build.flags_on`. The variant `name` is the build dir (cmake-build-<name>) and
the HIL report row; `flags` is the raw CFLAGS string (-D...=1) injected via
CFLAGS_CLI. No `variant` => a single build named after the board.
- build.py: --build-name <name> (dir) + --cflag=<token> (raw CFLAGS, repeatable,
=form survives the matrix's shell word-splitting); drop -f1/CFLAGS wrapping.
- hil_ci_set_matrix.py: emit one build arg per variant.
- hil_test.py: iterate variants; report row + build dir = variant name.
- hil_ci.sh: copy all cmake-build-<board>* dirs for -b runs.
- get_deps.py: accept (ignore) --build-name/--cflag from matrix args.
- tinyusb.json: migrate all 6 flags_on boards to variant.
* board_test: park CI build with busy spin instead of wfe
|
|
metrics.md (write_combine_markdown) is also used as the PR size comment when
there is no base-metrics baseline; use h2 for its title too so the sticky
comment heading is consistent (and not oversized) in that fallback case.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
|
The Size Difference Report and HIL comments rendered their titles at h1, which
is oversized inside a PR comment. Use h2 for both titles (with subsections
demoted to h3 to keep the hierarchy), and rename the HIL comment from
"HIL test results" to "Hardware-in-the-loop (HIL) Test Report" for consistency.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
|
Per review (HiFiPhile): Ninja Multi-Config is needed for IAR, otherwise the
optimization level can't be lowered to none for debug. Revert gen_presets.py
back to Ninja Multi-Config (keeping only the cmake-build-<board> binaryDir
change), and instead teach hil_test.py to locate <ex>.elf whether it sits
directly in the example dir (single-config) or under a per-config subdir like
RelWithDebInfo/ (multi-config).
Verified: stm32u083nucleo passes 13/13 remote HIL with a multi-config preset
build (rsync preserves the RelWithDebInfo/ subdir; the resolver finds it).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
|
Change the default configure preset binaryDir from build/<board> to
cmake-build-<board> (the dir name HIL expects) and switch the generator
from Ninja Multi-Config to single-config Ninja. Multi-Config nests
binaries under a RelWithDebInfo/ subdir, which hil_test.py does not look
in; single-config emits device/<ex>/<ex>.elf so preset-built firmware is
directly consumable by `hil_test.py -B examples`.
Regenerated BoardPresets.json (also picks up the tracked ch32v103c_bluepill
board that was missing from presets).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
|
Signed-off-by: HiFiPHile <[email protected]>
|
|
`git worktree add <path> <branch>` fails if <branch> is already checked out
elsewhere (main repo, another worktree). --detach checks out the ref at a
detached HEAD instead of claiming the branch, making the script work
regardless of what is currently checked out.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
|
|
- run() now catches subprocess.TimeoutExpired (only triggered by `cmake --build`'s
timeout=600) and returns CompletedProcess(rc=124) so the caller falls through to
error reporting and worktree cleanup instead of crashing with a traceback.
- code-size SKILL.md: document the actual default filter (per-side absolute
<checkout>/src/ path, not the old `tinyusb/src` substring) and adjust the
reporting guidance to match what the report rows actually contain.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
|
|
|
|
- Introduced a `code-size` skill under `.claude/skills` for evaluating TinyUSB code size changes between the base branch and current branch.
- Added `metrics_compare_base.py`, automating code size comparison with granular options for examples, boards, and CI-wide runs.
- Updated `AGENTS.md` to include quick references and usage guidance for the new feature.
|
|
|
|
- Add lib/fatfs to deps_mandatory in get_deps.py, pinned to R0.16
(commit 30ca13c6) from github.com/abbrev/fatfs mirror
- Move TinyUSB custom ffconf.h to examples/host/msc_file_explorer/src/
with project-specific settings: FF_CODE_PAGE=437, FF_USE_LFN=1,
FF_FS_RPATH=2, FF_VOLUMES=4, FF_FS_NORTC=1
- get_deps.py removes stock ffconf.h after clone to avoid conflict
- Remove vendored fatfs source files from git tracking
FatFS R0.16 includes important fixes since R0.15:
- Fixed FAT32 FsInfo regression (forced full FAT scan on f_getfree)
- Fixed f_readdir infinite loop (from R0.15b)
- f_getcwd/.. now works on exFAT
- Added FF_FS_CRTIME support
|
|
|
|
|
