| Age | Commit message (Collapse) | Author |
|---|
|
taking the rig (#3846)
test_the_upload_board_can_diverge_from_the_built_board called
get_family_boards without ci=True, so it pinned the developer's set, not the
runner's: the CI skip lists move the one-first pick on three families. It
held locally and went red on its first CI run. Pass ci=True, as
_prune_buildable already does, and pin the runner's twelve.
Rule 2 is a bare test/hil/ prefix, so the harness's own unit tests booked the
full 27-board rig for diffs that cannot reach it. Carve test/hil/test/** out
to rule 1b, beside test/{fuzz,unit-test}/**; the harness itself is untouched.
A test pins that directory's file list, so anything added there that the rig
does read fails rather than silently skipping hardware. Rule table updated in
the spec and its carbon in the docstring.
|
|
ci: an empty selection must build nothing, plus selector follow-ups
A PR whose build axis legitimately selected nothing rebuilt everything.
build.yml reads .build.families twice - as a |-joined regex, and implicitly
as "is anything selected" - but tested only -z "$FAMILY_REGEX", which an
empty list and a charset-rejected one both satisfy while meaning opposite
things. ci_set_matrix had already returned the correct all-empty matrix;
the fall-open branch discarded it. #3842 and #3840 each spent 74 cmake legs
on it. Branch on the two cases instead, rename FAM_* to FAMILY_*, and cover
the block with a test that extracts it from build.yml and executes it - it
had no test at all, which is how this shipped through two merges.
Follow-ups to the same machinery: glob.escape the repo root at five sites,
so a checkout path containing [ or * stops failing closed; drop the ci-full
label, read after the matrix was already computed and so never functional;
delete 13 mcu:MKL25ZXX / mcu:SAME5X skip tokens matching no board; carry the
rule table in the module docstring, guarded against drift; and pin six
selection behaviours a mutation pass proved untested.
Cut the selector's cost 1.8x (26.0s -> 14.6s) with 0 divergences over 260
paths, and stop scoping the membrowse upload by the PR example filter.
|
|
ci_select: classify the files that were reaching rule 17
|
|
Anchor the .github file alternatives. FUNDING.yml, labeler.yml and
membrowse_pr_message.j2 sat inside a group whose only `$` belonged to the
workflows/ branch, so they matched as prefixes: .github/labeler.yml.bak and
.github/FUNDING.yml.old were classified as metadata and would have selected
nothing. No such file exists today - the workflows/ alternative was already
anchored and ISSUE_TEMPLATE/ is a directory prefix on purpose.
Rule 12b had no test of its own: TestNoTrackedFileIsUnclassified only proved
src/typec no longer reaches rule 17, not that the answer is right. TestTypecRule
pins it - non-full, every selected example under typec/, all four src/typec files
answering alike, no rig board, and the set derived from CFG_TUC_ENABLED rather
than hardcoded, so it follows a new typec example on its own. Verified all four
fail with rule 12b removed.
|
|
The eight lru_cache layers take repo-RELATIVE paths - 'hw/bsp/<fam>',
'examples/<ex>/skip.txt', the literal 'hw/bsp' glob - while ci_select._in_repo()
chdirs around every call so one process can classify more than one tree. With no
cwd in the key the second tree gets the first tree's answers.
Reproduced: skip_example('host/bare_api','metro_m0_express') is False at the repo
root and STILL False after chdir into a tree where that board does not exist; only
cache_clear() gave the right answer. It bites the code-size skill's base-vs-branch
worktree compare, /pre-pr, and the first test that points classify_build at a
fixture tree. Master had no caching here, so the hazard arrived with it.
_cwd_cache puts os.getcwd() in the key. The 199-test suite passed before only
because every test happens to pass the real REPO; the new
TestCachesAreKeyedOnTheTree crosses trees deliberately.
Also adds the drift guard the class rule was missing. Ports, hw/mcu, get_deps
tokens and bsp families each have one; the class rule had only a comment claiming
vendor_host.c was the sole "enabled by no example config" case until its removal -
which src/class/bth falsifies today. TestClassesWithNoEnablingExample pins the set
to {bth}, so a class added before its first example, or an example config flipped
to 0, fails here instead of silently selecting nothing on both axes. Verified it
fires by adding a class dir nothing enables.
|
|
Rule 17 (unclassified -> full on both axes) is the fail-open net for paths nobody
anticipated, and it must stay that way: a wrong `full` costs runner minutes and is
visible in the run, a wrong `empty` costs a merged regression and is invisible.
But nothing in the tree should REACH it, and 254 tracked files did.
The cost was real. PR #3842 changed a skill, a README and .gitignore; .gitignore
matched no rule, so both axes went full and 74 cmake legs span up runners to do
checkout + toolchain + get_deps before skipping the build, plus the whole 30-board
rig. Three changes, none of which touch rule 17 itself:
1. _META_RE - repo metadata and tooling no Build step reads: .gitignore,
.gitattributes, .clang-format, .codespellrc, .pre-commit-config.yaml,
.readthedocs.yaml, .PVS-Studio/, .idea/, sonar-project.properties, the
packaging manifests, CMakePresets, udev rules, test/{fuzz,unit-test} (their own
jobs build those), the non-build .github/ files, and the tools/*.py scripts no
build invokes. Deliberately NOT included, and still full: .circleci/**,
.github/workflows/build*.yml, .github/actions/**, .github/scripts/**. The line
is "does a Build step read this", not "is it source".
2. Rules 15 and 16 now match what they already claimed. Row 15 names
examples/<role>/CMakeLists.txt and the regex never had it; row 16 says
tools/build*.py but anchored tools/build\.py$. Both got the right answer only
because rule 17 caught them on the way past. Also names their siblings -
family_support.mk, family_rules.mk, src/CMakeLists.txt, src/tinyusb.mk - and
.circleci/**, which generates the whole CircleCI matrix and was in no row at all.
3. src/typec/** gets row 12b. It is listed unconditionally by both build systems
but its body is `#if CFG_TUC_ENABLED`, which only examples/typec/power_delivery
sets - the same shape as the class rule, so the same answer: the examples that
enable it (stm32g4 and stm32u5 after the buildability prune), and nothing on the
rig, which runs no typec test. It was force-fulling 82 families and all 30 boards.
TestNoTrackedFileIsUnclassified walks every tracked file and asserts none reaches
rule 17, on both axes - 254 -> 0. Verified it fails when a new unclassified path
appears. That turns 17 into what it should be: unreachable for anything in the
tree, so it fires only for genuinely new shapes, and the author is told to write
the row rather than letting the fall-through pick an answer for them.
test_full_paths used sonar-project.properties as its stand-in for "unclassified";
that is now metadata, so the case moved to the new
test_repo_metadata_is_not_a_build_input, with test_the_build_machinery_is_still_full
pinning the other side of the line.
|
|
ci: scope the build matrix and HIL run to what a PR affects
|
|
Removing the obsolete host vendor driver also dropped the `#define CFG_TUH_VENDOR 0`
line from the six example configs that carried it. Put it back: host vendor is
coming, and the configs are where a reader looks for the set of host classes an
example can turn on.
Restored byte-identical to the pre-removal state, each file keeping its own column
alignment. The define is inert today - nothing under src/, hw/ or tools/ reads
CFG_TUH_VENDOR - and it is 0 everywhere, so ci_select still reads the vendor class
as enabled by no example and a change to it still selects nothing.
Note the option's default in src/tusb_option.h is still gone; implementing the
driver will need that back alongside the usbh driver-table entry.
|
|
family_list maps a family to the toolchains that build it, and seven hw/bsp
families are in neither: cxd56, efm32, espressif, f1c100s, pic32mz, py32f0,
same7x. Scoping to one of them intersected to nothing, so every toolchain key was
[], every cmake leg skipped on `if: inputs.build-args != '[]'`, code-metrics took
its no-metrics branch, and the PR went green from a build job that ran no
compiler. The only signal was a stderr line nothing greps for.
Not a coverage regression - master gave the same diff no compile coverage either,
since none of the other families compiles same7x's board.h. What is new is that
the gap used to be masked by the full matrix and is now the whole answer, and
that green now means "ran no compiler" rather than "compiled 64 families".
A selection whose families ALL miss is now unusable rather than empty: it prints
UNSCOPED, which build.yml and .circleci/config.yml already grep to drop the build
extras with it, and emits the full matrix. The two neighbouring cases keep their
own answers - an explicit families: [] is still a legitimate nothing-selected, and
a partial miss still scopes to the families that do build, noting the rest.
The contract test pinned an exact count of fall-open markers, which this would
have broken; it now pins the invariant (every message that emits the full matrix
carries the marker) and was checked to still fail when a marker is removed.
Also corrects the drift guard's note about espressif: hil-build-esp builds its
boards by name, but that job is gated on repository_owner, so on a fork an
espressif-only PR builds nowhere.
|
|
The roster had two ways to pass a cmake -D to a board's build: `build.args`,
applied to every variant, and `variant[].defines`, applied to one. They did the
same thing, and only metro_m4_express used the first - for MAX3421_HOST=1, which
is what makes it the one rig board that compiles hcd_max3421.c.
A board whose define is always on now carries a single variant named after itself,
which is exactly the shape `board.get('variant') or [{'name': name, 'flags': ''}]`
already synthesises everywhere - so the build dir, the HIL report row and the
variant-boundary handling are unchanged. raspberry_pi_pico has used that shape
for its flags all along.
Removes the BuildCfg type and the parallel code path from all four consumers:
hil_test.build_board, hil_pool_check's two builders, hil_ci_set_matrix and
ci_select.board_options.
Verified: the hil-build matrix entry is byte-identical
(`-b metro_m4_express -DMAX3421_HOST=1`), hil_test's build command is unchanged,
ci_select still selects the board for a max3421 diff with MAX3421_HOST in its
options, and a real build of dual/host_info_to_device_cdc and host/cdc_msc_hid on
that board still compiles hcd_max3421.c.
|
|
Every one of these dropped coverage silently - the worst failure mode here,
because the PR still goes green. Found by review, each reproduced first.
Selection rules:
* class_macros derived the config macro from the class DIRECTORY, so a change to
src/class/midi/midi2_device.c selected the midi_test examples (which do not
compile it) and never examples/device/midi2_device (the only one that enables
CFG_TUD_MIDI2, and the only one that does). The file's own macro is unioned in
where it differs - union, never replace: over-selecting costs a build,
under-selecting merges a break.
* the ${FAMILY_MCUS} fallback added for espressif fired on any family whose
_family_mcus came back empty, and _cmake_sets is if()-blind and keeps the FIRST
definition - so mcx/frdm_mcxn947 answered MCXA15, a token six examples' skip.txt
names, dropping 12 firmware images CMake builds. Limited now to families that
never spell set(FAMILY_MCUS ...) at all.
* lib_examples read only an example's top-level CMakeLists.txt/Makefile;
host/msc_file_explorer_freertos names lib/embedded-cli in src/CMakeLists.txt and
survived by luck. The whole example tree is scanned. (SEGGER_RTT and rt-thread
still resolve to nothing: all three references sit inside a LOGGER=rtt guard no
CI build sets - the documented ruling, not a miss.)
* get_family_boards applied ci_skip_boards/ci_preferred_boards only under
GITHUB_ACTIONS/CIRCLECI, so the selector answered differently on a laptop than
on a runner; _prune_buildable forces CI semantics. Its one-board pick also
abandoned the whole preferred list when entry one could not build the -e set,
and asked skip_example without the build's -D tokens.
* _config_enables and lib_examples still read with the locale encoding - under
LC_ALL=C the selector tracebacked on three tracked tusb_config.h files. The
whole selector and its suite run clean there now.
Workflows:
* the Membrowse Upload step omitted $EX_ARGS, but --one-first now picks the board
from the -e set, so it configured a different, empty build dir and uploaded
--identical for a board never compiled. It takes $EX_ARGS for the BOARD; the
target stays the aggregate, which has no DEPENDS and still records every example.
* blanking FAM_REGEX reset only build_filtered, leaving the build scoped while
code-metrics took the UNSCOPED branch and diffed a 1-family run against the full
averaged baseline. All three drop together now, as CircleCI's fall-open does.
* CircleCI's EX_ARGS had no character screen and is used unquoted, and its
code-metrics job still exit 1'd on an empty metrics set - which a scoped build
makes a legitimate outcome.
* a `ci-full` PR label now turns the scoping off for one PR. A selector bug
under-selects silently, and without a label the only ways back to a full matrix
are accidental.
Performance, since the selector gates every other job: family.cmake texts are read
once rather than per changed directory (a 6,000-file dep bump re-read 84 files
99,892 times) and _scrape_mcu is cached: 2.2s -> 0.29s there, 0.8s -> 0.33s on a
class diff.
Tests: a drift guard for hw/bsp families absent from ci_set_matrix.family_list
(they select zero legs now, where they used to ride the full matrix); the rule-4
port test asserted a SUBSET, which set() satisfies, so it could not fail on the
empty selection it exists to catch; the GITHUB_ENV guard test counted a SUM of two
guards. Drops metrics.py's --only-examples, which nothing called, and applies the
TOTAL scrub to the by-example branch that skipped it.
|
|
The binding rule table (17 rows x 3 answer columns), the measured effect per PR
shape, and the reasoning behind the parts that look surprising: why empty means
empty, why hw/mcu and lib are rules rather than full-matrix paths, why get_deps.py
is diffed as data, and which build system is the reference. The plan is the
task-by-task record of how it was built, kept as the origin trail.
|
|
Every PR built all 74 legs (2494 example builds on GHA cmake alone) and flashed
all 30 rig boards, whatever it touched. One classifier now walks the PR diff twice
and answers three questions: which families to build, which examples per family,
and which boards run which tests. Fail-open throughout - anything no rule
classifies, any exception, any unusable output falls back to the full matrix, and
a master push always builds everything.
test/hil/helper/hil_select.py moves to tools/ci_select.py: it is no longer HIL-only,
and tools/ is where the build side can import it. test_hil_select.py follows it as
test_ci_select.py.
Rules (docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md holds the
full table): a port selects the families whose family.cmake references it, and its
role - a dcd change skips host examples and vice versa; a class selects only the
examples whose tusb_config.h enables its CFG_TU[DH]_ macro, following cross-class
includes; an example selects itself; hw/bsp selects its family or board; hw/mcu and
lib select whoever references them. CMake is the reference for all of it - make
follows whatever cmake decides, family.mk is never scanned.
Empty means empty (maintainer ruling): a rule that classifies a path to nothing
selects nothing. Ports no family references, classes no config enables, libs no
example builds and hw/mcu paths that resolve nowhere are all real - nothing
compiles them, so nothing can validate them, and the master-push build is the net.
Structural tests pin each such case with an explicit allowlist, so the day one
stops being empty it fails pre-commit instead of silently narrowing CI.
Per-example builds: build.py grows a repeatable -e, resolved against the targets
CMake actually registered and batched into one `cmake --build --target a b c`.
build_utils mirrors CMake's family_filter (the whole FAMILY_MCUS list, ${...} and
string(TOUPPER ...) resolved) for the cmake side, while the make side keeps
master's algorithm verbatim - the two build systems answer differently and a shared
answer breaks lpc54's make link. hil-build gains this even on a full selection:
1702 example builds become 515.
Transport: the selection travels as a file, never an argv or env var - a mass-sweep
diff selects 261 KB against a 128 KiB exec limit, and E2BIG would fail the step
before its own fallback could run. CircleCI carries the example map inside the
generated config (pipeline parameters cap at 512 chars), swapped into the parameter
defaults by sentinel match, and drops the scoping wholesale if that rewrite fails.
Every PR-derived value written to $GITHUB_ENV/$GITHUB_OUTPUT is character-screened.
Code metrics follow the scoping: metrics.py emits per-example totals, and
metrics_pair_compare compares the (board, example) pairs present on both sides
instead of a scoped run against a full-matrix average.
The selector's own suite gates it in both providers: a selector that exits 0 with
valid-but-wrong JSON is the one failure fail-open cannot catch, so a red suite
means the full matrix.
|
|
get_deps matches a family token against a requested family name verbatim
(`f in deps_optional[d][2].split()`), so a token naming no hw/bsp directory makes
its entry unreachable:
hw/mcu/allwinner said 'fc100s'; the family is hw/bsp/f1c100s, and
f1c100s/family.cmake sets SDK_DIR to ${TOP}/hw/mcu/allwinner/f1c100s
hw/mcu/sony/cxd56/spresense-exported-sdk said 'spresense' (the SDK's name);
the family is hw/bsp/cxd56, whose family.cmake points SDK_DIR at it
`python3 tools/get_deps.py f1c100s` and `... cxd56` now fetch the SDK each of those
families builds against; before, both printed "no additional dependencies found".
docs/reference/dependencies.rst is generated from deps_all by tools/gen_doc.py, so
it is updated to match - column widths are unchanged (the widest cell is
lib/CMSIS_5's, untouched) and every row was cross-checked against deps_all.
|
|
Nine examples/host/*/only.txt gate on family:samd21, but 2a8811ebb merged the
samd2x and saml2x BSPs into hw/bsp/samd2x_l2x. skip_example takes `family:` from
the directory name, so since that rename every one of these examples has been
skipped on every board of the family, under make as well as cmake - although
hw/bsp/samd2x_l2x/family.cmake wires src/portable/microchip/samd/hcd_samd.c.
107 host firmware images were being compiled nowhere.
The merged family is wider than the old samd21 one, so three boards need an
explicit skip rather than the rename alone:
atsaml21_xpro, saml22_feather, sensorwatch_m0 - not samd21, so hcd_samd.c is
not compiled for them (previous commit); SAML22 has no host controller at all
curiosity_nano - SAMD21 with 16 KB RAM; msc_file_explorer_freertos overflows it
by 3688 bytes (ram 122.51%). Only that one example; the other eight fit.
Verified across the whole family: cmake 11 boards x 9/9 examples + curiosity_nano
8/9, three boards skipped, 0 failures; make 98 OK / 0 failed (was 98/9 before).
Device examples on saml21 and saml22 are unaffected.
|
|
family.cmake listed src/portable/microchip/samd/hcd_samd.c twice: once
unconditionally, and once inside `if(SAM_FAMILY STREQUAL "samd21")` under the
comment "Add HCD support for SAMD21 (has host capability)". The unconditional copy
defeated the gate, so cmake compiled the host controller driver for saml21 and
saml22 while family.mk compiled it for samd21 alone - and SAML22 has no host
controller at all (hcd_samd.c fails there with `unknown type name
'UsbHostDescriptor'`).
Nothing built the host examples on this family, so the divergence was invisible;
the next commit makes it matter.
|
|
vendor_host.c/.h implemented a CFG_TUH_VENDOR class driver that no example, board
or test ever enabled: usbh's driver table entry was compiled out everywhere, and
the six tusb_config.h files that mentioned the macro all set it to 0. Maintainer
call - dead code, not a shrinking of supported classes.
Removes the sources, the usbh driver-table entry, the CFG_TUH_VENDOR default in
tusb_option.h, the tusb.h include, both build-system source lists, the rp2040
family.cmake entry and the IAR project template rows.
|
|
docs: add hardware-in-the-loop rig reference
|
|
Document the ci and hfp HIL rigs in enough detail to reproduce one: bill of
materials with photos, BIOS/IOMMU and vfio-pci passthrough on the Proxmox
host, the Renesas uPD720201 firmware install, the guest software and
permissions, the one-hub-per-root-port USB topology rule and the per-box
split of probe and DUT hubs, how CI drives the rigs, and the operational
gotchas.
The attached-board table is generated from test/hil/tinyusb.json and
test/hil/hfp.json by tools/gen_doc.py into docs/reference/hil_boards.md,
which the page includes. Sphinx excludes that partial so it is not also
built as an orphan document.
Also exclude docs/superpowers/ from the Sphinx build: it holds internal
plans, specs and handoffs rather than published documentation, and since
nothing references them from a toctree each emitted "document isn't
included in any toctree" -- 26 warnings in total, so build_doc.py -W could
never pass. It now does.
|
|
hil: one-run scheduling with JSON result handoff; audit and correct the .claude instruction surface
|
|
variant warnings
The workflow-logic harness slices hil-validate.js between marker strings (the body is not
a module; the runtime wraps it, so markers are the only handle). A renamed marker used to
produce a garbage slice and a confusing ReferenceError; it now fails naming the missing
marker, proven by mutating the marker and watching the message.
The variant-warning loop in hil_ci.sh read variant_names through a process substitution --
the exact exit-status blindness the comment in resolve_build_dirs warns about, two
functions earlier in the same file. A plain command-substitution assignment is visible to
set -e, so a malformed roster now aborts instead of silently skipping the warnings.
|
|
The doc carried its own rename instruction for when the branch gained a number;
the branch is PR #3836.
|
|
~/code/pico/pico-sdk and $HOME/code/esp-idf/export.sh are the ci rig's private layout;
written into instructions they silently stop being true on tusb, a dev PC, or any
future rig. The docs now use the variables the vendors define -- PICO_SDK_PATH for the
Pico SDK and IDF_PATH for ESP-IDF, activated explicitly as `. "$IDF_PATH/export.sh"` --
and leave where the checkouts live to each host's profile.
The variables are only useful if the shells that agents actually get can see them, and
`ssh <rig> 'cmd'` is non-interactive AND non-login: it reads no profile, and Debian's
sshd-sourced ~/.bashrc returns at the interactive guard before most of the file. The ci
rig already keeps its exports in the section ABOVE that early-return; IDF_PATH now sits
there beside PICO_SDK_PATH, and the whole chain is verified from a plain non-interactive
ssh: both variables visible, `. "$IDF_PATH/export.sh"` activates ESP-IDF v5.5.3 with
idf.py on PATH -- no login shell, no alias, no hard-coded path. hil-pool-check documents
that placement so the next rig is set up the same way.
|
|
Four writers produce hil_report.md and three of them write no JSON - the no-boards
exit, the pool-guard fallback and _abandon_exit's text prepend. Those are exactly the
runs that failed, so hil_summary.py, which builds an agent's per-board verdicts from
the sidecar, sees nothing while a human reading the markdown sees the real story. The
scope note is markdown-only too, so a three-board PR run and a full run that lost 24
boards are indistinguishable in JSON.
Five tasks: put scope in the sidecar, render the markdown from the document, give the
two early-exit paths a document, make _abandon_exit set a caveat field instead of
prepending to a file it did not write, then pin the invariant that re-rendering the
JSON reproduces the markdown byte for byte. Split out because it is a hil_test.py
reporting refactor, and the abandon path runs while the interpreter is being torn
down - it deserves its own review.
|
|
Nothing checked the agents, workflows and skills against the code they describe, and
the surface had drifted into stating incompatible rules. This records the protocol
that found the defects and what it cost.
Method: parallel subagents extract every falsifiable claim into JSONL ledgers; a
validator re-reads each cited line and rejects any ledger whose quoted text is not
there, so an extractor that paraphrases or hallucinates fails a script rather than
reaching the verification queue - 1,387 claims, zero such failures. Verification runs
only in the main session, and the same gate pointed at `citation` then checks the
verifier's own work. Hard-earned evidence is source of truth: code is authoritative
about code, experience about hardware, so claims get a fourth verdict, EARNED, and
"no backing found" is never grounds for deletion. All 1,387 claims carry a verdict;
the behavior sweep deliberately never emits CONFIRMED from a token match, because
finding a claim's vocabulary proves presence, not truth.
Every real defect came from cross-document comparison - none from any mechanical
pass. A path-existence lint was built, measured (11 flags on the audited tree, all
false positives, and the target defect is lexically identical to correct text
elsewhere), and rejected; recorded so nobody rebuilds it.
|
|
Findings from an audit of the .claude instruction surface: 1,387 falsifiable claims
extracted with a quote-gate (zero hallucinated), each verdicted against the code, the
kernel at the rig's running version, or the rig itself. Only claims the current source
actively refutes were touched; hard-earned rig knowledge stands as source of truth.
usbtest told operators to stop the actions runner before touching hardware. Every
other file forbids exactly that since the per-board flock landed - following it stops
CI on a shared rig. Twice in the same file it said hil_test.py serializes usbtest
batteries; hil_lock.py budgets 2 concurrent per host controller, a profiled
throughput trade rather than a safety ceiling - while the recorded hazards stay: an
unbudgeted battery has hard-frozen the rig through a VFIO xHCI PCIe error, and a
marginal DUT port bouncing under concurrent batteries has killed a uPD720201 outright,
which lowering the widths does not fix. It also cited src/usb_descriptors.h and
src/tusb_config.h as if repo-relative (they are the example's own, and the comment sat
above the cd that establishes the base) and presented usbtest_do_ioctl() and
tools/usb/testusb.c as repo files when both are Linux kernel.
usb-kernel-recover called the Renesas ppps "real per-port" in its rig layout while
saying four sections earlier that VBUS stays up. Both describe the same silicon and
only the second was right: owner-confirmed, the cards advertise ppps and do not
implement it, so a root cycle is purely a re-enumeration - both places now say so and
warn against reading uhubctl's flag as power control. The layout listed three cards;
the rig has five (01/03/04/05/06:00.0; AMD 02:00.0 has none), re-derived from
lspci/uhubctl/sysfs and written as a derivation recipe because bus numbers renumber
every boot. The root-cycle rung also gains the board-flock requirement the other
files already demand - it avoids the KERNEL device_lock, which is a different lock,
and the text now names the two apart instead of reading as "no reservation needed".
CLAUDE.md listed src/tusb_config.h among the key files; no such path exists -
tusb_config.h is per-example, src/tusb_option.h is the file that lives there.
|
|
script directly
A probe whose firmware has wedged reports flash-failed with the probe present and
"probe toggle unconfirmed". The tool's own recovery cannot fix that: an authorized
toggle re-enumerates but never removes power, and hil_pool_check.py:304 already notes
that ST-Link, WCH-Link, CP210x and picoprobe keep their sysfs kobject across one. So
the check correctly gives up, and the operator was left to invent the next rung.
Write it down, as what this rig's hardware actually does rather than what uhubctl
advertises: the Renesas cards list their root hubs as ppps-capable but do not
implement it (owner-confirmed - VBUS never drops, only D+/D-), so a root-port cycle
is a harder forced re-enumeration that a wedged probe can ride out, worth exactly one
attempt; and the AMD 0000:02:00.0, where the WCH-Links live, has no port-power
switching at all - nothing to cycle, straight to a physical replug. Which card a
probe hangs off decides which case applies, so the procedure starts from readlink.
The ordering rules encode the shared-rig protocol: let the full run finish (a bounce
re-enumerates siblings and corrupts checks still in flight), hold --all with this
host's --config before the cycle (hil_lock.py hold validates nothing against the
roster and nothing maps a sysfs busport to a board name, so a narrower hand-listed
hold reserves nothing while reporting success - and --all defaults to tinyusb.json,
which on the tusb rig would reserve 27 boards that do not exist there), release
BEFORE the re-check (hil_pool_check.py self-locks every board it checks, so a hold
still in place makes the verification report locked against your own hold and verify
nothing), and drive the cycle through usb_recover.sh root-cycle by its full in-repo
path - it is on no PATH and sudo's secure_path excludes the checkout. Never a bare
`uhubctl -a cycle`: without -S it writes sysfs disable, whose disable_store takes the
root hub's lock uninterruptibly and then usb_disconnect()s the wedged child - the one
input that turns a probe wedge into a bus-wide wedge. Give the script the wedged
probe's own busport, not the hub path: the serial guard and the success check both
read the path you pass, and the hub's inode always changes when its own port cycles.
Reporting asks for both passes: a final table showing every board healthy hides that
a probe needed power-cycling to get there, which is the signal that it will recur.
Also: the ESP-IDF env hints name `. $HOME/code/esp-idf/export.sh` instead of the
`get-idf` alias, which lives only in interactive shells and fails from scripts.
|
|
hil-validate ran one hil-operator per board. That parallelizes at the wrong layer:
hil_test.py already schedules boards across host controllers and budgets concurrent
flashes and usbtest batteries per controller (FLASH_PARALLEL/USBTEST_PARALLEL), and
those permits live in one process - N parallel runs multiply the budget onto the same
uPD720201 cards for no wall-clock gain over one run that already parallelizes. The
workflow now spawns ONE operator with every board as repeated -b.
The operator no longer retypes the report table. Four consecutive max-effort review
rounds found ~15 defects in this file and every one was in reconstructing board
identity from transcribed prose: report rows are named per VARIANT (nanoch32v203 only
ever produces -fsdev/-usbfs rows), a variant need not start with its board's name,
lock contention is a `board-locked` cell rather than a phrase, and each fix introduced
the next round's bug - including a fake-green test that asserted an invariant with the
one input shape that could not break it. The new helper test/hil/helper/hil_summary.py
does the join where the roster lives and emits one machine verdict per board
({board, ran, pass, locked, detail}); the operator returns that JSON verbatim plus
`wedged`, the only field it authors, and the workflow reads fields, never parses a
string. Its cell classifier mirrors hil_test.py's own tally exactly: failures are
always marked ('fail' or a ❌ prefix, TestFail's contract), everything unmarked is a
pass - a passing test may return a plain metric cell like '13443 KB/s', and the
mirrored rule is what keeps a green table from becoming a red verdict.
hil_ci.sh kept only the LAST -b, so multi-board remote runs staged one board's
binaries and every other board died on the rig after its lock and flash slot were
spent. It now parses every -b spelling argparse accepts (with the -bt arms ordered
first, longest-match, so the <config>.failed retry form is never read as a board named
"t..."), pre-flights roster membership and build dirs for ALL boards before anything
is wiped or staged, warns per declared variant with no build dir (which hil_test.py
would silently green-skip), forwards HIL_* knobs as export lines in one %q word the
remote evals ('; '-joined so it round-trips under dash - an authorized
HIL_NO_BOARD_LOCK force must not silently no-op), keeps HIL_REPORT_DIR local because
the copy-backs look in REMOTE_DIR, and copies hil_report.json and the .failed re-run
spec back beside the markdown, deleting stale local copies first so a green run cannot
leave last run's spec looking current.
Retries preserve the fleet: the documented path is the <config>.failed spec, which
already begins with --accumulate; a fresh scoped re-run would unlink the report and
collapse the whole-fleet table to the retried boards alone.
The risky logic is executable, not argued about: .claude/workflows/test-hil-validate.mjs
pins the lookup/verdict helpers and runs in pre-commit (hil-validate-logic); nine
staging tests drive hil_ci.sh through an ssh stub that models the real thing (argv
joined into one string the remote re-splits, heredoc on stdin - the naive echo-stub
passed while the feature was broken); and deliberate mutations of the verdict logic
are all caught.
Validated on the rig: a 2-board run (usbtest 30/30 on both; the pre-fix classifier,
replayed against that run's real report, fails the fully-green stm32f723disco on its
two passing '13443 KB/s' cells), the .failed --accumulate retry (merged report kept
every earlier row), and a 10-run soak over random subsets of a 22-board pool - 43
board-slots, every failure signature matched pre-existing CI state or known flake,
zero tooling failures, no locks left behind.
|
|
dcd(ci_hs): stage the device address before priming the status stage
|
|
IMXRT1060RM 42.7.23 and UM10503 Table 478 both ask for the DEVICEADDR write
with USBADRA=1 to happen after the SET_ADDRESS data phase and before the
prime of the status stage, so the controller loads USBADR from its holding
register when the status stage is ACKed. The driver did it the other way
round, leaving a window between the ENDPTPRIME store and the DEVICEADDR
store: an IN answered inside that window ACKs with USBADRA still 0, so the
holding register is never consulted and the device keeps answering on
address 0 while the host has moved to the new one. Instruction timing alone
cannot open that window, but dcd_set_address() runs in task context, so any
interrupt landing between the two stores stretches it past a microframe.
Hardware discards a staged address on a SETUP or OUT to endpoint 0 and
zeroes USBADR on a bus reset, which covers a superseded SET_ADDRESS. What
it cannot cover is a SETUP latched before this write and still unconsumed
after the full CI_HS_BUSY_SPIN spin, which refuses the prime: condition 2
already fired for that earlier SETUP, so the stage would survive and load
USBADR on the next EP0 IN ACK of an unrelated transfer. USB 2.0 9.4.6 is
explicit that "the USB device does not change its device address until
after the Status stage of this request is completed successfully", so the
refused-prime path restores the previous USBADR rather than leaving a stage
armed. Restoring the previous value rather than writing zero keeps 9.4.6's
Address-state row correct, where a device already at a non-zero address
must stay there; on Linux that write is always a no-op, since hub_set_address
only issues SET_ADDRESS from USB_STATE_DEFAULT.
Cast dev_addr before the shift: it is uint8_t, promoted to int, so an
address of 64 or more reached the sign bit of a 32-bit int.
No errata applies: IMXRT1060CE_A Rev 1.3 lists only ERR050101 and ERR010661
for USB, IMXRT1060CE_B Rev 1.1 only ERR010661.
Validated on mimxrt1064_evk: 18/19 device+host tests, 6x usbtest 30/30, and
a 100-iteration forced re-enumeration A/B that is clean on both this change
and its parent (0/100 each). All 19 ci_hs boards build; unit tests 63/63;
PVS drops one diagnostic (the sign-bit shift) and adds none.
|
|
ci(circleci): key the toolchain cache off a file that always exists
|
|
ohci: fix double allocation of dummy TDs in gtd_find_free
|
|
Restore/Save Toolchain Cache have been failing on every job whose toolchain
is hosted on GitHub - arm-gcc, arm-clang, riscv-gcc, rx-gcc, ft9xx-gcc:
Restore Toolchain Cache
template: cacheKey:1:8: executing "cacheKey" at <checksum "toolchain_key">:
error calling checksum: open /home/circleci/project/tinyusb/toolchain_key:
no such file or directory
"Set toolchain url and key" only wrote toolchain_key when the URL was not a
github.com link, but both cache steps referenced {{ checksum "toolchain_key" }}
unconditionally, so for those toolchains the key could never be computed. The
job still went green because the build step does not depend on the cache, which
is why this went unnoticed - but the two steps are permanently red and the
toolchain is re-downloaded on every single run.
Key the cache on the toolchain name plus a checksum of toolchain.json instead.
That file is in the repo, so the checksum always resolves, and the key still
invalidates whenever a toolchain URL changes. toolchain_key is no longer needed.
Side effect worth calling out: GitHub-hosted toolchains are now cached rather
than skipped. That was the intent of the removed condition, but it is also what
broke the steps - CircleCI cannot skip a cache step on a value only known at
run time. Caching them also saves the repeated download.
|
|
|
|
UAC2: Add more terminal types
|
|
|
|
dcd(ci_hs): rework bus reset handling per the reference manual, and work around ERR050101
|
|
ERR050101: while an isochronous IN endpoint is active, an IN token addressed to
that same endpoint number on ANOTHER device sharing the host can silently
unprime one of this device's OUT endpoints - control, bulk, interrupt or
isochronous alike. NXP states it cannot be detected by software and raises no
interrupt, so the endpoint simply stops answering and the transfer never
completes.
The workaround is a uniqueness requirement rather than a particular number: the
isochronous IN endpoint must not share its number with any IN endpoint in use on
the bus. One family-wide constant therefore defeats it, since two affected
boards on the same hub then pick the same number and each becomes the other's
aggressor. CFG_TUSB_MIMXRT1XXX_ERRATA_ERR050101 is set only for the parts whose
errata list it - RT1015, RT1020, RT1024 and RT1050, where it is marked no fix
scheduled, plus RT1060 and RT1064 rev A - so RT1010 and the RT11xx family keep
the ordinary number and cannot collide with an affected board beside them.
Several affected boards on one hub can still be given distinct numbers with
-DEPNUM_ISO_IN.
The guard covers every example that has an isochronous IN endpoint: audio_test,
audio_4_channel_mic, uac2_headset, cdc_uac2, usbtest, video_capture and
video_capture_2ch. The video examples move the endpoint only when streaming
isochronously, since the bulk configuration is unaffected, and video_capture_2ch
takes two numbers because it has two streams.
The macro name follows CFG_TUSB_RP2_ERRATA_E2/E4/E15 already in tree, and its
is fixed, and which cannot be told apart at compile time - a way to define it to
0.
device_issues.rst records ERR050101 against every affected part with a link to
each errata sheet, and adds the LPC55S2x USB.3 speed-detection and USB.5
isochronous IN entries, neither of which TinyUSB works around. The branch's
design notes are included under docs/superpowers.
Verified: 340 wedge-free runs on mimxrt1064_evk, which previously wedged within
hours, and the macro resolving to endpoint 0x87 on mimxrt1064_evk against 0x83
on mimxrt1010_evk and stm32f407disco.
|
|
Flip the board to device-highspeed/host-fullspeed, matching lpcxpresso55s69
and the way it is cabled on the test rig, and add it to the rig pool with
the unique id read from its flash PFR. This is the first hardware coverage
the ip3511 high-speed device path has ever had, and it immediately exposed
the clear-stall type-bit bug fixed separately.
The port swap also exposed a build gap: family.mk only linked a host
controller for port 1, so make host builds on port 0 failed with undefined
references - mirror family.cmake and link the OHCI driver there. The board's
rhport defaults now come from family.cmake's guarded ones rather than a
duplicate copy, so a -D override on the command line wins.
|
|
The 8 KB main bank is packed tightly enough that only ~280 bytes remained above
.bss, and interrupt frames overflowed into the topmost task stack - a hard fault
in cdc_msc_freertos. Put the MSP at the top of the 2 KB USB SRAM bank, which
nothing else uses in either build system, so the stack no longer shrinks as .bss
grows. The Make build's CFG_TUSB_MEM_SECTION placement of endpoint buffers into
that bank is dropped so both build systems agree on the layout.
The headroom assert is written as an addition rather than a subtraction, since
linker script arithmetic is unsigned and an overflowing bank would underflow the
difference into a huge positive value and pass silently.
|
|
DEVCMDSTAT mixes read/write fields with write-1-to-clear latches, so a blind
read-modify-write writes a pending latch back as a one and silently clears it -
a setup consumed that way strands EP0. Mask the latches on every update.
The setup path follows the manual's order: acknowledge the latch, then read the
payload. The EP0 IN interrupt is cleared along with EP0 OUT, as the control
endpoint flowchart requires - a control IN completion latched before the setup
must not reach usbd after it, where it would be applied to the request the setup
just started and arm its status stage early.
The payload is copied a byte at a time out of a buffer now declared volatile:
the controller DMAs a new setup packet into it as soon as the latch is cleared,
and C orders volatile accesses only against each other, so gcc sinks a plain
memcpy below the guard read that follows at -O2 and -O3 - leaving only -Os, the
level CI builds, correct.
|
|
A refused transfer is a recoverable condition - a new setup superseding a
control response, for instance - rather than a bug, but every failure path
treated it as one. TU_ASSERT carries TU_BREAKPOINT, which is gated on a debugger
being attached rather than on CFG_TUSB_DEBUG, so on a rig where a probe is
always attached it halted the CPU even in release builds. Use TU_VERIFY on the
control transfer paths, including the multi-packet data stage continuation, and
drop the breakpoint from the endpoint transfer failure arm, which already marks
the endpoint ready again so the next transfer can proceed.
The result of usbd_control_xfer_cb() was separately dropped on the floor,
leaving EP0 neither armed nor stalled and nothing recorded. It is logged now,
and deliberately not stalled: a DCD refuses an EP0 prime when a newer setup is
already latched, and EP0 stalls are cleared by hardware when that setup arrives,
so a stall issued here would land after the auto-clear and stall the transfer
that superseded this one. The pending setup re-drives EP0 by itself.
|
|
A bus reset was detected only from the port change that ends it, which is late:
the manual asks the DCD to clear the endpoint semaphores, cancel every prime and
free the dTDs while the reset is still being driven. Enable the reset interrupt
and do all of that there, in the manual's order (IMXRT1060RM 42.5.6.2.1,
p.2394), including the two steps that were missing - confirming the port is
still being reset, and freeing the dTDs. A failed check means the cleanup
arrived late and the controller may be in an undefined state, so the manual's
remedy is carried out rather than noted: a controller reset, followed by the
full re-initialisation it then requires, since the reset detaches the device.
The port change that ends the reset is left with what the manual gives it, the
negotiated speed, which the new BUS_RESET_END event carries. A port change is
classified by the interrupt that preceded it: a suspend raises no port change of
its own, the resume that ends it does.
Every unbounded register spin is now bounded. They waited on bits the hardware
clears within a frame, but each could hang an interrupt handler outright on a
controller that had stopped responding. The endpoint flush follows all three
steps of IMXRT1060RM 42.5.6.6.5 (p.2413), repeating a flush the controller
refuses while a packet is in progress - previously reported as success.
EP0 setup handling is hardened alongside: the payload is copied out of the queue
head through the volatile qualifier before ENDPTSETUPSTAT is cleared, since that
clear releases the setup lockout and a back-to-back setup can overwrite the
buffer immediately after, and C orders volatile accesses only against each
other, so a plain memcpy may legally be sunk past the store.
There is deliberately no unplug detection. IMXRT1060RM 42.7.31 (p.2470) states a
zero Current Connect Status means the device "did not attach successfully or was
forcibly disconnected by the software writing a zero to the Run bit ... It does
not state the device being disconnected or suspended", so a cable pull raises no
port change at all; VBUS via OTGSC is the manual's disconnect indicator and is
board dependent.
Verified on mimxrt1064_evk: 30 forced bus resets each re-enumerating at high
speed with no descriptor errors, plus repeated full usbtest batteries at 30/30
across the series.
|
|
A driver that can see reset signalling begin has no way to say so: the only
event carries the negotiated speed, which does not exist until the reset
ends. On ChipIdea that left the stack believing it was still configured for
the whole reset window - 3 ms at minimum, tens of milliseconds in practice -
while the controller had already torn its endpoints down, so a class driver
writing in that window primed a disabled endpoint over a zeroed queue head.
Add DCD_EVENT_BUS_RESET_START for the leading edge and rename the existing
event to DCD_EVENT_BUS_RESET_END, keeping DCD_EVENT_BUS_RESET as an alias.
START is optional and END stays self-sufficient, so every other driver and
the unit tests are untouched.
|
|
This adds more standard terminal types that can
be used in descriptors to improve end user experience
when operating system can present more acurate
image for audio device terminal.
Signed-off-by: Jerzy Kasenberg <[email protected]>
|
|
read-doc skill: search the document library's database instead of the filesystem
|
|
Finding documents by walking the library tree misses anything the filename
does not carry - Calibre stores only a truncated title and the author there, so
the tags, series, publisher and description that hold most part numbers and
errata IDs are invisible to it. A zero-result tree search then reads as "the
document does not exist" rather than as a bad search; that happened here, and
led to a confident claim that a fully populated 14,000-file library was empty.
search.py queries metadata.db, ANDs its keywords across every metadata field
(including the stored filename), and prints the best matches first with the
exact path to read. Matching is NFKC + casefold, so a typed ASCII apostrophe or
mu reaches the titles that store the typographic ones.
Every printed path is checked on disk. Calibre renames <author>/<title> (<id>)
when metadata is edited and leaves the old directory behind, so a miss retries
by the stable book id before reporting MISSING - which distinguishes "the file
is not here right now" from "no such document". The gate tests for metadata.db
rather than the directory, since an unmounted or half-synced mountpoint is
still a directory.
Consumers that prescribed their own tree search - driver-reviewer, port-dev,
the driver-review workflow, and the calibre-library references in CLAUDE.md,
usbtest, etm-trace and target-debug - now point at the skill, which owns the
library's location.
|
|
hil, ci: contain a wedged USB stack instead of stranding the runner
|
|
Records the convention in CLAUDE.md -- deferred work is a SEPARATE scope that deserves
its own PR, written by another session, so it is handed off as a writing-plans doc in
docs/superpowers/followup/pr<NNN>-<topic>.md rather than accumulated in the PR that
found it.
Five handoffs from #3803: flasher_recover (convoy-safe recovery for J-Link boards, seven
validated on the rig), the blindness reporting gaps, the usbtest recovery reserve, the
IAR re-run spec, and the pci-rebind stranding question. Each carries what is already
established with its citations and measurements, what remains, and why it was split out.
One doc per follow-up, not one per PR: a per-PR file invites unrelated work into the
same document and rots as a unit.
|
|
Two things the rig taught us that the old guidance got wrong.
A usbfs ioctl wedged in D state cannot be freed on a running kernel. It holds
the device lock, so usb_disconnect() blocks behind it; reboot(2) walks
device_shutdown() and takes the same lock, so every userspace reboot stalls too.
Only sysrq b (emergency_restart, which skips device_shutdown) or hypervisor
action clears it -- all cited to the kernel source. The recovery ladder is
generic across rigs now (ci.lan, hifiphile, a bench PC) and ends at hypervisor
escalation only where host access exists. Two claims are corrected outright:
JLinkExe is NOT convoy-safe, and a park-flash cannot free a device-lock owner.
The hil skill's banner list is what an operator agent matches a report against,
so it enumerates the banners that actually exist, including the D-state note --
which is explicitly NOT a wedge, since a healthy in-flight testusb is
uninterruptible for most of every case and a concurrent CI battery would
otherwise turn a clean run red.
|
