| Age | Commit message (Collapse) | Author |
|---|
|
hil_test.py cannot run on Windows and never could: it imports helper.hil_lock,
whose module-level `import fcntl` is POSIX-only, so the harness fails at import
before a line of it executes. Past that it reads /sys/bus/usb, /dev/bus/usb,
/dev/serial/by-id and /proc, kills by process group, and takes flock board
locks -- none of which Windows has.
So the guards were protecting a platform the code cannot reach:
- run_cmd branched three ways on os.name to decide whether to set
start_new_session and whether to killpg. The non-POSIX arm called p.kill()
instead, which kills only the direct child -- exactly the semantics the whole
containment design rejects, since a flasher run through a shell reparents out
of reach. Dead code that documented the wrong answer.
- hil_test picked multiprocessing's default context on Windows "so it still
IMPORTS there". It does not import there.
- test_device_audio_test_freertos returned 'skipped' on nt before touching
ALSA, in a function only ever reached from a worker that cannot start there.
- Seven @unittest.skipIf(os.name == 'nt') decorators across the two suites.
These were the only ones with a real effect -- the unit tests DO import and
run on Windows, because they stub pyserial and mostly exercise pure logic --
but what they buy is a partially-green suite for a harness that cannot run,
and nothing verifies the set is correct: the hil-test hook only ever runs on
ubuntu-latest, so a missing guard fails silently until someone tries.
Removing them makes the POSIX assumption single and explicit rather than
scattered and half-honoured. Nothing changes on Linux: every removed branch was
the one already taken there.
Removing the run_cmd guards also removes their `else: p.kill()` arms. Those were
the Windows branches, and p.kill() reaches only the direct child -- a flasher run
through a shell keeps grandchildren it cannot touch, which is the semantics this
containment design rejects. RunCmdCleanupShape pins what is left: both cleanup
paths killpg, no try carries an else whose body would run when the kill
SUCCEEDED, and the BaseException path still re-raises. Structural rather than
behavioural because driving a real SIGINT into a blocked communicate() is
timing-dependent, and what actually breaks this block is an edit that rebinds a
branch -- which is a shape.
|
|
(#3852)
hid_generic_inout was the last unbounded blocking IO in the file. hidapi's
hidraw backend reads manufacturer/product via udev for each device reaching
create_device_info_for_device, both usb_string_attr served under the device
lock a wedged usbfs ioctl holds — and every DUT here is VID cafe, so a wedged
sibling stalls the walk.
A thread cannot bound it: cython-hidapi calls hid_open and hid_close bare
(0.15.0 hid.pyx), so they hold the GIL and the waiter can never resume.
Measured — a 1.0s bound never returned. run_cmd's killpg reaches a child
regardless; it gains an argv form for the -c body.
Filters on both ids: hidapi only runs the free uevent pre-check when ids are
passed (linux/hid.c:962), so an unfiltered walk sends every device straight to
the locked reads. Tests stall via ctypes.PyDLL, which unlike CDLL holds the
GIL — the shape a thread bound cannot cover.
|
|
Three readability changes with no behaviour change on the healthy path —
every pre-existing test passes untouched.
main() was 368 lines with try/finally three deep, its two abort paths
near-identical 40-line blocks; _abort_report holds that shape once, and the
controller-hint cache and pool construction move to their own helpers.
368 -> 279, test_board 180 -> 151, test_device_usbtest 164 -> 125.
test_hil_bounded.py cost 78s on every commit under test/hil/, mostly one 3s
post-flash settle paid by ten tests against a fake rig. Now 37s.
Fixes two pre-existing defects the extraction exposed: _write_failed_spec was
unguarded inside the abort path, so an OSError there replaced the caller's
RuntimeError and no report was written at all; _save_controller_hints overlaid
a startup snapshot onto the re-read cache, clobbering a concurrent job's newer
values. Also five comments that stated the opposite of the code, and both table
renderers measuring width with len() against two-column status marks.
|
|
hil_report.json and hil_report.md were written independently. Four writers
produced the markdown and three wrote no JSON at all -- and those three are the
paths where a run died, so a JSON consumer saw nothing exactly when it mattered:
the per-board verdicts an agent hands back reported the whole fleet as "no
report row" while a human read the real story from the markdown.
Every writer now goes through render_report(), so a table can never contain
something the JSON does not. The document gains `scope` (a three-board PR run
and a full run that lost 24 boards were indistinguishable) and `caveat` (how the
run ended). `banner` carries rig health across an --accumulate retry; `caveat`
records how a run ended and must not -- conflating them made a clean retry
publish an abandonment that never happened.
helper/hil_report.py owns the document end to end, dissolving the import cycle
that forced write_timeout_report to compose its own markdown and removing a
duplicate cell classifier kept in sync by hand. hil_summary.py is deleted; its
CLI moves there. hil_ci.sh uploads the sidecar so a remote --accumulate has a
merge bas
|
|
taking the rig (#3846)
test_the_upload_board_can_diverge_from_the_built_board called
get_family_boards without ci=True, so it pinned the developer's set, not the
runner's: the CI skip lists move the one-first pick on three families. It
held locally and went red on its first CI run. Pass ci=True, as
_prune_buildable already does, and pin the runner's twelve.
Rule 2 is a bare test/hil/ prefix, so the harness's own unit tests booked the
full 27-board rig for diffs that cannot reach it. Carve test/hil/test/** out
to rule 1b, beside test/{fuzz,unit-test}/**; the harness itself is untouched.
A test pins that directory's file list, so anything added there that the rig
does read fails rather than silently skipping hardware. Rule table updated in
the spec and its carbon in the docstring.
|
|
ci: an empty selection must build nothing, plus selector follow-ups
A PR whose build axis legitimately selected nothing rebuilt everything.
build.yml reads .build.families twice - as a |-joined regex, and implicitly
as "is anything selected" - but tested only -z "$FAMILY_REGEX", which an
empty list and a charset-rejected one both satisfy while meaning opposite
things. ci_set_matrix had already returned the correct all-empty matrix;
the fall-open branch discarded it. #3842 and #3840 each spent 74 cmake legs
on it. Branch on the two cases instead, rename FAM_* to FAMILY_*, and cover
the block with a test that extracts it from build.yml and executes it - it
had no test at all, which is how this shipped through two merges.
Follow-ups to the same machinery: glob.escape the repo root at five sites,
so a checkout path containing [ or * stops failing closed; drop the ci-full
label, read after the matrix was already computed and so never functional;
delete 13 mcu:MKL25ZXX / mcu:SAME5X skip tokens matching no board; carry the
rule table in the module docstring, guarded against drift; and pin six
selection behaviours a mutation pass proved untested.
Cut the selector's cost 1.8x (26.0s -> 14.6s) with 0 divergences over 260
paths, and stop scoping the membrowse upload by the PR example filter.
|
|
Anchor the .github file alternatives. FUNDING.yml, labeler.yml and
membrowse_pr_message.j2 sat inside a group whose only `$` belonged to the
workflows/ branch, so they matched as prefixes: .github/labeler.yml.bak and
.github/FUNDING.yml.old were classified as metadata and would have selected
nothing. No such file exists today - the workflows/ alternative was already
anchored and ISSUE_TEMPLATE/ is a directory prefix on purpose.
Rule 12b had no test of its own: TestNoTrackedFileIsUnclassified only proved
src/typec no longer reaches rule 17, not that the answer is right. TestTypecRule
pins it - non-full, every selected example under typec/, all four src/typec files
answering alike, no rig board, and the set derived from CFG_TUC_ENABLED rather
than hardcoded, so it follows a new typec example on its own. Verified all four
fail with rule 12b removed.
|
|
The eight lru_cache layers take repo-RELATIVE paths - 'hw/bsp/<fam>',
'examples/<ex>/skip.txt', the literal 'hw/bsp' glob - while ci_select._in_repo()
chdirs around every call so one process can classify more than one tree. With no
cwd in the key the second tree gets the first tree's answers.
Reproduced: skip_example('host/bare_api','metro_m0_express') is False at the repo
root and STILL False after chdir into a tree where that board does not exist; only
cache_clear() gave the right answer. It bites the code-size skill's base-vs-branch
worktree compare, /pre-pr, and the first test that points classify_build at a
fixture tree. Master had no caching here, so the hazard arrived with it.
_cwd_cache puts os.getcwd() in the key. The 199-test suite passed before only
because every test happens to pass the real REPO; the new
TestCachesAreKeyedOnTheTree crosses trees deliberately.
Also adds the drift guard the class rule was missing. Ports, hw/mcu, get_deps
tokens and bsp families each have one; the class rule had only a comment claiming
vendor_host.c was the sole "enabled by no example config" case until its removal -
which src/class/bth falsifies today. TestClassesWithNoEnablingExample pins the set
to {bth}, so a class added before its first example, or an example config flipped
to 0, fails here instead of silently selecting nothing on both axes. Verified it
fires by adding a class dir nothing enables.
|
|
Rule 17 (unclassified -> full on both axes) is the fail-open net for paths nobody
anticipated, and it must stay that way: a wrong `full` costs runner minutes and is
visible in the run, a wrong `empty` costs a merged regression and is invisible.
But nothing in the tree should REACH it, and 254 tracked files did.
The cost was real. PR #3842 changed a skill, a README and .gitignore; .gitignore
matched no rule, so both axes went full and 74 cmake legs span up runners to do
checkout + toolchain + get_deps before skipping the build, plus the whole 30-board
rig. Three changes, none of which touch rule 17 itself:
1. _META_RE - repo metadata and tooling no Build step reads: .gitignore,
.gitattributes, .clang-format, .codespellrc, .pre-commit-config.yaml,
.readthedocs.yaml, .PVS-Studio/, .idea/, sonar-project.properties, the
packaging manifests, CMakePresets, udev rules, test/{fuzz,unit-test} (their own
jobs build those), the non-build .github/ files, and the tools/*.py scripts no
build invokes. Deliberately NOT included, and still full: .circleci/**,
.github/workflows/build*.yml, .github/actions/**, .github/scripts/**. The line
is "does a Build step read this", not "is it source".
2. Rules 15 and 16 now match what they already claimed. Row 15 names
examples/<role>/CMakeLists.txt and the regex never had it; row 16 says
tools/build*.py but anchored tools/build\.py$. Both got the right answer only
because rule 17 caught them on the way past. Also names their siblings -
family_support.mk, family_rules.mk, src/CMakeLists.txt, src/tinyusb.mk - and
.circleci/**, which generates the whole CircleCI matrix and was in no row at all.
3. src/typec/** gets row 12b. It is listed unconditionally by both build systems
but its body is `#if CFG_TUC_ENABLED`, which only examples/typec/power_delivery
sets - the same shape as the class rule, so the same answer: the examples that
enable it (stm32g4 and stm32u5 after the buildability prune), and nothing on the
rig, which runs no typec test. It was force-fulling 82 families and all 30 boards.
TestNoTrackedFileIsUnclassified walks every tracked file and asserts none reaches
rule 17, on both axes - 254 -> 0. Verified it fails when a new unclassified path
appears. That turns 17 into what it should be: unreachable for anything in the
tree, so it fires only for genuinely new shapes, and the author is told to write
the row rather than letting the fall-through pick an answer for them.
test_full_paths used sonar-project.properties as its stand-in for "unclassified";
that is now metadata, so the case moved to the new
test_repo_metadata_is_not_a_build_input, with test_the_build_machinery_is_still_full
pinning the other side of the line.
|
|
family_list maps a family to the toolchains that build it, and seven hw/bsp
families are in neither: cxd56, efm32, espressif, f1c100s, pic32mz, py32f0,
same7x. Scoping to one of them intersected to nothing, so every toolchain key was
[], every cmake leg skipped on `if: inputs.build-args != '[]'`, code-metrics took
its no-metrics branch, and the PR went green from a build job that ran no
compiler. The only signal was a stderr line nothing greps for.
Not a coverage regression - master gave the same diff no compile coverage either,
since none of the other families compiles same7x's board.h. What is new is that
the gap used to be masked by the full matrix and is now the whole answer, and
that green now means "ran no compiler" rather than "compiled 64 families".
A selection whose families ALL miss is now unusable rather than empty: it prints
UNSCOPED, which build.yml and .circleci/config.yml already grep to drop the build
extras with it, and emits the full matrix. The two neighbouring cases keep their
own answers - an explicit families: [] is still a legitimate nothing-selected, and
a partial miss still scopes to the families that do build, noting the rest.
The contract test pinned an exact count of fall-open markers, which this would
have broken; it now pins the invariant (every message that emits the full matrix
carries the marker) and was checked to still fail when a marker is removed.
Also corrects the drift guard's note about espressif: hil-build-esp builds its
boards by name, but that job is gated on repository_owner, so on a fork an
espressif-only PR builds nowhere.
|
|
The roster had two ways to pass a cmake -D to a board's build: `build.args`,
applied to every variant, and `variant[].defines`, applied to one. They did the
same thing, and only metro_m4_express used the first - for MAX3421_HOST=1, which
is what makes it the one rig board that compiles hcd_max3421.c.
A board whose define is always on now carries a single variant named after itself,
which is exactly the shape `board.get('variant') or [{'name': name, 'flags': ''}]`
already synthesises everywhere - so the build dir, the HIL report row and the
variant-boundary handling are unchanged. raspberry_pi_pico has used that shape
for its flags all along.
Removes the BuildCfg type and the parallel code path from all four consumers:
hil_test.build_board, hil_pool_check's two builders, hil_ci_set_matrix and
ci_select.board_options.
Verified: the hil-build matrix entry is byte-identical
(`-b metro_m4_express -DMAX3421_HOST=1`), hil_test's build command is unchanged,
ci_select still selects the board for a max3421 diff with MAX3421_HOST in its
options, and a real build of dual/host_info_to_device_cdc and host/cdc_msc_hid on
that board still compiles hcd_max3421.c.
|
|
Every one of these dropped coverage silently - the worst failure mode here,
because the PR still goes green. Found by review, each reproduced first.
Selection rules:
* class_macros derived the config macro from the class DIRECTORY, so a change to
src/class/midi/midi2_device.c selected the midi_test examples (which do not
compile it) and never examples/device/midi2_device (the only one that enables
CFG_TUD_MIDI2, and the only one that does). The file's own macro is unioned in
where it differs - union, never replace: over-selecting costs a build,
under-selecting merges a break.
* the ${FAMILY_MCUS} fallback added for espressif fired on any family whose
_family_mcus came back empty, and _cmake_sets is if()-blind and keeps the FIRST
definition - so mcx/frdm_mcxn947 answered MCXA15, a token six examples' skip.txt
names, dropping 12 firmware images CMake builds. Limited now to families that
never spell set(FAMILY_MCUS ...) at all.
* lib_examples read only an example's top-level CMakeLists.txt/Makefile;
host/msc_file_explorer_freertos names lib/embedded-cli in src/CMakeLists.txt and
survived by luck. The whole example tree is scanned. (SEGGER_RTT and rt-thread
still resolve to nothing: all three references sit inside a LOGGER=rtt guard no
CI build sets - the documented ruling, not a miss.)
* get_family_boards applied ci_skip_boards/ci_preferred_boards only under
GITHUB_ACTIONS/CIRCLECI, so the selector answered differently on a laptop than
on a runner; _prune_buildable forces CI semantics. Its one-board pick also
abandoned the whole preferred list when entry one could not build the -e set,
and asked skip_example without the build's -D tokens.
* _config_enables and lib_examples still read with the locale encoding - under
LC_ALL=C the selector tracebacked on three tracked tusb_config.h files. The
whole selector and its suite run clean there now.
Workflows:
* the Membrowse Upload step omitted $EX_ARGS, but --one-first now picks the board
from the -e set, so it configured a different, empty build dir and uploaded
--identical for a board never compiled. It takes $EX_ARGS for the BOARD; the
target stays the aggregate, which has no DEPENDS and still records every example.
* blanking FAM_REGEX reset only build_filtered, leaving the build scoped while
code-metrics took the UNSCOPED branch and diffed a 1-family run against the full
averaged baseline. All three drop together now, as CircleCI's fall-open does.
* CircleCI's EX_ARGS had no character screen and is used unquoted, and its
code-metrics job still exit 1'd on an empty metrics set - which a scoped build
makes a legitimate outcome.
* a `ci-full` PR label now turns the scoping off for one PR. A selector bug
under-selects silently, and without a label the only ways back to a full matrix
are accidental.
Performance, since the selector gates every other job: family.cmake texts are read
once rather than per changed directory (a 6,000-file dep bump re-read 84 files
99,892 times) and _scrape_mcu is cached: 2.2s -> 0.29s there, 0.8s -> 0.33s on a
class diff.
Tests: a drift guard for hw/bsp families absent from ci_set_matrix.family_list
(they select zero legs now, where they used to ride the full matrix); the rule-4
port test asserted a SUBSET, which set() satisfies, so it could not fail on the
empty selection it exists to catch; the GITHUB_ENV guard test counted a SUM of two
guards. Drops metrics.py's --only-examples, which nothing called, and applies the
TOTAL scrub to the by-example branch that skipped it.
|
|
Every PR built all 74 legs (2494 example builds on GHA cmake alone) and flashed
all 30 rig boards, whatever it touched. One classifier now walks the PR diff twice
and answers three questions: which families to build, which examples per family,
and which boards run which tests. Fail-open throughout - anything no rule
classifies, any exception, any unusable output falls back to the full matrix, and
a master push always builds everything.
test/hil/helper/hil_select.py moves to tools/ci_select.py: it is no longer HIL-only,
and tools/ is where the build side can import it. test_hil_select.py follows it as
test_ci_select.py.
Rules (docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md holds the
full table): a port selects the families whose family.cmake references it, and its
role - a dcd change skips host examples and vice versa; a class selects only the
examples whose tusb_config.h enables its CFG_TU[DH]_ macro, following cross-class
includes; an example selects itself; hw/bsp selects its family or board; hw/mcu and
lib select whoever references them. CMake is the reference for all of it - make
follows whatever cmake decides, family.mk is never scanned.
Empty means empty (maintainer ruling): a rule that classifies a path to nothing
selects nothing. Ports no family references, classes no config enables, libs no
example builds and hw/mcu paths that resolve nowhere are all real - nothing
compiles them, so nothing can validate them, and the master-push build is the net.
Structural tests pin each such case with an explicit allowlist, so the day one
stops being empty it fails pre-commit instead of silently narrowing CI.
Per-example builds: build.py grows a repeatable -e, resolved against the targets
CMake actually registered and batched into one `cmake --build --target a b c`.
build_utils mirrors CMake's family_filter (the whole FAMILY_MCUS list, ${...} and
string(TOUPPER ...) resolved) for the cmake side, while the make side keeps
master's algorithm verbatim - the two build systems answer differently and a shared
answer breaks lpc54's make link. hil-build gains this even on a full selection:
1702 example builds become 515.
Transport: the selection travels as a file, never an argv or env var - a mass-sweep
diff selects 261 KB against a 128 KiB exec limit, and E2BIG would fail the step
before its own fallback could run. CircleCI carries the example map inside the
generated config (pipeline parameters cap at 512 chars), swapped into the parameter
defaults by sentinel match, and drops the scoping wholesale if that rewrite fails.
Every PR-derived value written to $GITHUB_ENV/$GITHUB_OUTPUT is character-screened.
Code metrics follow the scoping: metrics.py emits per-example totals, and
metrics_pair_compare compares the (board, example) pairs present on both sides
instead of a scoped run against a full-matrix average.
The selector's own suite gates it in both providers: a selector that exits 0 with
valid-but-wrong JSON is the one failure fail-open cannot catch, so a red suite
means the full matrix.
|
|
variant warnings
The workflow-logic harness slices hil-validate.js between marker strings (the body is not
a module; the runtime wraps it, so markers are the only handle). A renamed marker used to
produce a garbage slice and a confusing ReferenceError; it now fails naming the missing
marker, proven by mutating the marker and watching the message.
The variant-warning loop in hil_ci.sh read variant_names through a process substitution --
the exact exit-status blindness the comment in resolve_build_dirs warns about, two
functions earlier in the same file. A plain command-substitution assignment is visible to
set -e, so a malformed roster now aborts instead of silently skipping the warnings.
|
|
~/code/pico/pico-sdk and $HOME/code/esp-idf/export.sh are the ci rig's private layout;
written into instructions they silently stop being true on tusb, a dev PC, or any
future rig. The docs now use the variables the vendors define -- PICO_SDK_PATH for the
Pico SDK and IDF_PATH for ESP-IDF, activated explicitly as `. "$IDF_PATH/export.sh"` --
and leave where the checkouts live to each host's profile.
The variables are only useful if the shells that agents actually get can see them, and
`ssh <rig> 'cmd'` is non-interactive AND non-login: it reads no profile, and Debian's
sshd-sourced ~/.bashrc returns at the interactive guard before most of the file. The ci
rig already keeps its exports in the section ABOVE that early-return; IDF_PATH now sits
there beside PICO_SDK_PATH, and the whole chain is verified from a plain non-interactive
ssh: both variables visible, `. "$IDF_PATH/export.sh"` activates ESP-IDF v5.5.3 with
idf.py on PATH -- no login shell, no alias, no hard-coded path. hil-pool-check documents
that placement so the next rig is set up the same way.
|
|
script directly
A probe whose firmware has wedged reports flash-failed with the probe present and
"probe toggle unconfirmed". The tool's own recovery cannot fix that: an authorized
toggle re-enumerates but never removes power, and hil_pool_check.py:304 already notes
that ST-Link, WCH-Link, CP210x and picoprobe keep their sysfs kobject across one. So
the check correctly gives up, and the operator was left to invent the next rung.
Write it down, as what this rig's hardware actually does rather than what uhubctl
advertises: the Renesas cards list their root hubs as ppps-capable but do not
implement it (owner-confirmed - VBUS never drops, only D+/D-), so a root-port cycle
is a harder forced re-enumeration that a wedged probe can ride out, worth exactly one
attempt; and the AMD 0000:02:00.0, where the WCH-Links live, has no port-power
switching at all - nothing to cycle, straight to a physical replug. Which card a
probe hangs off decides which case applies, so the procedure starts from readlink.
The ordering rules encode the shared-rig protocol: let the full run finish (a bounce
re-enumerates siblings and corrupts checks still in flight), hold --all with this
host's --config before the cycle (hil_lock.py hold validates nothing against the
roster and nothing maps a sysfs busport to a board name, so a narrower hand-listed
hold reserves nothing while reporting success - and --all defaults to tinyusb.json,
which on the tusb rig would reserve 27 boards that do not exist there), release
BEFORE the re-check (hil_pool_check.py self-locks every board it checks, so a hold
still in place makes the verification report locked against your own hold and verify
nothing), and drive the cycle through usb_recover.sh root-cycle by its full in-repo
path - it is on no PATH and sudo's secure_path excludes the checkout. Never a bare
`uhubctl -a cycle`: without -S it writes sysfs disable, whose disable_store takes the
root hub's lock uninterruptibly and then usb_disconnect()s the wedged child - the one
input that turns a probe wedge into a bus-wide wedge. Give the script the wedged
probe's own busport, not the hub path: the serial guard and the success check both
read the path you pass, and the hub's inode always changes when its own port cycles.
Reporting asks for both passes: a final table showing every board healthy hides that
a probe needed power-cycling to get there, which is the signal that it will recur.
Also: the ESP-IDF env hints name `. $HOME/code/esp-idf/export.sh` instead of the
`get-idf` alias, which lives only in interactive shells and fails from scripts.
|
|
hil-validate ran one hil-operator per board. That parallelizes at the wrong layer:
hil_test.py already schedules boards across host controllers and budgets concurrent
flashes and usbtest batteries per controller (FLASH_PARALLEL/USBTEST_PARALLEL), and
those permits live in one process - N parallel runs multiply the budget onto the same
uPD720201 cards for no wall-clock gain over one run that already parallelizes. The
workflow now spawns ONE operator with every board as repeated -b.
The operator no longer retypes the report table. Four consecutive max-effort review
rounds found ~15 defects in this file and every one was in reconstructing board
identity from transcribed prose: report rows are named per VARIANT (nanoch32v203 only
ever produces -fsdev/-usbfs rows), a variant need not start with its board's name,
lock contention is a `board-locked` cell rather than a phrase, and each fix introduced
the next round's bug - including a fake-green test that asserted an invariant with the
one input shape that could not break it. The new helper test/hil/helper/hil_summary.py
does the join where the roster lives and emits one machine verdict per board
({board, ran, pass, locked, detail}); the operator returns that JSON verbatim plus
`wedged`, the only field it authors, and the workflow reads fields, never parses a
string. Its cell classifier mirrors hil_test.py's own tally exactly: failures are
always marked ('fail' or a ❌ prefix, TestFail's contract), everything unmarked is a
pass - a passing test may return a plain metric cell like '13443 KB/s', and the
mirrored rule is what keeps a green table from becoming a red verdict.
hil_ci.sh kept only the LAST -b, so multi-board remote runs staged one board's
binaries and every other board died on the rig after its lock and flash slot were
spent. It now parses every -b spelling argparse accepts (with the -bt arms ordered
first, longest-match, so the <config>.failed retry form is never read as a board named
"t..."), pre-flights roster membership and build dirs for ALL boards before anything
is wiped or staged, warns per declared variant with no build dir (which hil_test.py
would silently green-skip), forwards HIL_* knobs as export lines in one %q word the
remote evals ('; '-joined so it round-trips under dash - an authorized
HIL_NO_BOARD_LOCK force must not silently no-op), keeps HIL_REPORT_DIR local because
the copy-backs look in REMOTE_DIR, and copies hil_report.json and the .failed re-run
spec back beside the markdown, deleting stale local copies first so a green run cannot
leave last run's spec looking current.
Retries preserve the fleet: the documented path is the <config>.failed spec, which
already begins with --accumulate; a fresh scoped re-run would unlink the report and
collapse the whole-fleet table to the retried boards alone.
The risky logic is executable, not argued about: .claude/workflows/test-hil-validate.mjs
pins the lookup/verdict helpers and runs in pre-commit (hil-validate-logic); nine
staging tests drive hil_ci.sh through an ssh stub that models the real thing (argv
joined into one string the remote re-splits, heredoc on stdin - the naive echo-stub
passed while the feature was broken); and deliberate mutations of the verdict logic
are all caught.
Validated on the rig: a 2-board run (usbtest 30/30 on both; the pre-fix classifier,
replayed against that run's real report, fails the fully-green stm32f723disco on its
two passing '13443 KB/s' cells), the .failed --accumulate retry (merged report kept
every earlier row), and a 10-run soak over random subsets of a 22-board pool - 43
board-slots, every failure signature matched pre-existing CI state or known flake,
zero tooling failures, no locks left behind.
|
|
Flip the board to device-highspeed/host-fullspeed, matching lpcxpresso55s69
and the way it is cabled on the test rig, and add it to the rig pool with
the unique id read from its flash PFR. This is the first hardware coverage
the ip3511 high-speed device path has ever had, and it immediately exposed
the clear-stall type-bit bug fixed separately.
The port swap also exposed a build gap: family.mk only linked a host
controller for port 1, so make host builds on port 0 failed with undefined
references - mirror family.cmake and link the OHCI driver there. The board's
rhport defaults now come from family.cmake's guarded ones rather than a
duplicate copy, so a -D override on the command line wins.
|
|
A wedged USB device used to take the whole HIL run with it. Every worker that
touched the poisoned node blocked uninterruptibly, the pool could not be joined,
map_async discarded every board's result, and the job ran to the GitHub ceiling
with no report at all -- while the self-hosted runner's single job slot stayed
occupied and every queued job waited behind it.
Bound the calls a worker makes itself. read_sysfs, bounded_open and run_cmd all
answer within a wall clock; read_sysfs distinguishes "absent" from "unknown",
because a blocked read is not evidence of absence, and caps stranded readers at
four (each costs a thread and an fd for the life of the process) after which the
worker declares itself blind. mtype, the gio unmount, the libmtp session and the
arecord/iperf reaps go through those bounds; the MTP session runs in a disposable
subprocess, since libmtp's ctypes calls block unkillably in D state.
Bound the run. A pool guard (HIL_POOL_TIMEOUT, 60 min) fires before any job
ceiling and still writes a report. When the pool will not shut down, the sweep
kills what the workers spawned -- descendants, not just direct children, since
flashers run in their own session -- confirms each kill actually landed, and
exits early so the runner is freed. Whatever survived is named in the report.
Deliberately shallow past that point. We do not re-scan process groups, prove
pid ownership, or escalate through sudo: a root-owned survivor is reported, not
force-killed, because signalling a pid we cannot prove is ours is the worse
failure, and the job ceiling backstops whatever this misses. A D-state holder
was never killable anyway.
Recover instead of reporting a wedge. A HUNG usbtest case reflashes its own DUT
through its roster flasher, but only where the flasher can reach its probe past
a poisoned node -- openocd pinned to a validated vid_pid, or esptool. Where it
cannot, the run says so rather than reserving budget for a path that cannot fire.
Raise the CI ceilings above the pool guard so the guard fires first and still
writes its report, and pin --retry 1 on every HIL leg: the guard is a flat
constant and does not scale with max_retry, so argparse's default of 3 would
triple the serialized usbtest tail against an unchanged guard.
Split the module: execution in hil_test/hil_flash/usbtest, infrastructure in
helper/ (locking, health, selection, shared bounded IO), and the two matrix
generators into .github/scripts/ -- ci_set_matrix.py sat in workflows/, where
GitHub treats every file as a workflow definition. 193 tests cover the bounded
paths, the kill ladder, the guard and the selector against synthetic /proc trees
and PATH-injected fakes; a real wedge cannot be manufactured on demand.
|
|
Fix HFP HIL reliability issue
|
|
An XFER_COMPLETE dropped by a full event queue leaves its endpoint's
BUSY|CLAIMED state set forever - the consumer that normally clears it
never sees the event, so usbd_edpt_claim()/usbd_edpt_xfer() fail from
then on and the class never re-arms the endpoint. Clear both flags when
the enqueue fails: the completion is lost either way, but the endpoint
stays usable.
Unit test: arm a bulk endpoint, drop its completion against a full
queue, verify the endpoint can be claimed and re-armed.
|
|
A SETUP arriving while the event queue is full is silently dropped by
queue_event(), but _usbd_queued_setup has already been incremented. The
leaked count makes the event handler skip every subsequent SETUP
("Skipped since there is other SETUP in queue") forever: EP0 stays deaf
until tud_init() while the device otherwise looks alive - enumerated,
endpoints armed. Undo the increment when the enqueue fails.
Unit test: fill the queue so a SETUP is dropped, then verify the next
SETUP still completes a GET_DESCRIPTOR control transfer.
|
|
morse-cedricvandenbergh/fix/ncm-link-state-notify-retry
ncm: retry link-state notification, fix carrier lost on collision
|
|
The self-contained net_ncm fuzz harness #includes ncm_device.c and stubs
the usbd symbols it references rather than linking the device stack.
tud_network_link_state() now calls usbd_defer_func(), so add a matching
no-op stub to keep the harness linking.
|
|
by flasher extension (#3804)
test/hil: one openocd flasher, per-board verify and firmware extension
The four WCH boards move to `openocd`, leaving one flasher for all.
`verify` is now a per-board opt-out, not dropped fleet-wide: WCH cannot read flash back
over the WCH-Link sdi transport; the other seven openocd boards can, and say so explicitly.
FLASHER_SUFFIX decides each flasher's extension once — find_firmware returns the full path
and the flashers pass it through, so a build with only the wrong artifact is skipped rather
than failed mid-flash. --skip-flash bypasses the filter.
rescue_openocd() power-on-resets a wedged RP2040/RP2350 via its Rescue DP from the flash
retry; the probe has no reset line.
Drops unused openocd_adi, stflash, wlink_rs and uniflash, parks the unstable ra6m5_ek, and
tests that every roster flasher name dispatches.
|
|
|
|
hil, ci: scope HIL builds and tests to the boards a PR affects
Add test/hil/hil_select.py, a stdlib-only selector that maps a PR diff to the
rig boards, tests and BSP families a change can affect, and wire it into CI so
pull requests build and run only those. A port change picks its families' boards,
a class change picks the examples enabling that class, and device/host changes
prune the other role. Anything unclassified — infra, an unmapped port, a selector
error — falls back to the full matrix, and push/schedule runs are untouched.
Move the shared example lists to hil_examples.py; 54 hardware-free tests cover
the rules.
|
|
|
|
probes (#3794)
test/hil: add board-pool health check, split hil_test into focused modules (#3794)
Add test/hil/hil_pool_check.py: per-board rig health scan — probe presence,
light-example flash (dfu_runtime; device_info + serial check for host-only
boards), uid re-enumeration, safe recovery (probe authorized-toggle, board
reset), verified board_test re-park, USB topology report, and a markdown
summary table. Missing firmware is built on the spot (tools/build.py, idf.py
for espressif, one get_deps retry); row statuses: ok, flash-failed, failed,
locked. Board locks are always respected, never bypassed.
Refactor hil_test.py into hil_lock.py (flock protocol, controller permits,
hold/release/status CLI; replaces board_lock.py) and hil_flash.py (flashers,
find_firmware, run_cmd). Update WCH probe uids and the board roster in
tinyusb.json; add the hil-pool-check skill.
|
|
|
|
|
|
|
|
|
|
|
|
This reverts commit 80ffbff6e98a9c5053bba008ae2c5087f0351300.
|
|
Signed-off-by: Zixun LI <[email protected]>
|
|
bsp, hil: flash with the unified OpenOCD fork
https://github.com/hathach/openocd (branch tinyusb) is mainline plus every
config these boards need: RPi RP2350, ADI max32/max78, the MounRiver WCH
configs, and the wlinke adapter on mainline's riscv target. It is a superset
of the vendor forks, so one 'openocd' covers all boards; -DOPENOCD=/OPENOCD=
still select another, msdk's when MAXIM_PATH is set.
Drops family_flash_openocd_wch and the OPENOCD_WCH pair, dedups
family_flash_openocd_adi, aligns ch583's work area, and points hil at the
flasher's own config instead of generating one per probe.
Verified: HIL green on all four WCH boards and max32666fthr.
|
|
(#3789)
test/hil: replace PCI reset with root-port VBUS cycle for D-state recovery
pci-reset was documented as an FLR, but no controller on either rig has FLR, so
it issued a PCIe secondary bus reset on a live, driver-bound xHCI -- halting the
card until the PVE host was power-cycled, and returning success so the caller
could not tell. It destroyed the ci controller twice.
Replace it with root-cycle, which cuts VBUS at the xHCI root port and touches
only the root hub, so it never takes the per-device lock the wedged ioctl holds.
uhubctl needs -S, or its sysfs backend disconnects the child before cutting
power and blocks on that same lock. Success is proven by the device's sysfs
directory inode changing: node existence proves nothing, and devnum is reused
once the per-bus map wraps.
usbtest.py's hang path invokes it, then confirms via /proc that nothing still
holds the device node. Skill scripts now run from the repo; the drifted
/usr/local/sbin copies are deleted.
|
|
|
|
|
|
Rename usb-target-debug -> target-debug, usb-debug -> usb-kernel-debug,
usb-recover -> usb-kernel-recover (script filenames unchanged), and make all
debug skills/agents decide tool applicability by which end of the link runs
Linux: TinyUSB may run the device or host stack, and its peer may be a Linux
PC, another TinyUSB board, or a Linux gadget (e.g. Raspberry Pi UDC).
- usbmon: exists only when a Linux PC is the link's host
- usb-kernel-debug: either Linux end; allowlist gains dwc3/libcomposite/udc_core
for the gadget side of a Linux peer
- usb-sniffer: the only full-visibility capture when TinyUSB is the host
- target-debug: covers dcd_* and hcd_*/tuh_ debugging; channel choice by topology
- update target-debugger/hil-operator agents, pre-pr, hil-validate.js, and the
USB_RECOVER path constant in test/hil/usbtest.py
- CLAUDE.md: fold the dcd/hcd datasheet cross-check rule into the read-doc line
|
|
frdm_k64f as a USB host with a CH9102 CDC (TX-RX loopback) and a Lexar MSC
drive behind a hub; flasher = onboard OpenSDA J-Link. host/cdc_msc_hid passes
(CDC mount+echo, MSC mount + disk-size check). device_info remains a known
device_info/usbh limitation (its synchronous descriptor dump starves a 2nd
device's enumeration) and is not ci_fs-specific.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01ExGPLP5eU43LR7o6yYLpNi
|
|
A re-run attempt merged into an empty base: another PR's HIL job ran
between attempt 1 and the retry and rewrote the shared hil_report.json,
so the run-stamp guard (correctly) refused the foreign base but the
full-fleet results were lost - the retry report contained only the
re-run cells.
Give each (run id, job) its own report dir instead:
- attempts of the same run share a dir, so the retry always finds its
own sidecar and .failed spec intact
- interleaved runs of other PRs/jobs write elsewhere and cannot clobber
- the run-stamp mechanism (.failed.run file) becomes redundant and is
removed
- stale per-run dirs are pruned after 2 weeks
|
|
Full-fleet profiling (HIL_PROFILE=1 instrumentation, included) showed each
uPD720201 controller's serialized usbtest battery chain dominates wall time,
and a board whose marginal device port bounces during concurrent batteries
can wedge or kill the controller ("xHCI host not responding to stop endpoint
command"). Every such death traced to mimxrt1015's port (its old "kills the
uPD720201" reputation) - it is removed from the config until recabled;
mimxrt1064's enum-retry stalls were a loose device cable (re-seated).
nrf54lm20dk moves to boards-skip until its failing J-Link probe is replugged.
With the hardware fixed both cards run width-4 batteries plus full flash
churn clean, so scheduling stays simple: two symmetric knobs, flashes and
batteries budgeted per controller.
- schedule_boards(): dispatch boards round-robin across host controllers from
a persisted hint cache (~/.cache/tinyusb-hil/ctrl_cache.json), learned and
merge-on-write refreshed each run (concurrent HIL jobs keep each other's
entries). Only the cached PCI address is consumed - dispatch order and
first-flash budgeting, never battery serialization (batteries resolve live
or fail closed to an all-slot permit).
- HIL_FLASH_PARALLEL (8) and HIL_USBTEST_PARALLEL (4) are budgeted per
controller via lock slots assigned on first sight.
- re-runs: a failed run writes <report dir>/<config>.failed with the exact
re-run spec (--accumulate -b <failed board> -bt <board>:<its failed
tests>) instead of the inverted --skip-board list of everything that
passed; --skip-board is gone, --flasher/--exclude-flasher scope a config
across CI jobs by flasher type (no board names hardcoded in workflows),
and -a/--accumulate merges a re-run into the existing report. The spec is
stamped with GITHUB_RUN_ID and cleared on fresh runs, so a retry can never
consume a spec left behind by a different run's dead or skipped attempt.
- CI: esp-idf firmware builds move out of hil-build into hil-build-esp, and
the esptool-flashed boards run in their own hil-tinyusb-esp job, so the
main hil-tinyusb run starts as soon as the fast toolchains finish instead
of waiting on the slow esp-idf build (an esp toolchain flake previously
skipped the whole rig run). Artifacts are namespaced per toolchain so the
esp job downloads only esp-idf binaries.
- HIL_PROFILE=1: timestamped log lines, per-flash durations, permit-wait
logging, uid->controller map dump for analysis.
- hil_report: per-variant test duration as a dedicated trailing column,
recorded only by full runs.
Validated on the ci rig (fixed seeds 20260716/777, full fleet at 8/4):
738s/780s walls with only known-flake failures and no controller deaths,
vs 1134-1211s serialized-battery baseline.
|
|
incompatible
The MosChip MCS9990 card is physically removed from the rig: delete its
cases-11/25 SKIP workaround (and the now-orphaned SKIP accounting) from
usbtest.py and refuse to run outright if a DUT ever sits behind one again.
usb_recover.sh gains `hub-cycle <busport>`: uhubctl VBUS cycle of the port
feeding the device, walking upstream (parent hub -> root port) until it
re-enumerates. Verified on the rig: leaf-level recovery (13-4.4 usbtest
device) and full walk to the root port on a dead branch. SKILL.md updated
for the action and the two-Renesas topology (root-port ppps is real; leaf
1a40:0201 hubs fake their "ganged" switching).
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WxUeX4Yn26KibfjvDg2pN9
|
|
and mimxrt1015
edpt_schedule_packets() advanced xfer->buffer past each armed EP0
chunk, so the OUT-complete handler invalidated the cache at the
ADVANCED pointer: one line past the received data. The CPU then read
stale cached bytes instead of the DMA'd packet, and the misplaced
invalidate discarded a dirty line of whatever variable follows the
buffer - random neighbor corruption on every control-OUT data stage.
Found by usbtest ctrl_out (cases 14/21) on espressif_p4_function_ev
with DMA enabled, the first DWC2 target combining buffer DMA with a
data cache: usbd control state wedged after the first control write
(every later request stalled), and one build layout panicked in the
usbd memcpy with a wild pointer.
Rework the EP0 chunk bookkeeping so xfer->buffer always points at the
un-consumed position: the arm no longer advances it; instead the EP0
re-arm paths advance past each completed (full) chunk, invalidating it
first on the OUT side. The final OUT completion invalidates exactly
the received bytes of its last chunk, taken from DOEPDMA ("incremented
on every AHB transaction", databook 7.1.83 - the same semantics the
SETUP path relies on) before dma_setup_prepare() re-targets it. EP0
chunking state (ep0_pending) is now also dropped on bus reset and on
a new SETUP, so a stale latched completion can no longer re-arm EP0
DMA from dead state. No behavior change for targets without dcache.
While root-causing, the FIFO layout was cross-checked against the
DWC2 databook/programming guide v4.20a: the existing GDFIFOCFG
programming (EPInfoBaseAddr = otg_dfifo_depth - 2*ep_count, one SPRAM
word per endpoint direction for buffer DMA) is conformant and needs
no change; the P4 HS instance's reset GDFIFOCFG (0x03800400) merely
reflects a scatter/gather-sized EP_LOC_CNT of 128 that buffer DMA
does not need.
With the fix in place, enable the usbtest battery on the espressif
fleet: tools/build.py allowlists device/usbtest (a plain IDF component
like board_test/video_capture) and both espressif boards' only-lists
gain device/usbtest. Also re-enable device/usbtest on mimxrt1015_evk:
its skip predated the dcd_ci_hs stale-ACTIVE-overlay fix (already on
this branch), which cured the battery that previously killed the
uPD720201 host controller twice (2026-07-11 ROM fw, 2026-07-13 case 27
on fw 2.0.2.6); rig-validated 30/30 three consecutive runs.
Validated on rig (all 30/30): espressif_p4_function_ev(-DMA) (was
22/30 under DMA), espressif_s3_devkitm(-DMA), stm32f723disco(-DMA),
mimxrt1015_evk; p4/s3 slave-mode unaffected (DMA-only code path);
compile-checked stm32h743nucleo +TUD DMA, stm32f407disco,
stm32l476disco (device ports currently on the dead hub).
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017TQZrFfU3K4Y198aLsUpBC
|
|
- dcd_edpt_stall flushes the primed buffer (ENDPTFLUSH), but the aborted
transfer's dQH overlay can be left ACTIVE with mid-transfer state; the
next prime after clear-halt then resumes the stale overlay instead of
loading the fresh qtd, so post-halt IN reads return mid-buffer data
(usbtest case 13 'buf[32] = 56 (not 0)', with case 18 failing
downstream of the same corruption in the full battery). qhd_start_xfer
now clears overlay.active alongside overlay.halted before linking the
new qtd.
- test/hil(hfp): drop lpcxpresso43s67's device/usbtest skip - the
historical first-case wedge no longer reproduces on this branch, and
with the overlay fix the board runs 30/30 on its Fresco xHCI host
(previously 28/30 with deterministic case 13/18 failures).
mimxrt1064_evk (imxrt dcache path) 30/30 regression-clean.
- docs(hil skill): document the external hifiphile rig - pool
test/hil/hfp.json, SSH-reachable from htpc/ci with no outbound SSH,
exercised by the CI hil-tinyusb (hfp.json) job; never run HIL against
it during development unless the user explicitly asks.
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017TQZrFfU3K4Y198aLsUpBC
|
|
# Conflicts:
# .claude/skills/hil/SKILL.md
# test/hil/hil_test.py
|
|
Pool/config:
- record real uids (ra8m1_ek), enable usbtest for espressif s3/p4, then
park ra6m5_ek and ra8m1_ek in boards-skip (ra6m5's usbtest/MSC traffic
can kill the uPD720201 host on its ROM firmware; ra8m1 USBHS bring-up
pending); max32666/nrf54lm20 stay enabled - their MosChip flakiness
never wedges
- re-enable device/usbtest on HS boards (mimxrt1064, ch32v307) now that
uPD720201 firmware 2.0.2.6 fixes the command-ring death; mimxrt1015
stays skipped - its HS battery killed the controller on both ROM and
2.0.2.6 firmware (board-specific); match the moved host-test bundles
(f723 <-> rt1064); skip never-passing tests on the new
nrf5340dk/nrf54lm20dk boards and the detached pico host bundle, each
documented with a comment
Host-controller quirk gating in usbtest.py (auto-skip, self-heals on a
healthy xHCI):
- MosChip MCS9990 EHCI: case 25 (int-OUT never scheduled, FRINDEX bug)
and case 11 (unlinked reads complete short/EREMOTEIO)
- Renesas uPD720201 xHCI: firmware-gated. The card must run firmware
>= 2.0.2.6 (RAM-uploaded - it reverts to ROM on every power cycle):
on older firmware the command ring dies under unlink stress (a
Configure Endpoint command stops completing; the hub worker deadlocks
holding the device lock; only a host power cycle recovers; three
boards reproduced it). usbtest.py reads the FW version register (PCI
config 0x6c) and refuses to run at all on older firmware - hil_test
surfaces that as a failed test with the reason. On current firmware
the full 30-case battery runs (validated FS+HS: metro_m4, f723,
f723-DMA all 30/30).
Scheduling (hil_test.py):
- Shuffle each (board, variant)'s test order with a seeded RNG
(HIL_SHUFFLE_SEED to replay) so usbtest batteries and flash churn spread
across the timeline instead of convoying on one controller.
- Per-controller usbtest + flash semaphores: HIL_USBTEST_PARALLEL
(default 4) concurrent usbtest batteries and HIL_FLASH_PARALLEL
(default 8) concurrent flashes per host controller. Profiled on
uPD720201 firmware 2.0.2.6 across 8/1..12/8: wall time falls
22.2/14.3/12.5/10.8 min at usbtest width 1/2/3/4 and plateaus there;
zero controller errors everywhere; first battery case failures
(leaf-hub bandwidth stretch) appear at 12/8, and flash width 12 only
amplifies flasher-hub contention flakes - so 8/4 is the optimum. A
separate battery-window flash throttle was profiled and dropped.
- Give every example a unique hardcoded USB PID (0x4001-0x4022, usbtest
keeps 0x4010) instead of the PID_MAP interface bitmap: different
examples now always re-enumerate back-to-back, even on boards whose
CPU reset does not drop D+ (WCH CH58x), so the EXAMPLE_PID table and
same-PID adjacency reordering in hil_test.py are gone; only the
variant-boundary same-example repeat needs a swap.
- Report matrix: stable columns with the metric-bearing tests pinned
first (usbtest, cdc_msc_throughput, msc_file_explorer[_freertos]),
the rest alphabetical.
Fail fast:
- enum wait budget 8 s on the first attempt, 4 s on retries; dfu waits
are deadline-based so dfu-util's own runtime counts against the
budget.
A device-absent failure now costs ~3-5x a passing test (20-30 s)
instead of 10-30x (47-150 s).
- CI runs hil_test with --retry 1 and no in-run second pass: a broken
fixture fails the job fast instead of holding the self-hosted runner
for hours and blocking other PRs' HIL jobs. hil_test still writes the
.skip sidecar, so a manual re-run attempt only retests what failed.
Review fixes (multi-agent adversarial review of this commit):
- tinyusb_win_usbser.inf: the PID rework moved five CDC examples onto
even PIDs the INF's odd-only DeviceList never matched (legacy-Windows
usbser binding) - appended 0x4006/4008/400a/4020/4022 to both lists.
- usbtest example: USBTEST_TIER is now overridable and the descriptors
and pumps are tier-conditional, so a board whose DCD cannot serve a
tier lowers it instead of skipping the whole example - RA2A1 (RUSB2
with no isochronous pipe) builds at tier 3 via its BOARD_ define; the
host battery follows the tier advertised in bcdDevice. Tier-4 output
verified byte-identical after the refactor.
- dynamic_configuration's second config derived USB_PID + 11 = 0x4018,
colliding with net_lwip_webserver - now USB_PID + 0x0100, outside the
per-example space. tools/check_example_pids.py (pre-commit hook)
enforces PID uniqueness incl. derived and literal idProduct values.
- usbtest.py firmware gate: matched by device ID (uPD720201/720202,
both use the 0x6c FW register), and an unreadable version (setpci
missing/denied) now refuses with its own message instead of
masquerading as "firmware 0x00000000"; noted the gate is necessary
but not sufficient (board-specific kills stay per-board skips).
- hil_test: deadline waits use time.monotonic(); multiprocessing
context pinned to fork (raw semaphores in Pool initargs); flash and
usbtest permits unified into one fail-closed, exception-safe
ctrl_permit (unknown controller takes every slot and logs a warning
instead of silently borrowing slot 0); an all-skipped battery
reports as skip, not "0/0" failure; slow-body polls (mtp, printer,
disk read) go through a shared deadline-based wait_until so their
bodies count against the enum budget; throughput's FS detection
compares serials case-insensitively like every other walk; a missing
MSC read-speed line now fails the host msc_file_explorer test
instead of passing with an empty metric.
Hardening:
- fail fast (15 s) when a driver-registry sysfs write blocks: a wedged
device otherwise turns every subsequent battery into an unkillable
D-state writer and silently hangs the whole run
- usb-recover skill: a VM reboot is not a reliable cure (MosChip hubs
latch up across the PCIe reset); full host power cycle is
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HeF2gZ1M7GWkz6Av4BpKPg
|
|
Multi-agent dev/test harness: worker agents, workflows, /pre-pr gate, per-board HIL locks
|
