summaryrefslogtreecommitdiff
path: root/docs/reference
diff options
context:
space:
mode:
authorhathach <[email protected]>2026-08-20 16:43:49 +0700
committerhathach <[email protected]>2026-08-20 16:43:49 +0700
commit6905639b07c69fec68e9ebc77f7d27ac2775ee41 (patch)
tree2d46b0b6ae45b27df4874583c5830997df5829bd /docs/reference
parent7800876bf151a239046521232dc4603b156061be (diff)
hil: run every board in one hil_test.py and hand results across as JSON
hil-validate ran one hil-operator per board. That parallelizes at the wrong layer: hil_test.py already schedules boards across host controllers and budgets concurrent flashes and usbtest batteries per controller (FLASH_PARALLEL/USBTEST_PARALLEL), and those permits live in one process - N parallel runs multiply the budget onto the same uPD720201 cards for no wall-clock gain over one run that already parallelizes. The workflow now spawns ONE operator with every board as repeated -b. The operator no longer retypes the report table. Four consecutive max-effort review rounds found ~15 defects in this file and every one was in reconstructing board identity from transcribed prose: report rows are named per VARIANT (nanoch32v203 only ever produces -fsdev/-usbfs rows), a variant need not start with its board's name, lock contention is a `board-locked` cell rather than a phrase, and each fix introduced the next round's bug - including a fake-green test that asserted an invariant with the one input shape that could not break it. The new helper test/hil/helper/hil_summary.py does the join where the roster lives and emits one machine verdict per board ({board, ran, pass, locked, detail}); the operator returns that JSON verbatim plus `wedged`, the only field it authors, and the workflow reads fields, never parses a string. Its cell classifier mirrors hil_test.py's own tally exactly: failures are always marked ('fail' or a ❌ prefix, TestFail's contract), everything unmarked is a pass - a passing test may return a plain metric cell like '13443 KB/s', and the mirrored rule is what keeps a green table from becoming a red verdict. hil_ci.sh kept only the LAST -b, so multi-board remote runs staged one board's binaries and every other board died on the rig after its lock and flash slot were spent. It now parses every -b spelling argparse accepts (with the -bt arms ordered first, longest-match, so the <config>.failed retry form is never read as a board named "t..."), pre-flights roster membership and build dirs for ALL boards before anything is wiped or staged, warns per declared variant with no build dir (which hil_test.py would silently green-skip), forwards HIL_* knobs as export lines in one %q word the remote evals ('; '-joined so it round-trips under dash - an authorized HIL_NO_BOARD_LOCK force must not silently no-op), keeps HIL_REPORT_DIR local because the copy-backs look in REMOTE_DIR, and copies hil_report.json and the .failed re-run spec back beside the markdown, deleting stale local copies first so a green run cannot leave last run's spec looking current. Retries preserve the fleet: the documented path is the <config>.failed spec, which already begins with --accumulate; a fresh scoped re-run would unlink the report and collapse the whole-fleet table to the retried boards alone. The risky logic is executable, not argued about: .claude/workflows/test-hil-validate.mjs pins the lookup/verdict helpers and runs in pre-commit (hil-validate-logic); nine staging tests drive hil_ci.sh through an ssh stub that models the real thing (argv joined into one string the remote re-splits, heredoc on stdin - the naive echo-stub passed while the feature was broken); and deliberate mutations of the verdict logic are all caught. Validated on the rig: a 2-board run (usbtest 30/30 on both; the pre-fix classifier, replayed against that run's real report, fails the fully-green stm32f723disco on its two passing '13443 KB/s' cells), the .failed --accumulate retry (merged report kept every earlier row), and a 10-run soak over random subsets of a 22-board pool - 43 board-slots, every failure signature matched pre-existing CI state or known flake, zero tooling failures, no locks left behind.
Diffstat (limited to 'docs/reference')
0 files changed, 0 insertions, 0 deletions