Unnamed repository; edit this file 'description' to name the repository. http://cgit.235523.xyz/tinyusb.git/atom/test/hil/helper/hil_health.py?h=codex%2Fclass-reference-docs 2026-08-28T07:18:41Z 2026-08-28T07:18:41Z hathach thach@tinyusb.org 2026-08-27T08:26:43Z urn:sha1:20bb94fcf9ad7fca7fb685e53307f4d03b1340fd The modules were 21% comment, much of it review-cycle argument rather than guidance -- _kill_kids stated 'descendant by construction, no argv check needed' twice, eight lines apart. Deleting such comments outright makes maintenance worse: the next reader simplifies the thing the comment was defending. So the history moves to the 2026-07-30 fleet-wedge design doc, which gains a trim addendum recording what was removed, what was deliberately kept, and the rule that decided each -- the CI ceiling bounds how long a run burns, and does nothing about state that outlives it. One comment was not merely long but WRONG: the report wipe carried 'The unlink is DEFERRED to inside the pool try/except below', which is the opposite of what the code does -- it sits before Manager() with its own comment explaining why. That is the failure mode this pass is about, so it is deleted rather than reworded. Kept everywhere: citations that refute a plausible wrong reading. That usb_lock_device_interruptible is why the readers are killable, that usblp_mutex is driver-global, that rawmidi honours O_NONBLOCK where usblp does not. Two follow-ups are retired with them: pr3803-hil-blindness-reporting.md (there is no blindness to report any more) and pr3803-usbtest-recovery-reserve.md (the reserve is derived now). Kept: pr3803-flasher-recover.md, which PR #3832 implements, plus pr3803-pci-rebind-stranding.md and pr3803-hil-iar-rerun-spec.md, both independent of this work. 2026-08-26T06:41:05Z Ha Thach thach@tinyusb.org 2026-08-26T06:41:05Z urn:sha1:b15c720ab0a36ee16c1cf7984c7c0f4ccf3a7015 Three readability changes with no behaviour change on the healthy path — every pre-existing test passes untouched. main() was 368 lines with try/finally three deep, its two abort paths near-identical 40-line blocks; _abort_report holds that shape once, and the controller-hint cache and pool construction move to their own helpers. 368 -> 279, test_board 180 -> 151, test_device_usbtest 164 -> 125. test_hil_bounded.py cost 78s on every commit under test/hil/, mostly one 3s post-flash settle paid by ten tests against a fake rig. Now 37s. Fixes two pre-existing defects the extraction exposed: _write_failed_spec was unguarded inside the abort path, so an OSError there replaced the caller's RuntimeError and no report was written at all; _save_controller_hints overlaid a startup snapshot onto the re-read cache, clobbering a concurrent job's newer values. Also five comments that stated the opposite of the code, and both table renderers measuring width with len() against two-column status marks. 2026-08-25T08:04:42Z Ha Thach thach@tinyusb.org 2026-08-25T08:04:42Z urn:sha1:375e20090a0a60fd2d47ba6553d189e05482dfea hil_report.json and hil_report.md were written independently. Four writers produced the markdown and three wrote no JSON at all -- and those three are the paths where a run died, so a JSON consumer saw nothing exactly when it mattered: the per-board verdicts an agent hands back reported the whole fleet as "no report row" while a human read the real story from the markdown. Every writer now goes through render_report(), so a table can never contain something the JSON does not. The document gains `scope` (a three-board PR run and a full run that lost 24 boards were indistinguishable) and `caveat` (how the run ended). `banner` carries rig health across an --accumulate retry; `caveat` records how a run ended and must not -- conflating them made a clean retry publish an abandonment that never happened. helper/hil_report.py owns the document end to end, dissolving the import cycle that forced write_timeout_report to compose its own markdown and removing a duplicate cell classifier kept in sync by hand. hil_summary.py is deleted; its CLI moves there. hil_ci.sh uploads the sidecar so a remote --accumulate has a merge bas 2026-08-18T05:19:09Z hathach thach@tinyusb.org 2026-08-13T18:08:40Z urn:sha1:3963a1b70a572132aced1c1a0033e1c8249a0c7e A wedged USB device used to take the whole HIL run with it. Every worker that touched the poisoned node blocked uninterruptibly, the pool could not be joined, map_async discarded every board's result, and the job ran to the GitHub ceiling with no report at all -- while the self-hosted runner's single job slot stayed occupied and every queued job waited behind it. Bound the calls a worker makes itself. read_sysfs, bounded_open and run_cmd all answer within a wall clock; read_sysfs distinguishes "absent" from "unknown", because a blocked read is not evidence of absence, and caps stranded readers at four (each costs a thread and an fd for the life of the process) after which the worker declares itself blind. mtype, the gio unmount, the libmtp session and the arecord/iperf reaps go through those bounds; the MTP session runs in a disposable subprocess, since libmtp's ctypes calls block unkillably in D state. Bound the run. A pool guard (HIL_POOL_TIMEOUT, 60 min) fires before any job ceiling and still writes a report. When the pool will not shut down, the sweep kills what the workers spawned -- descendants, not just direct children, since flashers run in their own session -- confirms each kill actually landed, and exits early so the runner is freed. Whatever survived is named in the report. Deliberately shallow past that point. We do not re-scan process groups, prove pid ownership, or escalate through sudo: a root-owned survivor is reported, not force-killed, because signalling a pid we cannot prove is ours is the worse failure, and the job ceiling backstops whatever this misses. A D-state holder was never killable anyway. Recover instead of reporting a wedge. A HUNG usbtest case reflashes its own DUT through its roster flasher, but only where the flasher can reach its probe past a poisoned node -- openocd pinned to a validated vid_pid, or esptool. Where it cannot, the run says so rather than reserving budget for a path that cannot fire. Raise the CI ceilings above the pool guard so the guard fires first and still writes its report, and pin --retry 1 on every HIL leg: the guard is a flat constant and does not scale with max_retry, so argparse's default of 3 would triple the serialized usbtest tail against an unchanged guard. Split the module: execution in hil_test/hil_flash/usbtest, infrastructure in helper/ (locking, health, selection, shared bounded IO), and the two matrix generators into .github/scripts/ -- ci_set_matrix.py sat in workflows/, where GitHub treats every file as a workflow definition. 193 tests cover the bounded paths, the kill ladder, the guard and the selector against synthetic /proc trees and PATH-injected fakes; a real wedge cannot be manufactured on demand.