summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorhathach <[email protected]>2026-08-21 11:07:27 +0700
committerhathach <[email protected]>2026-08-21 11:07:27 +0700
commit04d0f71984117b8c72349f4584bd9e26a37b129c (patch)
tree1d87e53f8b3b6f94da3fcb03a5da5299c7cd815f /docs
parent696c7807f543a6c55656d81a8f6d8969584e9614 (diff)
ci: scope the build matrix and the HIL run to what a PR affects
Every PR built all 74 legs (2494 example builds on GHA cmake alone) and flashed all 30 rig boards, whatever it touched. One classifier now walks the PR diff twice and answers three questions: which families to build, which examples per family, and which boards run which tests. Fail-open throughout - anything no rule classifies, any exception, any unusable output falls back to the full matrix, and a master push always builds everything. test/hil/helper/hil_select.py moves to tools/ci_select.py: it is no longer HIL-only, and tools/ is where the build side can import it. test_hil_select.py follows it as test_ci_select.py. Rules (docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md holds the full table): a port selects the families whose family.cmake references it, and its role - a dcd change skips host examples and vice versa; a class selects only the examples whose tusb_config.h enables its CFG_TU[DH]_ macro, following cross-class includes; an example selects itself; hw/bsp selects its family or board; hw/mcu and lib select whoever references them. CMake is the reference for all of it - make follows whatever cmake decides, family.mk is never scanned. Empty means empty (maintainer ruling): a rule that classifies a path to nothing selects nothing. Ports no family references, classes no config enables, libs no example builds and hw/mcu paths that resolve nowhere are all real - nothing compiles them, so nothing can validate them, and the master-push build is the net. Structural tests pin each such case with an explicit allowlist, so the day one stops being empty it fails pre-commit instead of silently narrowing CI. Per-example builds: build.py grows a repeatable -e, resolved against the targets CMake actually registered and batched into one `cmake --build --target a b c`. build_utils mirrors CMake's family_filter (the whole FAMILY_MCUS list, ${...} and string(TOUPPER ...) resolved) for the cmake side, while the make side keeps master's algorithm verbatim - the two build systems answer differently and a shared answer breaks lpc54's make link. hil-build gains this even on a full selection: 1702 example builds become 515. Transport: the selection travels as a file, never an argv or env var - a mass-sweep diff selects 261 KB against a 128 KiB exec limit, and E2BIG would fail the step before its own fallback could run. CircleCI carries the example map inside the generated config (pipeline parameters cap at 512 chars), swapped into the parameter defaults by sentinel match, and drops the scoping wholesale if that rewrite fails. Every PR-derived value written to $GITHUB_ENV/$GITHUB_OUTPUT is character-screened. Code metrics follow the scoping: metrics.py emits per-example totals, and metrics_pair_compare compares the (board, example) pairs present on both sides instead of a scoped run against a full-matrix average. The selector's own suite gates it in both providers: a selector that exits 0 with valid-but-wrong JSON is the one failure fail-open cannot catch, so a red suite means the full matrix.
Diffstat (limited to 'docs')
-rw-r--r--docs/reference/hardware-in-the-loop.md5
-rw-r--r--docs/superpowers/followup/pr3803-flasher-recover.md24
2 files changed, 15 insertions, 14 deletions
diff --git a/docs/reference/hardware-in-the-loop.md b/docs/reference/hardware-in-the-loop.md
index 48c362f4f..cf7e3fe69 100644
--- a/docs/reference/hardware-in-the-loop.md
+++ b/docs/reference/hardware-in-the-loop.md
@@ -281,8 +281,9 @@ Both files are the source of truth — this table is generated from them.
`test/hil/hil_test.py`, which flashes each board and runs its tests. Espressif boards
run in `hil-tinyusb-esp`, gated on the slower ESP-IDF build, and `hil-hfp-iar` builds
with IAR inside the job.
-3. On pull requests, `test/hil/helper/hil_select.py` narrows the run to the boards a diff
- can affect, falling open to the full matrix when it cannot tell.
+3. On pull requests, `tools/ci_select.py` narrows the run to the boards a diff can
+ affect — and each board's build to the examples its tests need — falling open to the
+ full matrix when it cannot tell. The same pass scopes the build matrix.
4. Each board is arbitrated by a kernel flock in `/tmp/tinyusb-hil-locks/`, so interactive
work and CI can share the rig without colliding.
5. Each rig job uploads its report as an artifact; `pr_comment.yml` downloads them and
diff --git a/docs/superpowers/followup/pr3803-flasher-recover.md b/docs/superpowers/followup/pr3803-flasher-recover.md
index e9fff7480..1f71c990f 100644
--- a/docs/superpowers/followup/pr3803-flasher-recover.md
+++ b/docs/superpowers/followup/pr3803-flasher-recover.md
@@ -19,18 +19,18 @@ libjaylink, J-Link probes.
- Roster JSON: `test/hil/tinyusb.json`. `flasher_recover` is OPTIONAL; absent means today's
behaviour (`recover_flasher` returns the primary).
- Never change the shape of `board['flasher']` — it is read as a dict in `hil_flash`,
- `hil_test`, `usbtest`, `hil_pool_check`, `hil_select` and the roster lint, and is shipped
+ `hil_test`, `usbtest`, `hil_pool_check`, `ci_select` and the roster lint, and is shipped
as JSON to a subprocess.
- Flasher dispatch is by name: `getattr(hil_flash, f'flash_{name}')` / `reset_{name}`.
- `RECOVER_FLASH_TIMEOUT = 90`, `RECOVER_RESET_TIMEOUT = 30` (`usbtest.py`). Any board whose
flash cannot finish inside 90 s is not a candidate.
-- Tests run offline: `cd test/hil && python3 test/test_hil_select.py`.
+- Tests run offline: `cd test/hil && python3 test/test_ci_select.py`.
## What is already established
**Landed on PR #3803 and inert without roster entries:** `hil_flash.recover_flasher()`,
`convoy_safe()` accepting openocd-over-jlink, `hil_test` substituting the recovery flasher
-into `--recover-board`, and `test_hil_select.FlasherRecoverEntry` (4 tests).
+into `--recover-board`, and `test_ci_select.FlasherRecoverEntry` (4 tests).
**Verified in source:**
- openocd's jlink driver ignores `adapter usb vid_pid` — `jlink.c` never reads
@@ -77,7 +77,7 @@ is a different scope from containing a wedge; and it needs bench time on seven b
- `test/hil/hil_flash.py` — add `flash_openocd_seq` / `reset_openocd_seq`; extend
`convoy_safe` to accept the new name. This is the only file that learns the command form.
- `test/hil/tinyusb.json` — seven `flasher_recover` entries.
-- `test/hil/test/test_hil_select.py` — extend `FlasherRecoverEntry`; add a roster lint.
+- `test/hil/test/test_ci_select.py` — extend `FlasherRecoverEntry`; add a roster lint.
---
@@ -85,7 +85,7 @@ is a different scope from containing a wedge; and it needs bench time on seven b
**Files:**
- Modify: `test/hil/hil_flash.py` (beside `flash_openocd`, ~line 100)
-- Test: `test/hil/test/test_hil_select.py`
+- Test: `test/hil/test/test_ci_select.py`
**Interfaces:**
- Consumes: `_openocd_cmd_base(flasher)`, `hil_util.run_cmd`.
@@ -120,7 +120,7 @@ is a different scope from containing a wedge; and it needs bench time on seven b
- [ ] **Step 2: Run test to verify it fails**
-Run: `cd test/hil && python3 test/test_hil_select.py FlasherRecoverEntry -v`
+Run: `cd test/hil && python3 test/test_ci_select.py FlasherRecoverEntry -v`
Expected: FAIL — `module 'hil_flash' has no attribute 'flash_openocd_seq'`
- [ ] **Step 3: Write minimal implementation**
@@ -155,13 +155,13 @@ In `convoy_safe`, replace `if name != 'openocd':` with:
- [ ] **Step 4: Run test to verify it passes**
-Run: `cd test/hil && python3 test/test_hil_select.py FlasherRecoverEntry -v`
+Run: `cd test/hil && python3 test/test_ci_select.py FlasherRecoverEntry -v`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
-git add test/hil/hil_flash.py test/hil/test/test_hil_select.py
+git add test/hil/hil_flash.py test/hil/test/test_ci_select.py
git commit -m "hil: add openocd_seq flasher for convoy-safe recovery delivery"
```
@@ -171,7 +171,7 @@ git commit -m "hil: add openocd_seq flasher for convoy-safe recovery delivery"
**Files:**
- Modify: `test/hil/tinyusb.json`
-- Test: `test/hil/test/test_hil_select.py`
+- Test: `test/hil/test/test_ci_select.py`
**Interfaces:**
- Consumes: `flash_openocd_seq` / `reset_openocd_seq` from Task 1.
@@ -196,7 +196,7 @@ git commit -m "hil: add openocd_seq flasher for convoy-safe recovery delivery"
- [ ] **Step 2: Run test to verify it fails**
-Run: `cd test/hil && python3 test/test_hil_select.py FlasherRecoverEntry -v`
+Run: `cd test/hil && python3 test/test_ci_select.py FlasherRecoverEntry -v`
Expected: FAIL — `0 >= 7`
- [ ] **Step 3: Add the entries**
@@ -224,13 +224,13 @@ Add to each board below, using the SAME `uid` as its primary jlink entry:
- [ ] **Step 4: Run test to verify it passes**
-Run: `cd test/hil && python3 test/test_hil_select.py -v`
+Run: `cd test/hil && python3 test/test_ci_select.py -v`
Expected: PASS, and no other selector test regresses.
- [ ] **Step 5: Commit**
```bash
-git add test/hil/tinyusb.json test/hil/test/test_hil_select.py
+git add test/hil/tinyusb.json test/hil/test/test_ci_select.py
git commit -m "hil: give seven J-Link boards a convoy-safe recovery flasher"
```