ci: scope the build matrix and the HIL run to what a PR affects

Every PR built all 74 legs (2494 example builds on GHA cmake alone) and flashed
all 30 rig boards, whatever it touched. One classifier now walks the PR diff twice
and answers three questions: which families to build, which examples per family,
and which boards run which tests. Fail-open throughout - anything no rule
classifies, any exception, any unusable output falls back to the full matrix, and
a master push always builds everything.

test/hil/helper/hil_select.py moves to tools/ci_select.py: it is no longer HIL-only,
and tools/ is where the build side can import it. test_hil_select.py follows it as
test_ci_select.py.

Rules (docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md holds the
full table): a port selects the families whose family.cmake references it, and its
role - a dcd change skips host examples and vice versa; a class selects only the
examples whose tusb_config.h enables its CFG_TU[DH]_ macro, following cross-class
includes; an example selects itself; hw/bsp selects its family or board; hw/mcu and
lib select whoever references them. CMake is the reference for all of it - make
follows whatever cmake decides, family.mk is never scanned.

Empty means empty (maintainer ruling): a rule that classifies a path to nothing
selects nothing. Ports no family references, classes no config enables, libs no
example builds and hw/mcu paths that resolve nowhere are all real - nothing
compiles them, so nothing can validate them, and the master-push build is the net.
Structural tests pin each such case with an explicit allowlist, so the day one
stops being empty it fails pre-commit instead of silently narrowing CI.

Per-example builds: build.py grows a repeatable -e, resolved against the targets
CMake actually registered and batched into one `cmake --build --target a b c`.
build_utils mirrors CMake's family_filter (the whole FAMILY_MCUS list, ${...} and
string(TOUPPER ...) resolved) for the cmake side, while the make side keeps
master's algorithm verbatim - the two build systems answer differently and a shared
answer breaks lpc54's make link. hil-build gains this even on a full selection:
1702 example builds become 515.

Transport: the selection travels as a file, never an argv or env var - a mass-sweep
diff selects 261 KB against a 128 KiB exec limit, and E2BIG would fail the step
before its own fallback could run. CircleCI carries the example map inside the
generated config (pipeline parameters cap at 512 chars), swapped into the parameter
defaults by sentinel match, and drops the scoping wholesale if that rewrite fails.
Every PR-derived value written to $GITHUB_ENV/$GITHUB_OUTPUT is character-screened.

Code metrics follow the scoping: metrics.py emits per-example totals, and
metrics_pair_compare compares the (board, example) pairs present on both sides
instead of a scoped run against a full-matrix average.

The selector's own suite gates it in both providers: a selector that exits 0 with
valid-but-wrong JSON is the one failure fail-open cannot catch, so a red suite
means the full matrix.

1 files changed, 72 insertions, 6 deletions

diff --git a/.github/scripts/ci_set_matrix.py b/.github/scripts/ci_set_matrix.py
index 50ada5964..ee3609bed 100755
--- a/.github/scripts/ci_set_matrix.py
+++ b/.github/scripts/ci_set_matrix.py
@@ -1,5 +1,9 @@
 #!/usr/bin/env python3
+import argparse
 import json
+import os
+import subprocess
+import sys
 
 # toolchain, url
 toolchain_list = [
@@ -97,15 +101,77 @@ family_list = {
 }
 
 
-def set_matrix_json():
+def set_matrix_json(select=None):
+    sel_fams = None
+    if select:
+        # every shape check is explicit: this runs AFTER main()'s fail-open handler, so
+        # an AttributeError on e.g. {"build": ["stm32f4"]} would red the step instead
+        # of falling back to the full matrix - the outcome that handler exists to prevent
+        b = select.get('build') if isinstance(select, dict) else None
+        if not isinstance(b, dict):
+            b = {}
+        if b.get('full') is False:
+            fams = b.get('families')
+            if not (isinstance(fams, list) and all(isinstance(f, str) for f in fams)):
+                # key ABSENT (or not a list of names) is an unusable selection, not
+                # "nothing selected": scoping every toolchain to [] would build zero
+                # families and report a vacuous green. An explicit families: [] stays a
+                # legitimate nothing-selected.
+                print('ci_set_matrix: UNSCOPED - build.full is false but the families '
+                      'list is unusable, emitting the full matrix', file=sys.stderr)
+            else:
+                sel_fams = set(fams)
     matrix = {}
     for toolchain in toolchain_list:
-        filtered_families = [family for family, supported_toolchain in family_list.items() if
-                             toolchain in supported_toolchain]
-        matrix[toolchain] = filtered_families
-
+        fams = [family for family, tc in family_list.items() if toolchain in tc]
+        if sel_fams is not None:
+            fams = [f for f in fams if f in sel_fams]
+        matrix[toolchain] = fams
+    if sel_fams is not None:
+        # a family this file does not list builds on no toolchain, so the selection maps
+        # to an empty matrix and every leg skips - which looks exactly like a working
+        # scoped run. Say so: hw/bsp holds several families CI has never built
+        # (efm32, py32f0, ...) and espressif, whose boards are built by hil-build-esp
+        unbuilt = sorted(f for f in sel_fams if f not in family_list)
+        if unbuilt:
+            print(f'ci_set_matrix: selected families built by no toolchain here: '
+                  f'{", ".join(unbuilt)}', file=sys.stderr)
     print(json.dumps(matrix))
 
 
+def main():
+    parser = argparse.ArgumentParser()
+    group = parser.add_mutually_exclusive_group()
+    group.add_argument('--select', help='tools/ci_select.py JSON; scopes families when build.full is false')
+    # a whole selection as one argv/env value can exceed the exec limits on a big
+    # diff, which fails the calling step BEFORE it can fall open; callers that
+    # already have the selection on disk pass the path instead
+    group.add_argument('--select-file', help='file holding the same JSON as --select')
+    group.add_argument('--base', help='git ref: run tools/ci_select.py --base REF and scope from it')
+    args = parser.parse_args()
+
+    select = None
+    try:
+        if args.select:
+            select = json.loads(args.select)
+        elif args.select_file:
+            with open(args.select_file) as f:
+                select = json.load(f)
+        elif args.base:
+            root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+            r = subprocess.run([sys.executable, os.path.join(root, 'tools', 'ci_select.py'),
+                                '--base', args.base],
+                               capture_output=True, text=True, cwd=root, check=True)
+            select = json.loads(r.stdout)
+    except Exception as e:  # fail-open: an unusable selection must never turn into a red job
+        # UNSCOPED is the marker build.yml greps for: it must then drop the build extras
+        # (example map, family regex) too, or a full build gets labelled and filtered as
+        # a scoped one. Keep the token on every fall-open path.
+        print(f'ci_set_matrix: UNSCOPED - selection unusable ({e}), emitting the full '
+              f'matrix', file=sys.stderr)
+        select = None
+    set_matrix_json(select)
+
+
 if __name__ == '__main__':
-    set_matrix_json()
+    main()