summaryrefslogtreecommitdiff
path: root/docs/superpowers/specs
diff options
context:
space:
mode:
authorhathach <[email protected]>2026-07-09 23:34:29 +0700
committerhathach <[email protected]>2026-07-09 23:34:29 +0700
commite3dd9245ef08c457d7c6e5837e3f8d41bad6fd8a (patch)
tree4793e68120f4836e50cb3f51d2dd8d25a6095ab3 /docs/superpowers/specs
parentfa750d6bf045f1df4314d283dfe0508d0d066559 (diff)
feat: Claude Code multi-agent dev/test harness for TinyUSB
Add worker agents (builder, port-dev, driver-reviewer, hil-operator, pr-monitor), deterministic workflows (validate, fanout-dev, driver-review, hil-validate, full-check, pr-babysit) and a /pre-pr gate skill, so sessions can fan build/test/review/PR-triage work out to tiered subagents. pr-babysit drives a PR to green: triage CI + bot reviews, fix validated findings, verify, push, and reply-to + resolve each inline review thread (fixed or refuted). Replace the stop-the-runner HIL discipline with per-board flock locks: test/hil/board_lock.py plus a fail-open guard in hil_test.py let CI and dev sessions share the rig per board (locked boards fail fast and re-run; HIL_NO_BOARD_LOCK=1 is a user-authorized bypass). The actions-runner is never stopped. Design spec, implementation plan, and real-rig smoke evidence under docs/superpowers/. Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Rn1AN5DsTdFhRwhugfgKZi
Diffstat (limited to 'docs/superpowers/specs')
-rw-r--r--docs/superpowers/specs/2026-07-09-claude-agents-workflows-design.md163
1 files changed, 163 insertions, 0 deletions
diff --git a/docs/superpowers/specs/2026-07-09-claude-agents-workflows-design.md b/docs/superpowers/specs/2026-07-09-claude-agents-workflows-design.md
new file mode 100644
index 000000000..4638239a6
--- /dev/null
+++ b/docs/superpowers/specs/2026-07-09-claude-agents-workflows-design.md
@@ -0,0 +1,163 @@
+# Multi-Agent Dev/Test Setup for TinyUSB — Design
+
+Date: 2026-07-09
+Branch: worktree-claude-agents-workflows
+
+## Goal
+
+Give Claude Code sessions in this repo a reusable, efficient multi-agent harness
+for developing and testing TinyUSB: custom worker agents that already know the
+repo's build/test/rig discipline, and small deterministic workflows that fan
+them out. The orchestrator (main session) authors arguments and reads verdicts;
+workers do the volume.
+
+## Context
+
+- Existing process skills: `hil`, `code-size`, `pvs`, `build-doc`, `usbmon`,
+ `usb-debug`, `usb-recover`, `make-release` (`.claude/skills/`).
+- One prototype workflow exists in the master working tree (untracked):
+ `.claude/workflows/port-audit.js`. This design supersedes it.
+- No custom agent definitions exist yet (`.claude/agents/` absent).
+- Test infra: `test/unit-test` (ceedling), `test/hil` (`hil_test.py`,
+ `tinyusb.json`), `test/fuzz`; size metrics via
+ `tools/metrics_compare_base.py`.
+
+## Architecture
+
+Layered: **agents** (who does the work, with baked-in domain knowledge) ×
+**workflows** (deterministic fan-out/join) × **one skill** (human entry point).
+
+### Worker agents — `.claude/agents/*.md`
+
+Tiered models (owner revision 2026-07-09; originally all-opus): `port-dev`
+and `driver-reviewer` on **opus** at **xhigh**; `hil-operator` and
+`pr-monitor` on **sonnet**; `builder` on **haiku** (mechanical, log-heavy).
+
+| Agent | Effort | Role |
+|---|---|---|
+| `builder` | low | Build one board's example set with the canonical commands: `cmake -B cmake-build-<board> -DBOARD=<board> -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel` from `examples/` (exact dir name — HIL expects it), `python3 tools/get_deps.py` on missing deps, `. $HOME/code/esp-idf/export.sh` for Espressif boards, tolerate non-critical objcopy failures. Returns structured `{board, pass, failures: [{example, firstError}]}`. |
+| `port-dev` | xhigh | Implement one well-scoped change in one port / file set. Follows repo rules: C99, 2-space indent, snake_case, `TU_ASSERT`, no dynamic allocation, ISR work deferred to task context. Runs `clang-format` (repo `.clang-format`) on touched files before finishing. Cross-checks the MCU datasheet in `$HOME/Documents/calibre-library` when changing dcd/hcd register logic. Verifies with a targeted build of one board using the port. Returns `{item, diffstat, buildOk, notes}`. |
+| `driver-reviewer` | xhigh | Review one dcd/hcd directory against dimensions: correctness, ISR safety, register use vs. datasheet AND MCU errata (calibre library; missing erratum workarounds are findings), style. Returns structured findings `{file, line, snippet, why, severity, confidence}` — coverage-first (report everything; filtering happens downstream). |
+| `hil-operator` | default | All rig interaction — the actions-runner service is NEVER stopped; per-board flock locks arbitrate with concurrent CI. `hil_test.py` runs rely on its per-board self-locking; manual hardware work (JLink/GDB, usbtest, serial) is wrapped in `test/hil/board_lock.py hold/release`; rig-wide ops (uhubctl, pci-rebind) require `hold --all`; on wedge `usb_recover.sh` + dmesg. Used strictly serially — never two instances concurrently. |
+| `pr-monitor` | default | Triage one GitHub PR via `gh`: check CI status (`gh pr checks`), read failing run logs and classify each failure infra/flake vs real; re-run infra failures (`gh run rerun --failed`); harvest automated review comments (Codex/Copilot/Claude bots — knows their signals: Codex posts a "Didn't find any major issues" issue comment when clean; Copilot drops out of `requested_reviewers` when done; bot logins differ across APIs); adversarially validate each finding against the actual code. Returns structured triage `{ci: {status, infraRerun[], realFailures[]}, findings: [{source, file, line, claim, verdict, fixHint}]}`. Read/triage/re-run/reply only — never edits code. |
+
+### Workflows — `.claude/workflows/*.js`
+
+| Workflow | Args | Shape |
+|---|---|---|
+| `validate.js` | `{boards[], examples?, base?, skip?: ('unit'\|'size'\|'pvs')[]}` | One parallel stage: unit tests (ceedling) + one `builder` per board + code-size compare (`tools/metrics_compare_base.py` vs `base`, default master) + PVS analyze. Join → plain-JS verdict `{pass, failures[]}`. Barrier is correct here: the verdict needs all results. |
+| `fanout-dev.js` | `{task, items[], board?, review?, worktree?}` | `pipeline(items)`: `port-dev` per item → `builder` verify → optional `driver-reviewer` pass. Workers share the tree by default (ports are disjoint directories); `worktree: true` switches on per-agent worktree isolation for collision-prone tasks. Returns per-item results. |
+| `driver-review.js` | `{dirs[], dimensions?, question?}` | Supersedes `port-audit.js`. Scan stage per (dir × dimension) → adversarial verify per finding (verifier prompted to refute) → confirmed findings only. |
+| `hil-validate.js` | `{boards[], force?}` | Strictly serial `for` loop of `hil-operator` calls; each board is protected by `hil_test.py`'s own per-board flock, so the actions-runner keeps running throughout. Boards found locked (a concurrent CI job mid-test) are retried once at the end of the loop; boards still locked are returned in `locked[]` for a user force/wait/accept decision. `force: true` (user-authorized only) bypasses locks via `HIL_NO_BOARD_LOCK=1`. Returns per-board `{board, pass, detail}` plus `wedged[]` and `locked[]`. |
+| `full-check.js` | `{boards[], ...}` | Thin composer: `workflow('validate', ...)` → only if green → `workflow('hil-validate', ...)`. Single nesting level (children do not nest further). |
+| `pr-babysit.js` | `{pr, maxCycles?, autoPush?}` | Cycle until CI green + review threads resolved, or `maxCycles` (default 3): `pr-monitor` triage (blocks on `gh pr checks --watch` while CI runs) → valid findings + real CI failures grouped by file/port → `port-dev` fix per group (pipeline) → `driver-reviewer` verifies each fix addresses its finding → one commit + push per cycle. Every actioned inline comment is both **replied to and marked resolved** (GraphQL `resolveReviewThread`): refuted findings get the refutation, fixed findings get a "fixed in <sha>" note. `autoPush` defaults true; **invoking this workflow is the explicit push authorization** for follow-up commits on that PR branch (scoped exception to the hold-pushes-until-told rule). |
+
+### Board lock protocol — `test/hil/` (repo code)
+
+CI and dev sessions share the rig concurrently; the actions-runner service is
+never stopped. Arbitration is per-board kernel flocks in
+`/tmp/tinyusb-hil-locks/<board>.lock` — auto-released when the holder process
+dies (stale locks impossible; `/tmp` clears on reboot):
+
+- **`test/hil/board_lock.py`** (new tool): `hold <boards|--all> --reason TEXT`
+ spawns a background holder process flocking each board file (JSON
+ `{pid, reason, since}` written inside for debuggability); `release
+ <boards|--all>` kills holders; `status` lists them. `--all` is required
+ before rig-wide operations (uhubctl power cycling, pci-rebind — bus
+ renumbering affects every board).
+- **`hil_test.py` guard** (small patch to the per-board worker): take the
+ board's flock non-blocking before flashing and hold it for that board's
+ flash+test; on acquire it writes its own holder info
+ (`{pid, reason: "hil_test.py", since}`) so conflicts report truthfully in
+ both directions. If already held, FAIL the board immediately —
+ `FAILED (board locked: <holder info>)` — no flash, no waiting.
+ The CI job fails visibly for exactly those boards and `re-run failed`
+ passes once the lock is released (`build.yml` already retries the HIL step
+ once, absorbing short dev sessions). Guard defaults to proceeding if the
+ lock dir is absent/odd.
+- **Re-entrancy rule:** dev sessions do NOT pre-hold boards they are about to
+ run `hil_test.py` on (it self-locks; pre-holding deadlocks it).
+ `board_lock.py hold` is for hardware work outside `hil_test.py` only.
+- **Symmetric conflicts (CI running while an agent tests):** CI mid-test on a
+ board holds that board's flock, so the dev side hits it — `board_lock.py
+ hold` refuses showing the holder, and a dev `hil_test.py` run fails that
+ board fast. Two sessions can never double-flash a board.
+ `hil-validate.js` retries locked boards once at the end of its loop (CI
+ finishes a board in minutes); manual sessions wait and retry when the
+ holder reason is `hil_test.py`. Concurrent activity on *different* boards
+ is normal — CI's own `hil_test.py` already runs boards in parallel via
+ `multiprocessing.Pool`.
+- **User decision on persistent locks:** workers cannot prompt the user, so
+ boards still locked after the retry are returned in `locked[]`; the main
+ session then asks the user — **force** (re-invoke `hil-validate` with
+ `force: true`, which runs `hil_test.py` with `HIL_NO_BOARD_LOCK=1`: a
+ bypass, never killing the holder, at the user-accepted risk of colliding
+ with a mid-test CI job), **continue waiting** (re-invoke later), or
+ **accept** the partial result. Workers never force on their own; forcing
+ requires the user's explicit authorization relayed in the prompt.
+- **Propagation caveat:** CI enforces the guard only once the patch lands on
+ master (CI runs `hil_test.py` from each PR's merge-with-master ref). This
+ also widens blast radius beyond `.claude/` config into shared CI test
+ infra — the change is a small isolated guard, but it needs its own CI pass
+ and careful review on the eventual PR.
+
+### Entry point — `.claude/skills/pre-pr/SKILL.md`
+
+`/pre-pr` instructs the session to: scout the diff inline (cheap), map changed
+`src/portable/<vendor>/<ip>` and `src/class/*` to affected families and pick
+test boards from `hw/bsp` (fallback: `stm32f407disco`, `raspberry_pi_pico`),
+launch `full-check` with that board list, and summarize the verdict. Markdown
+carries the judgment; JS carries the orchestration.
+
+## Model & effort policy
+
+- Tiered worker models: `port-dev`/`driver-reviewer` **opus** `xhigh`;
+ `hil-operator`/`pr-monitor` **sonnet**; `builder` **haiku**.
+- Inline workflow stages: unit/size **haiku**; pvs **sonnet** (low effort);
+ pr-babysit push/replies **sonnet**.
+- Agent frontmatter `model:` is canonical for `agentType` calls; it is read
+ at session-start registration, so tier changes apply from the next session.
+- Deterministic control flow (loops, joins, filtering, verdict assembly) is
+ plain JS in the workflow scripts — zero model tokens.
+- The orchestrating session derives work lists inline (glob/grep) and passes
+ them as `args`; worker prompts carry paths, not file contents.
+
+## Error handling
+
+- Workers always return schema-validated structured output; validation retries
+ happen at the tool-call layer.
+- Workflows `filter(Boolean)` for killed agents and `log()` every skipped or
+ dropped item — no silent truncation.
+- `validate.js` reports partial results (a failed stage becomes a failure entry,
+ not a thrown error).
+- `hil-validate.js`: board access arbitrated by `hil_test.py` self-locking; a
+ locked board fails fast with the holder info (never forced); wedged boards
+ reported, not retried blindly.
+
+## Success criteria
+
+Each piece smoke-tested on a minimal real scope before the branch is done:
+
+1. `validate.js` on `stm32f407disco` + `raspberry_pi_pico` (real build, unit
+ tests, size compare, PVS) returns a correct verdict object.
+2. `fanout-dev.js` on a trivial 2-port task on this branch; diffs build clean
+ and are `.clang-format`-clean.
+3. `driver-review.js` on 2 driver dirs returns only verified findings.
+4. `hil-validate.js` on 1 board against the real rig with the actions-runner
+ ACTIVE throughout: while a `board_lock.py` hold is in place the run fails
+ fast citing the holder and returns the board in `locked[]`; with the lock
+ still held, `force: true` proceeds (user-authorized bypass); after
+ release a normal run passes; a direct `hil_test.py` run under a held lock
+ fails that board without flashing.
+5. `/pre-pr` end-to-end on this branch's own diff.
+6. `pr-babysit.js` with `autoPush: false` (dry cycle) on a real open PR:
+ CI failures classified correctly, at least one bot finding correctly
+ validated or rejected, proposed fixes produced but not pushed.
+
+## Out of scope
+
+- CI (GitHub Actions) integration — this harness is for interactive sessions.
+- Fuzzing orchestration.
+- usbtest/usbtest-host stress batteries (existing separate branch).
+- Removing the untracked `port-audit.js` prototype from the master working
+ tree happens when this branch merges (it is not tracked by git).