diff options
| author | hathach <[email protected]> | 2026-07-09 23:34:29 +0700 |
|---|---|---|
| committer | hathach <[email protected]> | 2026-07-09 23:34:29 +0700 |
| commit | e3dd9245ef08c457d7c6e5837e3f8d41bad6fd8a (patch) | |
| tree | 4793e68120f4836e50cb3f51d2dd8d25a6095ab3 /docs/superpowers/specs | |
| parent | fa750d6bf045f1df4314d283dfe0508d0d066559 (diff) | |
feat: Claude Code multi-agent dev/test harness for TinyUSB
Add worker agents (builder, port-dev, driver-reviewer, hil-operator,
pr-monitor), deterministic workflows (validate, fanout-dev, driver-review,
hil-validate, full-check, pr-babysit) and a /pre-pr gate skill, so sessions
can fan build/test/review/PR-triage work out to tiered subagents. pr-babysit
drives a PR to green: triage CI + bot reviews, fix validated findings, verify,
push, and reply-to + resolve each inline review thread (fixed or refuted).
Replace the stop-the-runner HIL discipline with per-board flock locks:
test/hil/board_lock.py plus a fail-open guard in hil_test.py let CI and dev
sessions share the rig per board (locked boards fail fast and re-run;
HIL_NO_BOARD_LOCK=1 is a user-authorized bypass). The actions-runner is
never stopped.
Design spec, implementation plan, and real-rig smoke evidence under
docs/superpowers/.
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Rn1AN5DsTdFhRwhugfgKZi
Diffstat (limited to 'docs/superpowers/specs')
| -rw-r--r-- | docs/superpowers/specs/2026-07-09-claude-agents-workflows-design.md | 163 |
1 files changed, 163 insertions, 0 deletions
diff --git a/docs/superpowers/specs/2026-07-09-claude-agents-workflows-design.md b/docs/superpowers/specs/2026-07-09-claude-agents-workflows-design.md new file mode 100644 index 000000000..4638239a6 --- /dev/null +++ b/docs/superpowers/specs/2026-07-09-claude-agents-workflows-design.md @@ -0,0 +1,163 @@ +# Multi-Agent Dev/Test Setup for TinyUSB — Design + +Date: 2026-07-09 +Branch: worktree-claude-agents-workflows + +## Goal + +Give Claude Code sessions in this repo a reusable, efficient multi-agent harness +for developing and testing TinyUSB: custom worker agents that already know the +repo's build/test/rig discipline, and small deterministic workflows that fan +them out. The orchestrator (main session) authors arguments and reads verdicts; +workers do the volume. + +## Context + +- Existing process skills: `hil`, `code-size`, `pvs`, `build-doc`, `usbmon`, + `usb-debug`, `usb-recover`, `make-release` (`.claude/skills/`). +- One prototype workflow exists in the master working tree (untracked): + `.claude/workflows/port-audit.js`. This design supersedes it. +- No custom agent definitions exist yet (`.claude/agents/` absent). +- Test infra: `test/unit-test` (ceedling), `test/hil` (`hil_test.py`, + `tinyusb.json`), `test/fuzz`; size metrics via + `tools/metrics_compare_base.py`. + +## Architecture + +Layered: **agents** (who does the work, with baked-in domain knowledge) × +**workflows** (deterministic fan-out/join) × **one skill** (human entry point). + +### Worker agents — `.claude/agents/*.md` + +Tiered models (owner revision 2026-07-09; originally all-opus): `port-dev` +and `driver-reviewer` on **opus** at **xhigh**; `hil-operator` and +`pr-monitor` on **sonnet**; `builder` on **haiku** (mechanical, log-heavy). + +| Agent | Effort | Role | +|---|---|---| +| `builder` | low | Build one board's example set with the canonical commands: `cmake -B cmake-build-<board> -DBOARD=<board> -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel` from `examples/` (exact dir name — HIL expects it), `python3 tools/get_deps.py` on missing deps, `. $HOME/code/esp-idf/export.sh` for Espressif boards, tolerate non-critical objcopy failures. Returns structured `{board, pass, failures: [{example, firstError}]}`. | +| `port-dev` | xhigh | Implement one well-scoped change in one port / file set. Follows repo rules: C99, 2-space indent, snake_case, `TU_ASSERT`, no dynamic allocation, ISR work deferred to task context. Runs `clang-format` (repo `.clang-format`) on touched files before finishing. Cross-checks the MCU datasheet in `$HOME/Documents/calibre-library` when changing dcd/hcd register logic. Verifies with a targeted build of one board using the port. Returns `{item, diffstat, buildOk, notes}`. | +| `driver-reviewer` | xhigh | Review one dcd/hcd directory against dimensions: correctness, ISR safety, register use vs. datasheet AND MCU errata (calibre library; missing erratum workarounds are findings), style. Returns structured findings `{file, line, snippet, why, severity, confidence}` — coverage-first (report everything; filtering happens downstream). | +| `hil-operator` | default | All rig interaction — the actions-runner service is NEVER stopped; per-board flock locks arbitrate with concurrent CI. `hil_test.py` runs rely on its per-board self-locking; manual hardware work (JLink/GDB, usbtest, serial) is wrapped in `test/hil/board_lock.py hold/release`; rig-wide ops (uhubctl, pci-rebind) require `hold --all`; on wedge `usb_recover.sh` + dmesg. Used strictly serially — never two instances concurrently. | +| `pr-monitor` | default | Triage one GitHub PR via `gh`: check CI status (`gh pr checks`), read failing run logs and classify each failure infra/flake vs real; re-run infra failures (`gh run rerun --failed`); harvest automated review comments (Codex/Copilot/Claude bots — knows their signals: Codex posts a "Didn't find any major issues" issue comment when clean; Copilot drops out of `requested_reviewers` when done; bot logins differ across APIs); adversarially validate each finding against the actual code. Returns structured triage `{ci: {status, infraRerun[], realFailures[]}, findings: [{source, file, line, claim, verdict, fixHint}]}`. Read/triage/re-run/reply only — never edits code. | + +### Workflows — `.claude/workflows/*.js` + +| Workflow | Args | Shape | +|---|---|---| +| `validate.js` | `{boards[], examples?, base?, skip?: ('unit'\|'size'\|'pvs')[]}` | One parallel stage: unit tests (ceedling) + one `builder` per board + code-size compare (`tools/metrics_compare_base.py` vs `base`, default master) + PVS analyze. Join → plain-JS verdict `{pass, failures[]}`. Barrier is correct here: the verdict needs all results. | +| `fanout-dev.js` | `{task, items[], board?, review?, worktree?}` | `pipeline(items)`: `port-dev` per item → `builder` verify → optional `driver-reviewer` pass. Workers share the tree by default (ports are disjoint directories); `worktree: true` switches on per-agent worktree isolation for collision-prone tasks. Returns per-item results. | +| `driver-review.js` | `{dirs[], dimensions?, question?}` | Supersedes `port-audit.js`. Scan stage per (dir × dimension) → adversarial verify per finding (verifier prompted to refute) → confirmed findings only. | +| `hil-validate.js` | `{boards[], force?}` | Strictly serial `for` loop of `hil-operator` calls; each board is protected by `hil_test.py`'s own per-board flock, so the actions-runner keeps running throughout. Boards found locked (a concurrent CI job mid-test) are retried once at the end of the loop; boards still locked are returned in `locked[]` for a user force/wait/accept decision. `force: true` (user-authorized only) bypasses locks via `HIL_NO_BOARD_LOCK=1`. Returns per-board `{board, pass, detail}` plus `wedged[]` and `locked[]`. | +| `full-check.js` | `{boards[], ...}` | Thin composer: `workflow('validate', ...)` → only if green → `workflow('hil-validate', ...)`. Single nesting level (children do not nest further). | +| `pr-babysit.js` | `{pr, maxCycles?, autoPush?}` | Cycle until CI green + review threads resolved, or `maxCycles` (default 3): `pr-monitor` triage (blocks on `gh pr checks --watch` while CI runs) → valid findings + real CI failures grouped by file/port → `port-dev` fix per group (pipeline) → `driver-reviewer` verifies each fix addresses its finding → one commit + push per cycle. Every actioned inline comment is both **replied to and marked resolved** (GraphQL `resolveReviewThread`): refuted findings get the refutation, fixed findings get a "fixed in <sha>" note. `autoPush` defaults true; **invoking this workflow is the explicit push authorization** for follow-up commits on that PR branch (scoped exception to the hold-pushes-until-told rule). | + +### Board lock protocol — `test/hil/` (repo code) + +CI and dev sessions share the rig concurrently; the actions-runner service is +never stopped. Arbitration is per-board kernel flocks in +`/tmp/tinyusb-hil-locks/<board>.lock` — auto-released when the holder process +dies (stale locks impossible; `/tmp` clears on reboot): + +- **`test/hil/board_lock.py`** (new tool): `hold <boards|--all> --reason TEXT` + spawns a background holder process flocking each board file (JSON + `{pid, reason, since}` written inside for debuggability); `release + <boards|--all>` kills holders; `status` lists them. `--all` is required + before rig-wide operations (uhubctl power cycling, pci-rebind — bus + renumbering affects every board). +- **`hil_test.py` guard** (small patch to the per-board worker): take the + board's flock non-blocking before flashing and hold it for that board's + flash+test; on acquire it writes its own holder info + (`{pid, reason: "hil_test.py", since}`) so conflicts report truthfully in + both directions. If already held, FAIL the board immediately — + `FAILED (board locked: <holder info>)` — no flash, no waiting. + The CI job fails visibly for exactly those boards and `re-run failed` + passes once the lock is released (`build.yml` already retries the HIL step + once, absorbing short dev sessions). Guard defaults to proceeding if the + lock dir is absent/odd. +- **Re-entrancy rule:** dev sessions do NOT pre-hold boards they are about to + run `hil_test.py` on (it self-locks; pre-holding deadlocks it). + `board_lock.py hold` is for hardware work outside `hil_test.py` only. +- **Symmetric conflicts (CI running while an agent tests):** CI mid-test on a + board holds that board's flock, so the dev side hits it — `board_lock.py + hold` refuses showing the holder, and a dev `hil_test.py` run fails that + board fast. Two sessions can never double-flash a board. + `hil-validate.js` retries locked boards once at the end of its loop (CI + finishes a board in minutes); manual sessions wait and retry when the + holder reason is `hil_test.py`. Concurrent activity on *different* boards + is normal — CI's own `hil_test.py` already runs boards in parallel via + `multiprocessing.Pool`. +- **User decision on persistent locks:** workers cannot prompt the user, so + boards still locked after the retry are returned in `locked[]`; the main + session then asks the user — **force** (re-invoke `hil-validate` with + `force: true`, which runs `hil_test.py` with `HIL_NO_BOARD_LOCK=1`: a + bypass, never killing the holder, at the user-accepted risk of colliding + with a mid-test CI job), **continue waiting** (re-invoke later), or + **accept** the partial result. Workers never force on their own; forcing + requires the user's explicit authorization relayed in the prompt. +- **Propagation caveat:** CI enforces the guard only once the patch lands on + master (CI runs `hil_test.py` from each PR's merge-with-master ref). This + also widens blast radius beyond `.claude/` config into shared CI test + infra — the change is a small isolated guard, but it needs its own CI pass + and careful review on the eventual PR. + +### Entry point — `.claude/skills/pre-pr/SKILL.md` + +`/pre-pr` instructs the session to: scout the diff inline (cheap), map changed +`src/portable/<vendor>/<ip>` and `src/class/*` to affected families and pick +test boards from `hw/bsp` (fallback: `stm32f407disco`, `raspberry_pi_pico`), +launch `full-check` with that board list, and summarize the verdict. Markdown +carries the judgment; JS carries the orchestration. + +## Model & effort policy + +- Tiered worker models: `port-dev`/`driver-reviewer` **opus** `xhigh`; + `hil-operator`/`pr-monitor` **sonnet**; `builder` **haiku**. +- Inline workflow stages: unit/size **haiku**; pvs **sonnet** (low effort); + pr-babysit push/replies **sonnet**. +- Agent frontmatter `model:` is canonical for `agentType` calls; it is read + at session-start registration, so tier changes apply from the next session. +- Deterministic control flow (loops, joins, filtering, verdict assembly) is + plain JS in the workflow scripts — zero model tokens. +- The orchestrating session derives work lists inline (glob/grep) and passes + them as `args`; worker prompts carry paths, not file contents. + +## Error handling + +- Workers always return schema-validated structured output; validation retries + happen at the tool-call layer. +- Workflows `filter(Boolean)` for killed agents and `log()` every skipped or + dropped item — no silent truncation. +- `validate.js` reports partial results (a failed stage becomes a failure entry, + not a thrown error). +- `hil-validate.js`: board access arbitrated by `hil_test.py` self-locking; a + locked board fails fast with the holder info (never forced); wedged boards + reported, not retried blindly. + +## Success criteria + +Each piece smoke-tested on a minimal real scope before the branch is done: + +1. `validate.js` on `stm32f407disco` + `raspberry_pi_pico` (real build, unit + tests, size compare, PVS) returns a correct verdict object. +2. `fanout-dev.js` on a trivial 2-port task on this branch; diffs build clean + and are `.clang-format`-clean. +3. `driver-review.js` on 2 driver dirs returns only verified findings. +4. `hil-validate.js` on 1 board against the real rig with the actions-runner + ACTIVE throughout: while a `board_lock.py` hold is in place the run fails + fast citing the holder and returns the board in `locked[]`; with the lock + still held, `force: true` proceeds (user-authorized bypass); after + release a normal run passes; a direct `hil_test.py` run under a held lock + fails that board without flashing. +5. `/pre-pr` end-to-end on this branch's own diff. +6. `pr-babysit.js` with `autoPush: false` (dry cycle) on a real open PR: + CI failures classified correctly, at least one bot finding correctly + validated or rejected, proposed fixes produced but not pushed. + +## Out of scope + +- CI (GitHub Actions) integration — this harness is for interactive sessions. +- Fuzzing orchestration. +- usbtest/usbtest-host stress batteries (existing separate branch). +- Removing the untracked `port-audit.js` prototype from the master working + tree happens when this branch merges (it is not tracked by git). |
