diff --git a/pstack/docs/guide/06-verify-and-ship.md b/pstack/docs/guide/06-verify-and-ship.md index 67cecbac..c53c4573 100644 --- a/pstack/docs/guide/06-verify-and-ship.md +++ b/pstack/docs/guide/06-verify-and-ship.md @@ -38,8 +38,6 @@ It writes `.cursor/skills/verify-/`, agent-facing instructions with exact L From then on, "verify it in the app" is a step any agent can execute, in this repo, with no setup conversation. -Once the verify skill works, a [`/swarm`](../../skills/swarm/SKILL.md) can split a full pass by feature-map entry and aggregate the results. - ## Keep the verification skill honest Apps change and feature maps rot. When yours drifts, run: @@ -48,7 +46,7 @@ Apps change and feature maps rot. When yours drifts, run: /maintain-verification-skill ``` -[`/maintain-verification-skill`](../../skills/maintain-verification-skill/SKILL.md) audits the generated skill: one read-only source reader per feature in parallel, then one live pass that drives every mapped feature. It ends in exactly one of three outcomes. `clean` means full coverage and nothing to ship. `changed` means one PR of proven corrections, confined to the verification skill's own directory. `blocked` names the blocker. It never edits product code. If the live pass catches a product regression, it reports the regression instead of papering over it in docs. +[`/maintain-verification-skill`](../../skills/maintain-verification-skill/SKILL.md) audits the generated skill: one read-only `maintain source readers` child per feature in parallel, then one live pass that drives every mapped feature. It ends in exactly one of three outcomes. `clean` means every feature got source coverage and a live drive or a `verified-unreachable` classification, and nothing to ship. `changed` means one PR of proven corrections, confined to the verification skill's own directory. `blocked` names the blocker. It never edits product code. If the live pass catches a product regression, it reports the regression instead of papering over it in docs. Do not wrap that pass in `/swarm`. ## Open the PR diff --git a/pstack/skills/maintain-verification-skill/SKILL.md b/pstack/skills/maintain-verification-skill/SKILL.md index a2680b91..68b214b6 100644 --- a/pstack/skills/maintain-verification-skill/SKILL.md +++ b/pstack/skills/maintain-verification-skill/SKILL.md @@ -8,13 +8,15 @@ disable-model-invocation: true A feature map rots the moment the app changes. This skill is the upkeep loop for a skill generated by `/create-verification-skill` (or any project-local verification skill with a feature map). The unit of rigor is the feature, not every sentence: cover every feature file from source and exercise every feature live, without terminalising every bullet. +Do not wrap this pass in `/swarm`. Source readers are the `maintain source readers` role, not swarm workers and not pstack candidates. The coordinator owns every live drive. + ## Outcomes Pick one, and say which: -- **clean** — every feature got source and live coverage; nothing worth shipping. No branch, no PR. +- **clean** — every feature got source coverage and either a live drive or a `verified-unreachable` with named prerequisite and route attempted; nothing worth shipping in the verification skill. No branch, no PR. A product gap named in the report does not by itself make the pass `changed` or `blocked`. - **changed** — one PR ships proven doc, harness, or map corrections. -- **blocked** — coverage could not finish or a proven fix could not ship safely. Say exactly what blocked it. +- **blocked** — a feature lacked a source-wave return, was neither driven nor classified `verified-unreachable`, or a proven fix could not ship safely. Say exactly what blocked it. ## Edit scope @@ -22,15 +24,15 @@ Only edit the verification skill's own directory (its SKILL.md, features/, and a ## Pass -0. **Locate the target.** Find the verification skill to maintain: the project-local skill whose body has launch/drive sections and a feature map (usually `.cursor/skills/verify-*/`). Several candidates → ask which one; none → stop and point at `/create-verification-skill` instead of inventing a target. +0. **Locate the target.** Find the verification skill to maintain: the project-local skill whose body has launch/drive sections and a feature map (usually `.cursor/skills/verify-*/`). If several match, pick the one whose path appears in the current diff. If that does not uniquely identify one skill, stop. Name the candidates. Do not guess. If none match, stop and point at `/create-verification-skill` instead of inventing a target. 1. **Index hygiene.** Read the feature map README and glob its sibling files. Fix missing, extra, duplicate, or dead entries. Lightweight; no generated inventory. -2. **Source wave.** One read-only subagent per feature file, launched concurrently. Each explains "how does this user-facing feature work?" from source, flags likely doc drift with citations, and returns one concise live-verification recipe. Children never drive the app and never edit files. Return shape: feature summary / source entry points / likely drift or none / one recipe. +2. **Source wave.** One `maintain source readers` child per feature file, launched concurrently. Spawn each with `subagent_type: generalPurpose`, `readonly: true`, and your configured maintain source readers model (default `inherit-parent`, omit Task `model`). Each explains "how does this user-facing feature work?" from source, flags likely doc drift with citations, and returns one concise live-verification recipe. Children never drive the app and never edit files. Return shape: feature summary / source entry points / likely drift or none / one recipe. 3. **Reconcile.** Every feature file has a returned summary. Merge overlapping recipes into as few app states as practical. Spot-check cited drift; don't re-prove clean claims. Sweep recent churn for user-facing surfaces missing from the map — require a concrete source path before calling one missing. -4. **Live pass.** Required even when source looks clean. The coordinator owns all driving; follow the verification skill's own launch model — one long-lived instance driven serially for servers and UIs, or a fresh isolated session per drive for short-lived CLIs (the skill's Launch section decides, not this one). Exercise every feature at least once, and hold three invariants the whole pass, whatever the failure: (1) never drive an instance you haven't health-checked since it last did something surprising — doctor before first drive, doctor on each fresh session where sessions are the unit, doctor again after any failed drive, and where doctor can't see the failure (a wedged UI state on a healthy process), reset to a known state or relaunch rather than hoping; (2) evidence captured so far survives every cleanup, checked at its named location, not assumed; (3) nothing a drive started outlives that drive's usefulness — failed-iteration residue is cleaned whether the session is stuck, exited, or shared (for a shared instance, clean the residue, not the instance). A doctor failure caused by skill drift is drift: fix it under edit scope and retry once — restart whatever the fix invalidated, nothing more — before calling the pass `blocked`. A feature that can't be reached is `verified-unreachable` only with the concrete prerequisite (auth, entitlement, OS, external state) and the route attempted; if the map omits that prerequisite, that's drift. Any harness fix from triage gets re-driven live before it ships. Final teardown happens after the last drive of the run — including those re-proofs — so nothing outlives the run (evidence stays, per the skill). +4. **Live pass.** Required even when source looks clean. The coordinator owns all driving. Drive the map's `Driving it with ` text, not the child's recipe. Child recipes are merge input for spotting drift. Follow the verification skill's own launch model — one long-lived instance driven serially for servers and UIs, or a fresh isolated session per drive for short-lived CLIs (the skill's Launch section decides, not this one). Exercise every feature at least once, and hold three invariants the whole pass, whatever the failure: (1) never drive an instance you haven't health-checked since it last did something surprising — doctor before first drive, doctor on each fresh session where sessions are the unit, doctor again after any failed drive, and where doctor can't see the failure (a wedged UI state on a healthy process), reset to a known state or relaunch rather than hoping; (2) evidence captured so far survives every cleanup, checked at its named location, not assumed; (3) nothing a drive started outlives that drive's usefulness — failed-iteration residue is cleaned whether the session is stuck, exited, or shared (for a shared instance, clean the residue, not the instance). A doctor failure caused by skill drift is drift: fix it under edit scope and retry once — restart whatever the fix invalidated, nothing more — before calling the pass `blocked`. A feature that can't be reached is `verified-unreachable` only with the concrete prerequisite (auth, entitlement, OS, external state) and the route attempted; if the map omits that prerequisite, that's drift. Any harness fix from triage gets re-driven live before it ships. Final teardown happens after the last drive of the run — including those re-proofs — so nothing outlives the run (evidence stays, per the skill). 5. **Triage.** Wrong or missing user-POV description → doc drift, fix it. Working behavior the harness can't drive → harness gap, fix it; a harness fix follows the same helpers rule as generation (scripts executable, invocation documented in the skill body). App behavior that's actually broken → product gap; record it for the user, keep it out of this PR. diff --git a/pstack/skills/setup-pstack/SKILL.md b/pstack/skills/setup-pstack/SKILL.md index ee882deb..6a2488a3 100644 --- a/pstack/skills/setup-pstack/SKILL.md +++ b/pstack/skills/setup-pstack/SKILL.md @@ -19,7 +19,7 @@ The default role-to-model mapping is the rule shape shown in step 5 below. If `~ ### 3. Map and confirm -Show every role with its current model, marking any real slug not in the detected set as needing a choice. Ask whether to accept as-is or change specific roles, offering the detected models plus `inherit-parent` and `auto` (both mean: this role runs on the parent chat model, which is how Auto users stay on Auto) as the options. Prefer AskQuestion over free text. For panel roles (how critics, arena runners, architect runners, interrogate reviewers) the value is a list, and one subagent runs per entry, alias entries included, so the list length sets the count. `arena cross-judge pool` is also a list, but Arena selects one value from it whose model family differs from the parent's when possible. `swarm workers` is the default model for every worker unless a race or comparison assigns another model per arm. +Show every role with its current model, marking any real slug not in the detected set as needing a choice. Ask whether to accept as-is or change specific roles, offering the detected models plus `inherit-parent` and `auto` (both mean: this role runs on the parent chat model, which is how Auto users stay on Auto) as the options. Prefer AskQuestion over free text. For panel roles (how critics, arena runners, architect runners, interrogate reviewers) the value is a list, and one subagent runs per entry, alias entries included, so the list length sets the count. `arena cross-judge pool` is also a list, but Arena selects one value from it whose model family differs from the parent's when possible. `swarm workers` is the default model for every worker unless a race or comparison assigns another model per arm. `maintain source readers` is the model for each `/maintain-verification-skill` source-wave child (default `inherit-parent`). They are not swarm workers. ### 4. Validate @@ -52,6 +52,7 @@ reflect judgment, divergent, synthesizer: claude-fable-5-thinking-max arena runners: claude-fable-5-thinking-max, gpt-5.6-sol-max, grok-4.6-fast-xhigh, claude-opus-5-thinking-xhigh arena cross-judge pool: claude-fable-5-thinking-max, gpt-5.6-sol-max, grok-4.6-fast-xhigh, claude-opus-5-thinking-xhigh swarm workers: grok-4.6-fast-xhigh +maintain source readers: inherit-parent architect runners: claude-fable-5-thinking-max, gpt-5.6-sol-max, grok-4.6-fast-xhigh, claude-opus-5-thinking-xhigh interrogate reviewers: claude-fable-5-thinking-max, gpt-5.6-sol-max, grok-4.6-fast-xhigh, claude-opus-5-thinking-xhigh ```