Skip to content

feat(plugin): add stage-scoped verification agent - #282

Merged
hffmnnj merged 26 commits into
mainfrom
feat/stage-verification-role
Aug 3, 2026
Merged

hffmnnj merged 26 commits into
mainfrom
feat/stage-verification-role

Conversation

@hffmnnj

@hffmnnj hffmnnj commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a dedicated verification agent for execution-stage quality checks
  • register its model and thinking defaults across plugin adapters
  • expose the new role in settings and example configuration
  • add focused registration and prompt-invariant coverage

Verification

  • plugin typecheck passes
  • plugin suite: 2328 passed, 0 failed
  • settings changes are baseline-equivalent with no diagnostics in edited pages

hffmnnj and others added 26 commits August 3, 2026 06:21
Register a per-wave verification role distinct from the acceptance
verifier, across AGENT_ROLES, DEFAULT_MODEL_MAP, and a new agent
definition so the coupled registries stay compile-consistent
(Record<AgentRole, string> ties DEFAULT_MODEL_MAP to the role list).
Bump the AGENT_ROLES length assertion, whose `as const` makes it a
typecheck-level dependency on the role list rather than a runtime one.
Update hard-coded agent counts (14->15) and prompt-file count (23->24),
add the new role to the DEFAULT_MODEL_MAP roster, assert its derived
high thinking level, and re-baseline the prompt-audit census (files,
bytes, bold spans, absolute-language total) for the added agent file.
Two invariant suites still flag the new prompt itself and are tracked
separately.
Add the standard core-protocol boot pointer in a Mandatory First Steps
section (required by the memory-first-boot semantic invariant shared by
every agent) and prefix the delegation prohibition with an explicit
"Do not" cue so the tool-grant scanner recognizes it as prohibitive
rather than a required grant. Refresh the agents-directory byte snapshot
for the added prose; file and bold-span counts are unchanged.
…intained settings surfaces

Keeps the webfront role taxonomy and inlined DEFAULT_MODEL_MAP in sync
with the canonical plugin registry so the new wave-verifier role is
listed and resolves its default model in all agent/model settings pages.
Both adapters consume the shared AGENT_ROLES / DEFAULT_THINKING_LEVELS
registries, so a single role addition flows through with no adapter
branch. Pin that contract with focused assertions: V1 config.agent
surface plus chat.params high-level resolution, and V2 agent-transform
high-variant application.
Introduces isVerifierDispatchAllowed (role x phase dispatch check,
fail-open on undetermined roles) and isWaveVerified (gate satisfied
by >=1 verification row with none failing; an explicit skip row is
an auditable escape). Also centralizes the pass|fail|skip -> passed|
failed|skipped status mapping in one place so future integration
points stop duplicating it. No DB access, hooks, or tool changes —
runtime enforcement is wired up separately.
…tion

Adds a tool.execute.before guard that denies a task delegation to
goop-verifier outside accept or goop-wave-verifier outside execute,
with an actionable message naming the correct role and stage. Real
blocking requires a thrown error to reach the host, but the shared
hook chain (mergeHooks/chainHandlers) unconditionally swallows every
handler error via safeHandler so a denial would never propagate.
Introduces IntentionalToolDenialError, a narrowly-scoped sentinel that
safeHandler rethrows instead of catching — every other error, from
this hook or any other, keeps today's catch-and-log behavior
unchanged. Fails open on unrecognized tools, roles, or an
undeterminable workflow phase.
…ification

A wave can no longer transition to done/completed, and the workflow can no
longer auto-progress execute→accept, without at least one passing or
explicit-skip verification row for the final wave. Zero rows and any failing
row block; an explicit skip is an auditable escape. Same-call verifications
are considered before the completion decision, and denial rolls back the
whole call atomically, including the verification rows just supplied.

Also fixes a stored-vocabulary bug at the verification insertion seam:
goop_write_wave was writing raw pass|fail|skip into a column whose
predicates (and the DB's own passed|failed|skipped enum) expect the
canonical form, so a stored 'fail' silently satisfied the not-equal-to-
'failed' gate check. Normalisation now happens once, at insertion.
…atrix

Seed explicit verification rows in fixtures that drive a wave to a
complete status — the completion gate now rejects bare complete-status
writes, so durability and integration scenarios carry the evidence their
semantics already implied. Update read-side assertions to the canonical
DB status vocabulary (passed/failed) that recordVerification normalises
to at the insertion seam. Add a merged-pipeline timing matrix (both
verifier roles x five phases) through createHooks plus a contrast case
proving ordinary hook errors degrade safely while a deliberate denial
propagates.
The execute command now runs a mandatory verification step after each
wave's tasks before the wave can be marked complete or its PR opened.
This closes the gap where a wave could finish and stack a PR with no
verification record, even though the runtime already rejects the
completion transition without one.

Changes:
- Add a wave verification gate to goop-execute.md: dispatch the wave
  auditor scoped to the wave, record verification rows via the wave
  tool, remediate gaps with a bounded retry budget, and only then mark
  the wave complete and open its PR
- Align wiring-checklist.md and task-decomposition.md boundary and
  handoff guidance with the runtime gate and its predicate
- Update the prompt-surface audit snapshot counts for the prose delta
Wave verification is now a distinct goop-wave-verifier dispatch during
execute; goop-verifier is acceptance-only across the dispatch-patterns,
orchestrator, help, and verifier prompt surfaces. Update agent counts to
the 15-role roster and refresh the prompt-audit snapshot.
…ication turns

Wave 3 Task 3 (MH3/MH7/MH8). Adds end-to-end integration coverage for the
wave-verification remediation loop and fixes a lazy-autopilot stall across
verification-only turns.

MATERIAL DEVIATION surfaced to ADL: MH3's literal 'fail row → remediation
→ later passing evidence → wave completion succeeds' is impossible under
locked Wave 2 D2 (verifier-stage.ts:104-107; insert-only rows; gate
predicate rejects any wave with a historical fail). The integration
scenarios model the supported auditable mechanisms instead:

  - Happy path: pass row recorded in the completion call (or earlier with
    no fail row present) → wave completion succeeds → auto-progression
    fires execute → accept.
  - Gate enforcement: fail row blocks completion; a later pass row does
    NOT unblock (locked D2 pinned end-to-end); the supported terminal
    state per goop-execute.md:50 is to open a high-severity blocker via
    goop_blocker, which the lazy-autopilot G_high-severity-blocker guard
    observes and uses to suppress further nudges.

Lazy-autopilot fingerprint fix (rate-limit.ts): the progress fingerprint
previously digested only phase|currentWave|task-status-digest. A
verification-only turn (wave-verifier records evidence without changing
any task status) left the fingerprint identical, so the consecutive
counter did not reset and the session was eventually abandoned despite
real forward motion. Added a verification-row digest slot
(|v:<check_name:status,...>) ordered by insertion, so a
verification-only turn moves the fingerprint and resets the counter.

Does not weaken the Wave 2 completion or stage-dispatch gates. Existing
fail-closed session/project guards, cooldown, cap, and V1-only behavior
preserved.
Reconciles MH3's bounded remediation loop with the W2 D2 append-only
verification-row invariant. Rows are never deleted or updated, but
isWaveVerified now computes the effective status per check_name as
the latest (highest-id) row for that check rather than "zero failed
rows across all history ever". A later pass/skip for the SAME check
supersedes that check's own prior fail; a pass for a DIFFERENT check
never resolves another check's unresolved fail. The predicate sorts
by row id itself, so it stays correct regardless of caller ordering.

- verifier-stage.ts: VerifiedRowLike gains id/check_name; isWaveVerified
  reduces to latest-per-check before checking for a failing status.
- verifier-stage.test.ts: adds latest-per-check coverage (same-check
  supersession, different-check non-supersession, order independence).
- auto-progression.test.ts, goop-write-wave/index.test.ts: replace the
  "later pass never unblocks" fixtures with same-check-unblocks /
  different-check-stays-blocked pairs.
- integration.test.ts: replaces the permanent-fail Scenario B with the
  literal fail -> remediate -> same-check pass -> complete -> progress
  story, and adds Scenario B2 for the different-check block + blocker +
  lazy-autopilot-nudge-suppression path.
- goop-execute.md, goop-wave-verifier.md: correct prose that pinned the
  superseded "any historical fail blocks forever" reading.
- prompt-audit/index.test.ts: mechanically re-measured byte/abs-hit
  snapshots for the two reworded docs.

Full plugin suite: 2511 pass / 0 fail. Typecheck clean. Biome: no new
issues (pre-existing tabs-vs-spaces and organizeImports/format items
unchanged).
Add a self-contained positive-fixture-inventory describe block to each of
the five write tools, locking every currently-valid call shape as a passing
fixture before Wave 4 Task 2/3 tightens runtime validation.

Inventory per tool:
- goop_write_db: 6 shapes (full write, append, patch, replace-all,
  items[], blank-doc patch workaround)
- goop_write_section: 5 shapes (section create, patch, replace-all,
  items[], blank-doc patch workaround; append N/A — no mode field)
- goop_save_note: 5 shapes (create, patch, replace-all, items[],
  empty-body patch workaround; append N/A)
- goop_write_wave: 2 generic + 5 tool-specific shapes (wave write,
  items[], task_update, task_updates[], verifications[], traceability[],
  traceability-only without wave_number; patch/append/blank-doc N/A)
- goop_blocker: 1 generic + 3 tool-specific shapes (open, resolve, list,
  items[]; patch/append/blank-doc N/A)

Test-only; no production code changed. Inapplicable shapes documented
inline with structural rationale. SPEC: MH9, MH11.
… helper (W4.T2)

Replace presence-based isPatchActive/resolveContentForWrite branching with
a pure resolveWriteMode() resolver (src/shared/write-mode.ts) that decides
full-write/append, patch, or batch intent — and rejects four ambiguous
shapes before any mutation:

1. meaningful content + old_string together
2. new_string/replace_all without old_string (previously silently dropped)
3. mode: "append" combined with patch fields
4. items[] combined with a top-level operation field

Empty-string content stays a documented neutral placeholder when combined
with items[] (matches existing batch fixtures); old_string presence alone
(even "") still activates the blank-document patch workaround. Applied to
both the single-call and per-item batch paths, so an ambiguous item now
rolls back the whole batch transaction instead of silently mismatching.

All W4.T1 pinned positive fixtures remain green. Adds full truth-table
coverage for the resolver plus rejection/mutation-safety tests on the tool.

SPEC: MH10, MH9
…bling tools (W4.T3)

- goop_write_section: reuse resolveWriteMode() for both the single-section
  and per-item batch paths, since it shares goop_write_db's content/
  old_string/new_string/replace_all shape (no append mode). content +
  old_string together and items[] + meaningful top-level operation fields
  are now rejected before mutation instead of old_string silently winning.
- goop_save_note: add a note-specific resolveNoteMode() mirroring
  resolveWriteMode()'s intent for a CREATE-or-PATCH-keyed-on-note_id shape
  that doesn't map onto the generic content/old_string fields. Rejects
  note_id + meaningful create fields together, and patch-only fields
  (old_string/new_string/replace_all) supplied without note_id, in both
  the single and batch paths.
- goop_write_wave: audited against the one-shot standard; already the
  reference implementation. Added a missing regression test for
  task_update + task_updates[] supplied together (already guarded by the
  existing ignoredFields mechanism, previously untested for this pair).
  No production change.
- goop_blocker: audited and found the analogous defect — items[] silently
  ignored meaningful top-level action/description/severity/wave_id/id/
  resolution/status. Added the same ignoredFields rejection used by
  goop_write_wave (workflow_id excluded: it is genuinely consumed as the
  per-item default).

All W4.T1 positive fixtures remain green, including empty-content/
section_key/note-field neutral batch placeholders.
…y-loop guarantee

Lock the V2 JSON Schema boundary (convertToolArgsToJsonSchema) for every
createTools() entry: assert properties exist and `required` equals an explicit
intended set per tool. Catches a new tool added without a contract entry and
any drift in requiredness — the exact defect class that let optional patch
fields leak into `required` and turn full-document writes into silent patches.

Add a retry-loop regression proving the W4.T2 runtime validation returns a
single actionable error (naming conflicting fields and valid shapes), performs
no mutation, and differs from the prior silent-patch success so an agent
cannot loop on the identical payload.

Host-side V1 conversion remains outside plugin control (opencode #20807,
#13618); documented in-file. Runtime validation is the real safety guarantee.
… (W5.T1)

Single integration test file wires the seams that unit tests already
cover in isolation through their real production entry points:
- goop-wave-verifier registration via the V1 config hook and the V2
  registerHooksV2 agent/catalog transform adapter (shared AGENT_ROLES
  path)
- the real merged tool.execute.before guard chain (createHooks +
  DEFAULT_HOOK_FACTORIES) blocking/permitting goop-verifier and
  goop-wave-verifier by phase
- goop_write_wave's wave-completion verification gate: zero rows and a
  failing row both block completion; a later pass row for the same
  check_name supersedes it (append-only, latest-per-check)
- the real auto-progression hook advancing execute -> accept once the
  final wave carries verification evidence
- goop_acceptance_audit surfacing the append-only fail+pass history

Covers MH2, MH4, MH5, MH7.

Verification: rtk proxy bun test packages/opencode-plugin/ --bail=3
--timeout=10000 (2606 pass, 0 fail); rtk proxy bun run --cwd
packages/opencode-plugin typecheck (clean); focused biome check on the
new file (clean).
… test

Wave 5 Task 3 wiring/quality-gate check: lint diff against the Wave 4
base (1a9ed89) isolated exactly one net-new regression introduced by
this workflow — a formatter-only violation in
stage-gating.integration.test.ts (added in W5.T1, a4b0eb1). Applied
`biome format --write` scoped to that single file only; whitespace-only,
behavior-preserving. Post-fix lint diagnostics (10 errors / 17 warnings)
now match the Wave 4 baseline file-and-rule set exactly, confirming no
lint regression remains from this workflow. Remaining baseline errors
belong to files this workflow never touched (prompt-audit/report.ts,
continuation-prompt.test.ts, system-transform.test.ts) or were already
present at the Wave 4 boundary, and are out of this task's scope.
…config

Convert tab indentation to Biome's configured 2-space style and collapse
signatures/calls that fit within lineWidth:100 in
verifier-stage.ts and verifier-stage.test.ts. Formatting-only; no token,
type, test, or behavior change. Removes the two workflow-owned lint
diagnostics blocking the acceptance gate; unrelated baseline lint debt in
other modules remains classified as skip.
test(plugin): prove verification lifecycle end to end
fix(plugin): harden write tool payload contracts
feat(plugin): add staged verification and remediation
feat(plugin): enforce stage-scoped verifier delegation
@hffmnnj
hffmnnj merged commit 77c4fef into main Aug 3, 2026
0 of 2 checks passed
@hffmnnj
hffmnnj deleted the feat/stage-verification-role branch August 3, 2026 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant