Skip to content

feat(plugin): add staged verification and remediation - #284

Merged
hffmnnj merged 15 commits into
feat/stage-verification-enforcementfrom
feat/execute-verification-loop
Aug 3, 2026
Merged

hffmnnj merged 15 commits into
feat/stage-verification-enforcementfrom
feat/execute-verification-loop

Conversation

@hffmnnj

@hffmnnj hffmnnj commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Summary

Adds a per-stage verification loop before branch completion and reserves the final audit role for acceptance. Remediation can now record new evidence without deleting historical results, while autonomous continuation recognizes verification-only progress.

Changes

  • Document the required verification, remediation, retry-limit, and blocker flow before completion
  • Restrict final-audit guidance to acceptance and align role counts across documentation
  • Evaluate append-only verification history using the latest result for each check
  • Include verification evidence in autonomous progress fingerprints
  • Add end-to-end coverage for fail, remediation, pass, completion, and progression

Testing

  • Full plugin suite: 2511 passed, 0 failed across 123 files
  • Independent focused verification: 285 passed, 0 failed
  • TypeScript typecheck: clean
  • Changed-file lint: no new findings

Notes

Depends on #283. No database schema changes; historical verification rows remain immutable.

hffmnnj added 8 commits August 3, 2026 11:36
The execute command now runs a mandatory verification step after each
wave's tasks before the wave can be marked complete or its PR opened.
This closes the gap where a wave could finish and stack a PR with no
verification record, even though the runtime already rejects the
completion transition without one.

Changes:
- Add a wave verification gate to goop-execute.md: dispatch the wave
  auditor scoped to the wave, record verification rows via the wave
  tool, remediate gaps with a bounded retry budget, and only then mark
  the wave complete and open its PR
- Align wiring-checklist.md and task-decomposition.md boundary and
  handoff guidance with the runtime gate and its predicate
- Update the prompt-surface audit snapshot counts for the prose delta
Wave verification is now a distinct goop-wave-verifier dispatch during
execute; goop-verifier is acceptance-only across the dispatch-patterns,
orchestrator, help, and verifier prompt surfaces. Update agent counts to
the 15-role roster and refresh the prompt-audit snapshot.
…ication turns

Wave 3 Task 3 (MH3/MH7/MH8). Adds end-to-end integration coverage for the
wave-verification remediation loop and fixes a lazy-autopilot stall across
verification-only turns.

MATERIAL DEVIATION surfaced to ADL: MH3's literal 'fail row → remediation
→ later passing evidence → wave completion succeeds' is impossible under
locked Wave 2 D2 (verifier-stage.ts:104-107; insert-only rows; gate
predicate rejects any wave with a historical fail). The integration
scenarios model the supported auditable mechanisms instead:

  - Happy path: pass row recorded in the completion call (or earlier with
    no fail row present) → wave completion succeeds → auto-progression
    fires execute → accept.
  - Gate enforcement: fail row blocks completion; a later pass row does
    NOT unblock (locked D2 pinned end-to-end); the supported terminal
    state per goop-execute.md:50 is to open a high-severity blocker via
    goop_blocker, which the lazy-autopilot G_high-severity-blocker guard
    observes and uses to suppress further nudges.

Lazy-autopilot fingerprint fix (rate-limit.ts): the progress fingerprint
previously digested only phase|currentWave|task-status-digest. A
verification-only turn (wave-verifier records evidence without changing
any task status) left the fingerprint identical, so the consecutive
counter did not reset and the session was eventually abandoned despite
real forward motion. Added a verification-row digest slot
(|v:<check_name:status,...>) ordered by insertion, so a
verification-only turn moves the fingerprint and resets the counter.

Does not weaken the Wave 2 completion or stage-dispatch gates. Existing
fail-closed session/project guards, cooldown, cap, and V1-only behavior
preserved.
Reconciles MH3's bounded remediation loop with the W2 D2 append-only
verification-row invariant. Rows are never deleted or updated, but
isWaveVerified now computes the effective status per check_name as
the latest (highest-id) row for that check rather than "zero failed
rows across all history ever". A later pass/skip for the SAME check
supersedes that check's own prior fail; a pass for a DIFFERENT check
never resolves another check's unresolved fail. The predicate sorts
by row id itself, so it stays correct regardless of caller ordering.

- verifier-stage.ts: VerifiedRowLike gains id/check_name; isWaveVerified
  reduces to latest-per-check before checking for a failing status.
- verifier-stage.test.ts: adds latest-per-check coverage (same-check
  supersession, different-check non-supersession, order independence).
- auto-progression.test.ts, goop-write-wave/index.test.ts: replace the
  "later pass never unblocks" fixtures with same-check-unblocks /
  different-check-stays-blocked pairs.
- integration.test.ts: replaces the permanent-fail Scenario B with the
  literal fail -> remediate -> same-check pass -> complete -> progress
  story, and adds Scenario B2 for the different-check block + blocker +
  lazy-autopilot-nudge-suppression path.
- goop-execute.md, goop-wave-verifier.md: correct prose that pinned the
  superseded "any historical fail blocks forever" reading.
- prompt-audit/index.test.ts: mechanically re-measured byte/abs-hit
  snapshots for the two reworded docs.

Full plugin suite: 2511 pass / 0 fail. Typecheck clean. Biome: no new
issues (pre-existing tabs-vs-spaces and organizeImports/format items
unchanged).
Add a self-contained positive-fixture-inventory describe block to each of
the five write tools, locking every currently-valid call shape as a passing
fixture before Wave 4 Task 2/3 tightens runtime validation.

Inventory per tool:
- goop_write_db: 6 shapes (full write, append, patch, replace-all,
  items[], blank-doc patch workaround)
- goop_write_section: 5 shapes (section create, patch, replace-all,
  items[], blank-doc patch workaround; append N/A — no mode field)
- goop_save_note: 5 shapes (create, patch, replace-all, items[],
  empty-body patch workaround; append N/A)
- goop_write_wave: 2 generic + 5 tool-specific shapes (wave write,
  items[], task_update, task_updates[], verifications[], traceability[],
  traceability-only without wave_number; patch/append/blank-doc N/A)
- goop_blocker: 1 generic + 3 tool-specific shapes (open, resolve, list,
  items[]; patch/append/blank-doc N/A)

Test-only; no production code changed. Inapplicable shapes documented
inline with structural rationale. SPEC: MH9, MH11.
… helper (W4.T2)

Replace presence-based isPatchActive/resolveContentForWrite branching with
a pure resolveWriteMode() resolver (src/shared/write-mode.ts) that decides
full-write/append, patch, or batch intent — and rejects four ambiguous
shapes before any mutation:

1. meaningful content + old_string together
2. new_string/replace_all without old_string (previously silently dropped)
3. mode: "append" combined with patch fields
4. items[] combined with a top-level operation field

Empty-string content stays a documented neutral placeholder when combined
with items[] (matches existing batch fixtures); old_string presence alone
(even "") still activates the blank-document patch workaround. Applied to
both the single-call and per-item batch paths, so an ambiguous item now
rolls back the whole batch transaction instead of silently mismatching.

All W4.T1 pinned positive fixtures remain green. Adds full truth-table
coverage for the resolver plus rejection/mutation-safety tests on the tool.

SPEC: MH10, MH9
…bling tools (W4.T3)

- goop_write_section: reuse resolveWriteMode() for both the single-section
  and per-item batch paths, since it shares goop_write_db's content/
  old_string/new_string/replace_all shape (no append mode). content +
  old_string together and items[] + meaningful top-level operation fields
  are now rejected before mutation instead of old_string silently winning.
- goop_save_note: add a note-specific resolveNoteMode() mirroring
  resolveWriteMode()'s intent for a CREATE-or-PATCH-keyed-on-note_id shape
  that doesn't map onto the generic content/old_string fields. Rejects
  note_id + meaningful create fields together, and patch-only fields
  (old_string/new_string/replace_all) supplied without note_id, in both
  the single and batch paths.
- goop_write_wave: audited against the one-shot standard; already the
  reference implementation. Added a missing regression test for
  task_update + task_updates[] supplied together (already guarded by the
  existing ignoredFields mechanism, previously untested for this pair).
  No production change.
- goop_blocker: audited and found the analogous defect — items[] silently
  ignored meaningful top-level action/description/severity/wave_id/id/
  resolution/status. Added the same ignoredFields rejection used by
  goop_write_wave (workflow_id excluded: it is genuinely consumed as the
  per-item default).

All W4.T1 positive fixtures remain green, including empty-content/
section_key/note-field neutral batch placeholders.
…y-loop guarantee

Lock the V2 JSON Schema boundary (convertToolArgsToJsonSchema) for every
createTools() entry: assert properties exist and `required` equals an explicit
intended set per tool. Catches a new tool added without a contract entry and
any drift in requiredness — the exact defect class that let optional patch
fields leak into `required` and turn full-document writes into silent patches.

Add a retry-loop regression proving the W4.T2 runtime validation returns a
single actionable error (naming conflicting fields and valid shapes), performs
no mutation, and differs from the prior silent-patch success so an agent
cannot loop on the identical payload.

Host-side V1 conversion remains outside plugin control (opencode #20807,
#13618); documented in-file. Runtime validation is the real safety guarantee.
hffmnnj and others added 7 commits August 3, 2026 13:05
… (W5.T1)

Single integration test file wires the seams that unit tests already
cover in isolation through their real production entry points:
- goop-wave-verifier registration via the V1 config hook and the V2
  registerHooksV2 agent/catalog transform adapter (shared AGENT_ROLES
  path)
- the real merged tool.execute.before guard chain (createHooks +
  DEFAULT_HOOK_FACTORIES) blocking/permitting goop-verifier and
  goop-wave-verifier by phase
- goop_write_wave's wave-completion verification gate: zero rows and a
  failing row both block completion; a later pass row for the same
  check_name supersedes it (append-only, latest-per-check)
- the real auto-progression hook advancing execute -> accept once the
  final wave carries verification evidence
- goop_acceptance_audit surfacing the append-only fail+pass history

Covers MH2, MH4, MH5, MH7.

Verification: rtk proxy bun test packages/opencode-plugin/ --bail=3
--timeout=10000 (2606 pass, 0 fail); rtk proxy bun run --cwd
packages/opencode-plugin typecheck (clean); focused biome check on the
new file (clean).
… test

Wave 5 Task 3 wiring/quality-gate check: lint diff against the Wave 4
base (1a9ed89) isolated exactly one net-new regression introduced by
this workflow — a formatter-only violation in
stage-gating.integration.test.ts (added in W5.T1, a4b0eb1). Applied
`biome format --write` scoped to that single file only; whitespace-only,
behavior-preserving. Post-fix lint diagnostics (10 errors / 17 warnings)
now match the Wave 4 baseline file-and-rule set exactly, confirming no
lint regression remains from this workflow. Remaining baseline errors
belong to files this workflow never touched (prompt-audit/report.ts,
continuation-prompt.test.ts, system-transform.test.ts) or were already
present at the Wave 4 boundary, and are out of this task's scope.
…config

Convert tab indentation to Biome's configured 2-space style and collapse
signatures/calls that fit within lineWidth:100 in
verifier-stage.ts and verifier-stage.test.ts. Formatting-only; no token,
type, test, or behavior change. Removes the two workflow-owned lint
diagnostics blocking the acceptance gate; unrelated baseline lint debt in
other modules remains classified as skip.
test(plugin): prove verification lifecycle end to end
fix(plugin): harden write tool payload contracts
@hffmnnj
hffmnnj merged commit 4378dcb into feat/stage-verification-enforcement Aug 3, 2026
@hffmnnj
hffmnnj deleted the feat/execute-verification-loop branch August 3, 2026 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant