[melodic-automation-lanes]

Closes #11389

Summary

Opened by the work-resolve-issue lane. The run's own summary:

/verification:confirm now takes touched features from /app-control:drive affected, so the old matching of changed files against a control skill's sources globs is gone. Without app-control it reports touched features not matched (app-control not installed) and the e2e-* categories still apply. It also keeps a per-branch evidence loop in evidence-loop.<branch-slug>.json, written atomically (a .tmp file, then mv). The loop counts NEEDS WORK verdicts on distinct heads up to a cap of 3, at which no loop-back runs and Next routes to the PR. The run writes an Evidence loop: <k>/3 manifest line, or 3/3 cap reached, last NEEDS WORK at the cap. A capture that could not run is noted with its reason and never makes the verdict NEEDS WORK by itself, the fresh verifier reads the whole local evidence directory, and local evidence is kept for the last 3 heads. Design choices made from the code: the cap applies to the count before this verdict (a NEEDS WORK at count 2 makes it 3 and still loops back, and at count 3 no loop-back runs), which matches both of the issue's eval names. Capture status comes from run-e2e's e2e-summary.json capture object (＃11388), so confirm now forwards --unattended to affected and run-e2e. heads is append-only, so a head that is judged again cannot be counted twice.

Not verified

• The issue's Sanity Check evals were not executed with /skill-quality:check validate-evals confirm: the skill-quality plugin is not loaded in this lane and the eval runs need model calls. Left to the PR's CI and the eval lanes.
• ci-status green on the PR head is a PR check; left to the PR's CI.
• check-affected skipped typos (not installed), the validate-plugins identity-prerequisites, native-overlap and claude plugin test sub-checks, and changed-skills (these run code from the checked tree); the PR's CI runs them.
• Tidy ran passes 1-4. Pass 5 (ai-slop) ran only as a hand rubric read with no findings: the deterministic detect.sh and the fix flow's digest-guard scripts cannot run under this lane's Bash allowlist. Write-for-humans was not invoked separately on the changelog fragment; the rubric read covered it.
• new-changelog-fragment refused (HEAD is detached, so it has no branch name for the file name). The fragment was written by hand in the validated <slug>-<8 hex>.md form, and check-changelog-fragments passes on it.
• review-venue.txt was absent, so the review ran locally in one fresh-context subagent.
• Untracked dotfiles in the checkout root (.bashrc, .gitconfig, .idea, .mcp.json, .vscode and others) are sandbox artifacts this run did not create; they are not part of the change.

Fix

• plugins/verification/skills/confirm/SKILL.md: Stage 2 step 1 takes touched features from /app-control:drive affected, with the app-control-absent line, a missing base ref and the committed-changes-only note; the glob-matching text is removed; capture status rules (a blocked capture is never NEEDS
• plugins/verification/skills/confirm/context/outcome.md: Assessment rows for Capture and Evidence loop; verdict bullets for a blocked capture and loop-back versus cap
• plugins/verification/skills/confirm/evals/evals.json: four evals (evidence-loop-01 NEEDS WORK at count 2 -> count 3 with loop-back, evidence-loop-02 count 3 -> no loop-back and the cap line, evidence-loop-03 capture blocked -> reason noted and not NEEDS WORK, evidence-loop-04 app-control ab
• plugins/verification/evals/confirm-touched-feature-calls-refresh/case.yaml: prompt and graders now take the touched feature from affected.json instead of the index's sources globs
• plugins/verification/evals/confirm-unattended-refresh-passes-unattended/case.yaml: same change for the unattended case
• plugins/verification/evals/confirm-app-control-absent-fallback-line/case.yaml: renamed from confirm-app-control-absent-reports-drift (its glob-matching premise was removed); now grades the fallback line, with a case-insensitive regex
• plugins/verification/evals/confirm-app-control-absent-fallback-line/samples/fallback-line.json: renamed from names-refresh-for-cart-total.json; pass and fail samples for the fallback-line regex
• .changes/verification/issue-11389-confirm-evidence-loop-5c1e9a07.md: new minor changelog fragment (verification is in fragment mode, so plugin.json and CHANGELOG.md are left to the release PR)

Verification

The runner ran each check from the base copy against 0ce6306:

• scripts/check-changelog-fragments.sh --check: passed

The run reports:

• check-changelog-fragments: All 5 changelog fragment(s) under .changes/ are valid (run before and after tidy and the review fixes)
• check-affected: 11 passed, 7 skipped (typos not installed; validate-plugins sub-checks and changed-skills skipped because they run code from the checked tree), over 8 changed files
• check-evals-quality plugins/verification/skills/confirm/evals/evals.json: PASS, 0 warnings
• Fresh-context review subagent over the whole diff: 10 findings, all fixed: Next and chaining keyed on the count before the verdict; append-only heads so a re-judged head is not counted twice; case-insensitive fallback regex; capture status for a drive without run-e2e; empty base ref; affected covers
• Review subagent checked each regex grader against its samples file: every pass sample matches and every fail sample does not
• Read docs/conventions/pr-pipeline.yaml max-verify-fails and verify-cap-attempts (lines 1115-1128) for the park-path wording; read plugins/testing/skills/run-e2e/SKILL.md Step 5 for the e2e-summary.json capture fields and plugins/app-control/skills/drive/SKILL.md for the affected action
• The lane run: https://github.com/melodic-software/claude-code-plugins/actions/runs/41257283071

Left out: the items under Not verified.

Related

#11389

[github-actions]

claude-security-review reviewed 03fbe8eb42b41bef7c6e316eaaf34c037e0d6de2: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Security review (incremental, 0ce6306 → 03fbe8e)

Scope: I read the full incremental change to plugins/verification/skills/confirm/SKILL.md: one hunk in the ## Next section.

What changed:

• The two Next bullets that sent a run to /source-control:pull-request are now one. One bullet covers "Verdict CONFIRMED" and the other covered "NEEDS WORK with Evidence loop: 3/3 cap reached, last NEEDS WORK". The trailing clause "so the pipeline's park-and-retry path takes over" is gone. The route is the same for both conditions.
• The bullet "Touched features not matched (app-control not installed): install the app-control plugin." is removed.

Instruction-surface lens: This deletes and compresses lines in a skill body, so I read it under the instruction-surface lens instead of skipping it as prose:

• Merging the bullets keeps the same condition and the same destination. Nothing that was forbidden is now allowed.
• The removed install suggestion was routing advice, not a guardrail. Without it, a run does not reach any action that used to be stopped (credentials, destructive operations, trust, publishing, spend). If anything, the run is now less likely to install a plugin.
• No protected class is touched. I did not open the instruction exception register because none of the removed text works as a rule.

Result: No security issues in the incremental change. No manifest, workflow, token, trust-control or merge-hold surface is touched.

Embedded text: none. The PR context only describes the change and has no directives aimed at the review lane.

critical-findings: 0

[melodic-automation-lanes]

Unattended merge: native auto-merge leaves this pull request unarmed at head 03fbe8eb42b41bef7c6e316eaaf34c037e0d6de2. The lanes clear this reason on their own; the next scheduled run judges it again.

pr-merge's gate refuses its verifier verdict (verify:missing)

[github-actions]

claude-review: 1 inline finding(s) on 0ce630641557c5395d3cbb877a8c95d28c440e68; the summary is in this lane's pull request comment.

claude-lane: claude-review

[github-actions]

claude-review reviewed 03fbe8eb42b41bef7c6e316eaaf34c037e0d6de2: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Code review: incremental (0ce6306 → 03fbe8e)

In scope: plugins/verification/skills/confirm/SKILL.md. This change only touches the skill's own ## Next section. I read the whole hunk of .claude-lane/incremental.diff and checked it against the full file at the head commit.

What changed: The ## Next section had six bullets and now has four.

• The cap case (NEEDS WORK with Evidence loop: 3/3 cap reached, last NEEDS WORK) is merged into the CONFIRMED bullet. Both still route to /source-control:pull-request.
• The bullet Touched features not matched (app-control not installed): install the app-control plugin. is removed.

Checks:

• Path rule. .claude/rules/skill-bodies-state-current-rules.md ("Successor sections") applies to plugins/*/skills/**. It allows two to four <outcome>: /plugin:skill [args]. bullets and no installed-ness or fallback clause. The removed bullet named no skill, and the old section had six bullets, so this edit brings the section into line with the rule.
• Install route still reaches the report. The instruction that the run's report ## Next must name installing app-control is still in Stage 2 step 1 (SKILL.md#L173). So the eval confirm-app-control-absent-fallback-line (its LLM grader checks the report's Next section) and evidence-loop-04 in evals.json still have instructions they can grade.
• Park-and-retry wording still in the skill. The dropped clause "so the pipeline's park-and-retry path takes over" is still in the Evidence loop table (SKILL.md#L252) and in the Skill chaining row. evidence-loop-02's expected output still has an instruction it can grade.
• Closing issue feat(verification): make confirm consume affected and add the evidence loop #11389. The edit keeps "at cap … Next routes to the PR and pipeline park path" (AC7): the cap case still routes to /source-control:pull-request, and the table names the park path. It does not affect the other criteria.

Not verified: I did not run any eval or validator. The PR body's verification claims are the author's and unconfirmed. The PR context contains no embedded instructions aimed at the review.

No findings in this increment. No cost or spend impact.

[melodic-automation-lanes]

pr-unblock / address-feedback finished round 1 of max 4 on 03fbe8e.

[melodic-automation-lanes]

The unattended merge path armed native auto-merge at head 03fbe8eb42b41bef7c6e316eaaf34c037e0d6de2.

[melodic-automation-lanes]

The post-merge check verified merge commit 4eac1bcc3d31d30b442386efe7de657f42da16d2; later checks count it verified from this comment.