[melodic-automation-lanes]

Closes #12190

Summary

Opened by the work-resolve-issue lane. The run's own summary:

run-validity.py has a new 'plugins loaded' check. It reads each with-arm run's trace init line and fails the run when plugin_errors has an entry for a suite.plugins plugin, or when the plugins list leaves one out. The INVALID reason names the plugin and the error type (for example 'playgrounds (dependency-unsatisfied)'), and it also lists load errors filed under another name, such as a missing dependency. Each suite plugin's name comes from its .claude-plugin/plugin.json, falling back to the directory name. A trace entry matches by bare name, namemarketplace, or path. Field names follow the Agent SDK SDKSystemMessage type. A with-arm run whose init line has no plugins list is a WARN, not UNCHECKED: existing fixtures and older traces have no plugins list, and the issue's criteria allow either.

Not verified

• test_run_validity.py / run-validity.test.sh not executed: this lane's Bash runs only the named wrappers, so no python; left to the PR's CI. The fail-before-fix property was reasoned (the pre-fix script has no such check and returns VALID), not observed
• The init line's exact plugin naming for a --plugin-dir plugin in a claude plugin eval sandbox (bare name, nameinline, or a sandbox path) was not observed in a real trace; the check accepts bare name, name<anything> or an exact path. A smoke run on a real eval trace would confirm it
• ai-slop pass 5 ran as a rubric read only: the detect.sh detector is not runnable in this lane; no tells found in the 4 doc lines or the fragment
• review-venue.txt absent, so the review ran locally in one fresh-context subagent

Fix

• plugins/evals/skills/plugin-eval/scripts/run-validity.py: new 'plugins loaded' check (suite_plugins, init_line, names_plugin, load_error, check_plugins_loaded), run after 'session started' in main(); header docstring row; init-line field constants; the traces line notes the new check warns for unres
• plugins/evals/skills/plugin-eval/scripts/test_run_validity.py: plugin_load fixture helper plus 5 tests: plugin_errors dependency-unsatisfied with the plugin not loaded (INVALID, reason names plugin and error), plugin missing with no error, an empty list or a dependency's error (INVALID), plugin load
• plugins/evals/skills/plugin-eval/reference/reading-results.md: bullet describing the plugins loaded check
• .changes/evals/work-resolve-issue-issue-12190-41369641496-1-888488f5.md: patch changelog fragment under ＃＃＃ Fixed

Verification

The runner ran each check from the base copy against e4aa281:

• scripts/check-changelog-fragments.sh --check: passed
• scripts/check-changed-skills.sh --merge-base: passed

The run reports:

• read https: //code.claude.com/docs/en/agent-sdk/typescript: SDKSystemMessage init carries plugins: {name, path}[] and plugin_errors?: {plugin, type, message, path?}[]; plugin_errors lists load failures and is omitted when nothing failed; type is an open set
• .lane/checks/bin/check-changelog-fragments: all 9 fragments valid (after adding the required ＃＃＃ section)
• .lane/checks/bin/check-changed-skills: plugin-eval PASS, 0 errors, 5 pre-existing size warnings
• .lane/checks/bin/check-affected: 12 passed, 6 skipped (skips are the checks that run code from the checked tree, and typos is not installed)
• fresh-context review subagent: traced each new test by hand against the script and r2-clean-runs.json and found every assertion would pass; found no existing test broken (suites that set suite.plugins without a plugins list now WARN, and no exact verdict assertion covers them); 2 wording nits fixed
• The lane run: https://github.com/melodic-software/claude-code-plugins/actions/runs/41369641496

Left out: the items under Not verified.

Related

#12190

[github-actions]

claude-security-review reviewed 14e83ef291465ac0a6ed6b64ebbab6f57d9d248b: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #12206 (incremental, 911f768 → 14e83ef)

No security issues found.

Scope. I read the one hunk in .claude-lane/incremental.diff, in plugins/evals/skills/plugin-eval/scripts/run-validity.py. The other three files in this PR haven't changed since 911f768, which this lane already reviewed with no findings.

What changed. Only a comment block above the TRACE_PLUGINS / TRACE_PLUGIN_ERRORS constants (lines 230–237). The old three-line note on the Agent SDK SDKSystemMessage init fields is now a prose description plus a Pointer / As-of / Recheck-trigger record pointing at the Agent SDK TypeScript docs URL. The constants ("plugins", "name", "path", "plugin_errors", "plugin", "type", "message", "path") and all executable code are unchanged.

What I checked:

• Behaviour: none changed. The diff adds no code, I/O, subprocess call or network call. The docs URL sits in a comment, and nothing fetches it.
• Instruction surfaces: the edit is a code comment in an eval script, not a standing instruction, so nothing is deleted or softened.
• Controls: the file matches no trusted-path glob. It touches no trust control, merge hold, kill switch, token scope or plugin manifest.
• PR context: the description has no directives aimed at this lane.
• Prior findings: the lane's one earlier post (round 0 of 3) had no findings, so nothing repeats. The round cap does not apply.

critical-findings: 0

[github-actions]

claude-review: 1 inline finding(s) on e4aa281df99c9da8cbaf49f91218b2ed8054c95c; the summary is in this lane's pull request comment.

claude-lane: claude-review

[github-actions]

claude-review reviewed 14e83ef291465ac0a6ed6b64ebbab6f57d9d248b: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Code review: PR #12206, incremental since 911f7683

Scope. This round covers only the change since 911f7683: one hunk in plugins/evals/skills/plugin-eval/scripts/run-validity.py (L230-L237), which rewrites the comment above the init-line field constants. I read that whole hunk and checked it against the code that reads those constants (names_plugin, check_plugins_loaded, L795-L844). The other three files are unchanged since the last round and I did not re-review them.

Prior finding resolved. The advisory finding in discussion_r4438148976 is fixed. The owner asked for this fix in the thread.

• The comment no longer repeats the upstream SDKSystemMessage field shapes or the "absent when no plugin failed" claim.
• It now states the check's own choices in its own words: a missing failure list counts as none failed, and a loaded name matches with or without an @marketplace suffix.
• It adds the record .claude/rules/skill-bodies-state-current-rules.md asks for:
  • a pointer to the anchored docs section (https://code.claude.com/docs/en/agent-sdk/typescript#sdksystemmessage)
  • an as-of date (2026-11-01)
  • a recheck trigger (the init message renames or drops plugins/plugin_errors, or renames an entry field)
• What the comment says matches the code: L823 defaults plugin_errors through as_list(init.get(...)), and L799 accepts name or name@….

Closing issue #12190. Last round I judged both criteria met. This hunk changes only a comment, so that does not change. I have not run the tests (this lane has no shell), so whether they pass is still unverified.

Other checks

• Embedded instructions: none found in the PR context, settled-threads or prior-findings files.
• Plugin manifests: no plugin.json or marketplace.json changed.
• Paved paths: nothing applies.
• Round cap: not reached (2 of 3 rounds with findings before this one).
• Hold: none applies.

No new findings.

[melodic-automation-lanes]

The unattended merge path armed native auto-merge at head e4aa281df99c9da8cbaf49f91218b2ed8054c95c.

[melodic-automation-lanes]

The pr-unblock lane stopped address-feedback on this pull request: the pull request has an unresolved review thread the lane does not take (a reviewer it does not act on, a review of an older head, or a thread it already answered), so no lane resolves it and a session builder takes it. The lane labels it needs-session for a session builder, who makes the change this lane cannot, and the run ends in a failing pr-unblock/address-feedback status on the head the lane gated.

• Reason: feedback-untaken-threads
• Head: 911f7683c9d5459c557293037938a546112fd730
• Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/41381723197

When a retry on this head can change the outcome, the lane runs one on its own, up to its per-head cap, and edits this comment if it stops again; a run that does not stop clears the status. When no retry can (a review thread the lane does not take or settle, or a failing check after the retries are spent), a person resolves the threads or gets the checks green, then dispatches the workflow below: that run finds the cause gone and clears the status. On a pull request the lanes App opened, the dispatchers start that run once per stop; on any other, a person does. A new commit to the branch starts clean.

To run it again on this head now, dispatch the pr-unblock workflow:

gh workflow run pr-unblock.yml -R melodic-software/claude-code-plugins -f pr-number=12206 -f activity=address-feedback

[melodic-automation-lanes]

Unattended merge: native auto-merge leaves this pull request unarmed at head 14e83ef291465ac0a6ed6b64ebbab6f57d9d248b. The lanes clear this reason on their own; the next scheduled run judges it again.

pr-merge's gate refuses its verifier verdict (verify:missing)

[melodic-automation-lanes]

pr-unblock / address-feedback finished round 1 of max 4 on 14e83ef.

[melodic-automation-lanes]

The unattended merge path armed native auto-merge at head 14e83ef291465ac0a6ed6b64ebbab6f57d9d248b.