fix(repo-fleet-hygiene): canonical resolution, substitution rungs, and report accuracy - #1861
Conversation
…ore both substitution rungs, and cover the real failures
Six defects found by auditing the plugin against one real 11-repository
fleet, plus the behavioural coverage that would have caught them.
Canonical resolution could select a linked worktree, because discovery
reaches a worktree and its own main worktree through the same glob and
both map to one --git-common-dir dedup key, so glob order decided the
winner. Every emitted handoff carries that path. add_target now resolves
each candidate to its main worktree via the worktree porcelain before
the tie-break.
The allowed-tools grant named ${CLAUDE_PLUGIN_ROOT}, which is not among
the variables substituted in allowed-tools or skill content, so the
skill's one permission grant was inert. The collector also read
CLAUDE_PROJECT_DIR from an environment that never provides it, leaving
the project config rung unreachable and the no-scope fallback silently
becoming $PWD. Both now use the documented substitution points.
Report text no longer asserts a fixed scope provenance the run
contradicts, no longer prints a hardcoded mutation tally, and presents
Windows paths in a form Windows tools accept. setup no longer prescribes
a full fleet walk as its config-verification step and can set maxDepth.
The tier table covers every emitted kind, with a test asserting set
equality so the drift cannot recur.
Fixes #1797
Fixes #1798
Fixes #1799
Fixes #1800
Fixes #1803
Refs #1801
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ment --project-dir Retargeting a supplied or discovered path to the repository of record is a substitution the operator did not ask for. Reporting only the resolved path would repeat the defect class this change fixes elsewhere -- report text that does not match the run's own inputs -- so the header now states it on a `Resolved to main worktree:` line, with an assertion pinning it. `--project-dir` was parsed and documented in the skill but missing from the script's own usage output. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ocuments it MERGED_PR_WINDOW landed between the gh-timeout comment and the constants it describes, and the retarget block carried two stacked comments for one statement. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
… and reject empty scope values Two reporting-accuracy defects in the header this change rewrote. Several linked worktrees of ONE repository each produced their own `Resolved to main worktree:` line, so a repository the count above reports once read as N repositories. The disclosure is now one line per repository of record, naming every path that resolved into it. An empty `--root`/`--repo`/`--config` value was counted toward the scope the header reports and then skipped by the discovery loops, overstating the count. It now stops the run, like every other invalid CLI input. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 7m 16s —— View job Code review — PR #1861Reviewed against This is a well-documented, carefully self-verified fix set — every claim in the PR body checks out against the diff (the #1797 canonical-resolution fix reuses the pre-existing Two suggestions, nothing blocking: 🟡 Suggestion —
|
|
Claude finished @kyle-sexton's task in 3m 36s —— View job Security review
Reviewed all 9 changed files (
1 finding — the [IMPORTANT / confidence: needs verification]
Before this change the rule contained a substitution ( The open question — which the PR body itself flags under "Unverified" and defers to This repo's own custom instructions call out No other issues found — no command/path injection, no auth/authorization gaps, and no unsafe handling of credentials in the reviewed diff. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 041595ddb1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…tup evals The new setup eval prompts illustrated cross-volume and fleet-root scenarios with literal `C:/Users/...` and `D:/repos` paths, which the machine-specific- paths hygiene gate rejects. The scenarios need two volumes, not two specific volumes, so the prompts now use `<home>` / `<fleet-root>` placeholders and say in prose that the config and root sit on different Windows volumes. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude has reviewed this PR 3 times. The lane skips further automatic reviews after 5; deleting this comment resets the count. |
|
Claude finished @kyle-sexton's task in 1m 5s —— View job Security review
This is a re-run triggered by a new commit on the branch. The only change since my prior security review (posted against commit My findings from the prior pass stand: No new vulnerabilities in this diff. 1 finding carried forward (unchanged by the new commit) — flagging again since it's still merge-relevant: [IMPORTANT / confidence: needs verification]
Before this change, the rule contained a substitution ( The open question (which the PR body itself flags under "Unverified" and defers to No other issues found — no command/path injection, no auth/authorization gaps, no unsafe credential handling, and no workflow-permission changes in this PR (no |
|
Claude finished @kyle-sexton's task in 2m 37s —— View job Code review — PR #1861 (re-review after push)
This push adds one commit on top of what was already fully reviewed: What it does: the three Verified:
Status of earlier feedback on this PR: unchanged by this commit, since it only touches the setup evals file.
No new issues found in this commit. Nothing here blocks merge on my end; the one still-open item worth resolving before merge is the security lane's |
…allowed-tools
The doc stated that ${CLAUDE_PROJECT_DIR} is the ONE substitution expanded in
allowed-tools. The skills page documents two — ${CLAUDE_SKILL_DIR} (v2.1.129+)
and ${CLAUDE_PROJECT_DIR} (v2.1.196+) — in both skill content and allowed-tools
Bash rules, and the SKILL_DIR floor is the earlier of the two.
The stale claim is load-bearing: it is the reason a skill's own bundled-script
grant looks unfixable, and it produced a false review finding against the
correct fix. Also names ${CLAUDE_PLUGIN_ROOT} as excluded, since that is the
token skills actually reach for and it leaves the grant inert.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 1m 44s —— View job Code review — PR #1861 (re-review after push)
This push adds one commit on top of what was already fully reviewed: What it does: rewrites the anti-pattern-2 paragraph that previously claimed Verified against the live docs, per this repo's fresh-docs mandate (this is a documented-harness-behavior change): fetched Consistency check across the rest of the file: the pre-existing "Known gap" section ( Resolves the open review thread: Codex's comment on No new issues found in this commit. Prior open items (the two 🟡 suggestions on |
|
Claude finished @kyle-sexton's task in 1m 0s —— View job Security review — PR #1861 (re-run after push, commit
|
… re-probe Work-in-progress save before machine wind-down. Addresses the independent reviewer's CRITICAL on PR #1861: canonical retarget must re-probe main_worktree()'s answer with rev-parse --show-toplevel and adopt it only when it resolves to a working tree different from the current top (submodule / --separate-git-dir / bare-worktree shapes otherwise adopt a Git admin directory). Gate sweep and remaining regression fixtures NOT yet run — see the PR comment for exact state. Refs #1797 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RhS3T7ShwJgKTrvk2Mvd3C
|
Session wind-down state marker (autonomous lane stopped by operator): Commit c1b3265 is a WORK-IN-PROGRESS save of the fix for the independent reviewer's merge-blocking CRITICAL (canonical retarget adopting a Git admin directory for submodule / --separate-git-dir / bare-worktree shapes). The re-probe approach follows the reviewer's verified fix: Remaining before this PR is merge-ready:
A cold agent can resume from this comment plus the reviewer's findings (recorded in the session that produced this PR); the worktree is claude-code-plugins-issue-1797 on this machine but everything needed is in this PR and the six linked issues. |
|
Claude encountered an error after 8s —— View job I'll analyze this and get back to you. |
|
Claude encountered an error after 1s —— View job I'll analyze this and get back to you. |
|
Warning Automated review did not complete — this is an infrastructure failure, not a review verdict. Treat any Claude comment on this PR (including a placeholder like "I'll analyze this and get back to you") as incomplete, not "no findings."
Re-run the job to retry the review. A new push does not re-trigger this lane. |
|
Claude finished @kyle-sexton's task in 1m 48s —— View job Security review — PR #1861 (re-run after push, commit
|
Summary
Six defects found by auditing
repo-fleet-hygieneagainst one real 11-repository fleet, plus thebehavioural coverage that would have caught them. Every claim below was re-verified against
origin/mainbefore fixing; one turned out to be already fixed and is called out as such.#1797 — canonical resolution could select a linked worktree. Recursive discovery reaches a linked
worktree and its own main worktree through the same glob (a linked worktree carries
.gitas afile, so it matches), and both map to one
--git-common-dirdedup key — so the winner was decidedby glob order. A sibling sorting before the canonical directory under
LC_ALL=Csilently becameCanonical:, and every emitted handoff carries that path, so per-repository cleanup would be aimedat a checkout that is not the repository of record while the true canonical checkout was
deduplicated away and never reported.
add_targetnow resolves each candidate to its main worktree —the first record of
git worktree list --porcelain, which lists the main worktree first wherever itruns — before the dedup tie-break.
rev-parse --show-toplevelcannot make this distinction: inside alinked worktree it returns the linked root, which is why evidence rule 1 as written could not
distinguish them either. The extra probe is gated on
.gitbeing a file rather than a directory, soan ordinary sweep pays nothing per repository. The substitution is disclosed rather than applied
silently — the operator named one path and the report is about another, which would otherwise be the
same defect class as the header text fixed under #1800 — on one
Resolved to main worktree:line perrepository naming every path that resolved into it, so several worktrees of one repository cannot
read as several repositories against the discovered count.
#1798 — two variable substitutions used where the documented contract does not provide them.
Verified against the current skills documentation fetched this session
(https://code.claude.com/docs/en/skills#available-string-substitutions): the substitution table
lists
${CLAUDE_SKILL_DIR}and${CLAUDE_PROJECT_DIR}, and the page states Claude Code substitutesthose two "in two places: the skill's markdown content, and Bash rules in the
allowed-toolsfrontmatter".
${CLAUDE_PLUGIN_ROOT}is not among them, so theallowed-toolsgrant stayed aliteral string and the skill's one permission grant was inert in a workflow built for unattended
sweeps; it now uses
${CLAUDE_SKILL_DIR}. Separately the collector readCLAUDE_PROJECT_DIRfromits own environment, which does not carry it — leaving the project rung of the config ladder
unreachable and the zero-argument fallback silently becoming
$PWD, an agent session's incidentalworking directory. The skill body now passes
--project-dir "${CLAUDE_PROJECT_DIR}", where thedocumented substitution applies; the environment variable is still honored for callers that have it,
and a value left as an unexpanded
${...}placeholder counts as absent rather than as a literaldirectory name. When no project directory resolves, the run now stops with the scope remedies
instead of auditing whatever directory the shell was sitting in.
#1799 — confidence model and evidence rules had drifted from the collector. The tier table
covered 12 of 24 emitted kinds, so a consumer meeting an untabulated kind had no documented
disposition. It now covers all 25, and a test asserts set equality in both directions between
the table and the collector's emitted kinds — a drift detector rather than a generator, so this
cannot silently recur.
ACKNOWLEDGEDis documented as a prominence demotion of anUNKNOWN, not afifth value on the confidence axis. Evidence rule 3 now describes the mechanism that actually runs
(one batched query per repository plus a privacy-gated per-branch fallback) rather than a per-branch
authoritative query. Two undisclosed dependencies are stated: the
LOWancestry tier is near-inertunder squash merges, and
missing-worktreevsprunable-worktreeturns on the user-tunablegc.worktreePruneExpirewindow rather than on evidence strength. The two reference files with nopointer from the hub are now linked, and the two differently-scoped
repositorieslabels arequalified.
#1800 — report text asserting things the run contradicts. The header printed a fixed
Config: none (… current-project scope)literal that could contradict the very next lines anddescribed a mode that was not reachable at all; a computed
Scope:line now names each contributingrung and its entry count, which also discloses that config-supplied scope is additive to
CLI-supplied scope.
Mutation count: 0was a hardcoded literal that would have read identically ina build that mutated — replaced by a statement of the enforcing mechanism (the read-only git/gh
command allowlists), because a real counter would undercount: most probes run inside command
substitution, so increments are lost with the subshell. MSYS
/c/...paths are converted toC:/...for presentation only, since the report is actionable text whose paths get pasted intotools that reject the MSYS form; every comparison and filesystem test keeps the native form. An empty
--root/--repo/--configvalue now stops the run rather than being counted toward the scope theheader reports and then skipped by the discovery loops.
Item 4 of #1800 (the hard-failure path naming no remedy) is already fixed on
origin/mainby the#1771 work — the rejection names
--root,--repo,--config, and/repo-fleet-hygiene:setup apply, andSKILL.mddiscloses the CLI hard-fail semantics. Verified via the existing passingassertions; no code was needed.
#1801 —
setupdefects (partial; see Related).applystep 5 prescribed running the collector,which is the full fleet walk the skill states it never performs — minutes of per-repository network
queries in a step described as validating that a config parses. Verification is now config-only, with
an end-to-end run offered as an explicit handoff.
--max-depthis now settable through the skillthat owns the file carrying it (and was missing from the audit skill's
argument-hint). Thecross-volume exception to the relative-path preference is stated, the determinism claim is softened
to guidance since no deterministic component backs it, and the project-vs-user-global gotcha now
names the two cases that collapse it.
#1803 — evals missed every behaviour that actually failed. Split by what each surface can
actually test rather than routing everything into
evals.json, which would have reproduced the gapbeing filed. Deterministic fixtures went to
audit-fleet.test.sh: canonical selection against anearlier-sorting linked worktree (written red first — it failed on both canonical assertions before
the fix), the computed scope-provenance line, merged-PR window truncation, unauthenticated-
ghdegradation, and tier-table drift. Model-graded rows went to
evals.json: privacy-gated branches asunverified rather than unmerged, squash-merge semantics, worktree disposability as deliberately out
of scope, and for
setupconfig-only verification, cross-volume paths, andmaxDepth. One evalpinned the boundary violation being fixed (it expected
applyto "run a read-only audit") and wascorrected.
This also required a behaviour change: a full merged-PR window now emits
merged-pr-window-truncated(
UNKNOWN). Previously a repository with more than 200 merged PRs lost the remainder silently, and abranch merged before the window produced no merged finding — indistinguishable in the report from a
branch that was never merged. It cannot distinguish "exactly 200" from "far more" and deliberately
errs toward warning, since an unfalsifiable reassurance was the #1800 sin and an over-cautious
warning is the correct direction to err.
Test plan
audit-fleet.test.sh— 71 assertions, all passing (16 new). The repo-fleet-hygiene:audit canonical resolution can select a linked worktree, pointing every handoff at the wrong path #1797 fixture was confirmed redbefore the fix (both canonical assertions failed; the linked worktree won) and green after.
scripts/check-changed-skills.sh origin/main— both skills PASS, 0 errors, 0 warnings.check-changelog-parity.sh--check,--check-order,--check-bump origin/main— all pass.markdownlint-cli2over the plugin — 0 issues in 7 files.ajv --spec=draft2020againstplugins/skill-quality/reference/evals.schema.json— bothevals.jsonfiles valid.shellcheckon both shell files — clean (three diagnostics introduced during the change werefixed, not suppressed).
check-skill-portability.sh,check-shell-portability.sh,validate-plugins.sh,validate-plugin-contracts.mjs,check-plugin-manifest-presence.sh,check-silent-skips.sh— pass.generate-catalog.mjs/generate-cheatsheet.mjs— already in sync, no regeneration needed.--repoat a linked worktreenow reports the main worktree;
/c/Usersrenders asC:/Users; a no-scope run with no projectdirectory stops with the remedies instead of silently auditing
$PWD.Review
Three fresh-context reviewer subagents were spawned and none returned findings, so a bespoke
rationale-withheld human-style review was not obtained. Three other independent passes did run:
${CLAUDE_SKILL_DIR}is not expanded inallowed-tools, citingdocs/conventions/permission-rule-hygiene/README.md. Investigated andrejected on primary-source evidence — the current skills page states Claude Code substitutes
${CLAUDE_SKILL_DIR}and${CLAUDE_PROJECT_DIR}in both skill content andallowed-toolsBashrules, with the SKILL_DIR floor (v2.1.129) the earlier of the two. The cited local doc was stale and
is corrected here; its wrong sentence is what made this grant look unfixable and is what produced
the false finding. Full citation in the review thread.
reviewandsecurity-reviewlanes — both pass.form) produced two real defects, both reproduced against live fixtures and fixed in
group the retarget disclosure per repository and reject empty scope values: several linkedworktrees of one repository each emitted their own
Resolved to main worktree:line, so onerepository read as several against the header's own count; and an empty
--root/--repovalue wascounted toward the reported scope then skipped by the discovery loops.
Self-verification worth naming: the #1797 fixture was written red first; the tier-table drift
detector was proven non-vacuous by deleting a row and confirming failure;
to_native_pathwasunit-tested against non-path values, slash-commands, branch paths and multi-occurrence input; the
placeholder-rejection pattern and the empty-array expansion idiom were tested directly; and the whole
change was exercised end-to-end against a real Windows fleet and a purpose-built two-worktree repo.
The highest-value remaining check is
main_worktree()'sread -r -d ''inside a commandsubstitution, and whether any new assertion is vacuous.
Related
Fixes #1797
Fixes #1798
Fixes #1799
Fixes #1800
Fixes #1803
Refs #1801 — items 1-5 delivered. Item 6 (a
--no-configescape hatch so a persisted user-globalconfig can be suppressed for one run) is deferred: it is a feature add rather than a defect fix, and
it carries an unresolved semantic question (whether it suppresses only the
fleet.root/fleet.repoladder or also
[canonical …]sections). The additive behaviour it complains about is now at leastdisclosed by the new
Scope:header line and asetupgotcha. #1801 stays open for it.