docs(claude-ops): re-verify plugins skill probes against Claude Code 2.1.261 - #3760
Merged
Merged
Conversation
…3734) Re-runs the empirical probes the plugins skill's recheck triggers arm, and updates each stamp to the version it was actually re-verified on. Confirmed unchanged on 2.1.261: update -s project writes no committed settings; the install/uninstall/enable/disable project-scope write matrix including key-created-when-absent and whole-file re-serialization; enable -s project gating on the merged effective value; -s project keying on the literal cwd; and no CLI verb reaping an install record by path. Corrected: a bare plugin name now resolves for claude plugin update, so the gotcha records the change and rests its rule on cross-marketplace ambiguity instead; the --ids refusal of --all is fleet-state.sh's own guard, not a Claude Code CLI fact, and the attribution is fixed rather than restamped. Not re-run, each marked in place with its reason and keeping its old stamp: the userConfig unset-key placeholder probe (attempted, inconclusive because the positive control failed), mid-session update path resolution and the /reload-plugins warning (both need an interactive session), and the prune and --force version gates (current docs state neither version). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019gWgHogJFQeCne5U7vHKAE
…3734) The gotchas header asserted that the CLI-behaviour sections were re-run, which overclaimed: the plugin list/details section and the spoke-substitution section were not part of the pass. It now names the re-run, changed, and not-re-run sections individually. The scope-semantics header stated a count of three not-re-run claims that its own body contradicted further down. The count is gone and the list is complete. The spoke-substitution gotcha now records that the same failed positive control weakened the SKILL.md half of the contrast it rests on, and the userConfig heading says "no enum type" rather than "no enum field", since the schema does use an enum keyword to constrain type. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019gWgHogJFQeCne5U7vHKAE
kyle-sexton
marked this pull request as ready for review
September 5, 2026 16:12
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Contributor
|
Claude finished @kyle-sexton's task in 2m 33s —— View job CI code review complete
This is a docs-only re-verification pass for the One finding posted inline:
Checked and found consistent (no issues):
Not independently re-run: the probe session itself (throwaway project under |
Contributor
|
Claude finished @kyle-sexton's task in 50s —— View job Security review —
|
Contributor
|
Last security-reviewed head: |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Contributor
|
Claude has reviewed this PR 1 time. The lane skips further automatic reviews after 5; deleting this comment resets the count. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3e04036585
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ion bump (#3734) The 2.1.261 re-run of the "update -s project does not write committed settings" probe was a no-op update: the plugin was already at its latest version, so nothing was written to installed_plugins.json and an empty git status did not re-verify the invariant sync relies on. Re-ran it against a throwaway local marketplace serving probe-plugin at 0.1.0, installed at project scope into a scratch repo, reverted the dirtied committed settings to clean, bumped the marketplace, and updated: the record advanced to 0.1.1 with a new lastUpdated and installPath while git status stayed empty. A second 0.1.1 -> 0.1.2 cycle hash-compared both settings.json and settings.local.json across the update. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019gWgHogJFQeCne5U7vHKAE
…dex (#3734) The header block enumerates which claims carry the 2.1.261 stamp. The update -s project settings exemption lives in its own section, so a reader auditing that index could not trace it to "the write matrix". Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019gWgHogJFQeCne5U7vHKAE
This was referenced Sep 5, 2026
kyle-sexton
added a commit
that referenced
this pull request
Sep 6, 2026
…mmit (#3775) Closes #3681 ## Summary `claude plugin update` re-points an install record's `gitCommitSha` in `installed_plugins.json` without rewriting the plugin's cache directory when the manifest version number is unchanged across the two commits, because the cache is keyed by version. The record then claims the new commit while the directory still holds the older build, and the version-and-sha check every delivery step relies on passes in exactly that state. On the reporting machine six plugins were in that state at once, twelve stale files in the worst case, including a reviewed hook dispatcher and two `hooks.json` files. The issue asked for a decision between (a) a repo rule that every plugin change bumps the version, (b) promoting the file-level compare into the claude-ops plugin-fleet audit as a standing check, or (c) an upstream report. This takes (b). (a) already exists as the CHANGELOG-parity gate and only bites when a branch is delivered without a bump, so it cannot catch the case that motivated the issue; (c) is worth filing but does nothing for machines today, and the updater's behaviour on an unchanged version may be intentional. (b) is local, cheap, and durable: every record carries the sha and the marketplace's `installLocation` is a git clone, so the files can be compared byte for byte. ## Fix - New `cache-content-check.sh` in the `plugins` skill: a read-only per-install compare of every file under a cache directory against the recorded commit in the marketplace clone. Two git processes per install (`git ls-tree -r <sha>` for the expected blob ids, one `git hash-object --stdin-paths` batch for the cache files), set difference plus hash inequality. Detects changed files, files the commit has that the cache lacks, and files deleted at the commit but still in the cache. Honours the marketplace's `.gitignore` (a live plugin root carries `__pycache__` and `node_modules`, which the first real run flagged as stale until this filter existed) and re-hashes mismatches with `--path` so `.gitattributes` CRLF rules do not produce false positives. Never writes: no `git fetch` (a sha absent from the clone reports `sha-not-local`), no `claude plugin` call. Same CR-safe `--ids` contract as `fleet-state.sh`, exit 2 on usage or malformed input. - `sync.md` gains Step 5b running the check in both `sync` and `audit` (from the run journal or scratch dir respectively), ungated, because an unchanged version number is precisely the case in which every other step reports success. SKILL.md's Report template gains a `Cache content:` row naming affected ids and the remediation the issue proved (remove that version directory under the cache and re-run `claude plugin update <id>`); the check never repairs. - `scope-semantics.md` records the mechanism (stamped Claude Code 2.1.259, issue evidence, not re-run) and the shallow-clone fact observed on 2.1.261: the marketplace clone under `~/.claude/plugins/marketplaces/` carries `.git/shallow` with three commits, so records naming older commits are unverifiable without a fetch the audit refuses to perform. - claude-ops `0.42.12` -> `0.42.13` with CHANGELOG entry (main took 0.42.8 through 0.42.12 mid-flight). ## Verification - `cache-content-check.test.sh`: 24 cases, 0 failed. Fixture-driven with a throwaway git repo as `installLocation` at two commits: match; older file in cache -> stale-content naming the path; file deleted at sha still in cache -> stale-content; sha not in clone -> sha-not-local; `--ids` emits only stale ids CR-free; malformed `installed_plugins.json` -> exit 2; unknown marketplace -> exit 2 naming it; absent project path skipped and counted; misaligned hash batch refused rather than compared (stub git truncating the hash batch); plugin directory renamed between commits still matches because the manifest is read at the recorded sha; a pathspec matching nothing at the sha is `no-source-at-sha`, not stale; non-ASCII filename round-trips NUL-safe; tracked symlink hashed mode-aware (skips where the host cannot create one); record without gitCommitSha yields `no-git-commit-sha`; process-count probe has a floor so it cannot pass vacuously. - `fleet-state.test.sh`: 86 cases, 0 failed. `shellcheck -S info` clean on both new scripts. `typos` clean. `markdownlint-cli2`: 0 issues in 54 files. `validate-plugin-contracts.mjs`: pass. Test suites are auto-discovered by `scripts/run-plugin-tests.sh`, no registry edit needed. - Real machine, read-only, against the committed script: `--marketplace melodic-software --scope user` -> checked 74, match 63, stale_content 0, unverifiable 11 (all `sha-not-local`, the shallow-clone case above), skipped_absent_project_paths 0. - `scripts/affected-tests.sh --run` not run locally (hangs on `check-changed-skills.test.sh` on this host); CI is the statement of record. ## Related - #3688 (stale project-scope records; this check skips absent project paths and counts them rather than reaching into them) - #3763 (marketplace remove leaves a cache tree behind, found during #3734) - #3760 (the re-verification pass that stamped the no-reap-by-path claim this check respects) 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_019gWgHogJFQeCne5U7vHKAE --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Sep 6, 2026
…m and why nothing reaps them (#3786) No related issue: this records verified facts for the #3688 decision without closing it; the structural remediation stays open on that issue for the operator. ## Summary The `plugins` skill reports stale project-scope install records and states that no CLI verb removes one by path, but it never recorded where those records come from. A user reading a count in the hundreds had no way to tell a careless install habit from a repo producing them on its own. A review pass against the current Claude Code docs (plugins, plugins-reference, settings-reference, cloud-environments, discover-plugins, claude-directory, changelog through 2.1.261) established the mechanism; this PR writes it down where the skill reasons about those records. ## Fix - New `scope-semantics.md` subsection "Where project-scope records come from, and why the skill cannot reap them". A repo's committed `.claude/settings.json` `enabledPlugins` block is the documented cloud install mechanism; project scope outranks user scope in `enabledPlugins` precedence; so every project-scope `true` that duplicates a user-scope install gets its own version-pinned record keyed by that checkout's absolute path. Nothing reaps the result: `git worktree remove` does not touch `~/.claude`, no CLI verb removes a record by path, and the documented retention sweep covers nothing under `~/.claude/plugins/`. Changelog 2.1.224 shows per-project records are a live mechanism. Synced plugins (`<name>@synced`, cloud and Cowork only, no install record) are recorded as the contrast case. - Two questions are recorded as open probes, not asserted: which code path writes the records locally (one machine's 64 records sharing a single `installedAt` second is written as an observation on 2.1.261, with the probe that would settle it named), and whether a project-scope `false` writes any record. - `SKILL.md`'s stale-records section gains a short pointer to the subsection and keeps its existing boundary that the tool owning the directories is where records should be dropped. `gotchas.md` gains a gotcha on a committed block manufacturing records, citing the subsection. No remediation is proposed anywhere; that decision is open on #3688. - claude-ops `0.42.13` -> `0.42.14` with CHANGELOG entry. Docs only; no script changed; SKILL.md frontmatter unchanged. ## Verification - `markdownlint-cli2` over `plugins/claude-ops/**/*.md`: 0 issues in 54 files. `typos`: clean. `node scripts/validate-plugin-contracts.mjs`: pass (the retirements append-only sub-check reports skipped locally because `VALIDATE_CONTRACTS_BASE_REF` is unset; CI sets it). - SKILL.md frontmatter diffed byte-identical against `origin/main`. No em dashes in added prose. ## Related - #3688 (the decision this documents; stays open) - #3760 (re-verified the no-reap-by-path claim this section builds on) - #3775 (cache-content check, which skips absent project paths on the same reasoning) 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_019gWgHogJFQeCne5U7vHKAE --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Sep 6, 2026
Closed
kyle-sexton
added a commit
that referenced
this pull request
Sep 6, 2026
…63 (#3813) No related issue: this records the two probe results the #3688 decision was waiting on, without closing it; the structural remediation stays open on that issue. ## Summary `scope-semantics.md`'s "Where project-scope records come from" section carried two open probes: whether a local session writes project-scope install records for a repo's committed `enabledPlugins` block when the user already holds the plugins at user scope, and whether a project-scope `false` writes any record. Both were run on 2026-09-06 against Claude Code 2.1.263 in a scratch repository under the temp directory, with the three plugin state files hashed before and after. ## Fix - Probe 1: two `true` entries duplicating user-scope installs produced exactly two new `scope: "project"` records at the first headless session start, same `installedAt` millisecond, pinned to the user scope's version, `installPath` pointing at the user scope's existing cache directory. No cache directory was created. The section now states the write as a verified fact with the recipe and drops the "open probe" and "leading hypothesis" wording. - Probe 2: one `false` entry for a user-installed plugin left `installed_plugins.json` byte-identical and the session reported that plugin's skill unavailable while a sibling stayed available. Recorded as "a `false` writes nothing". - Also observed and recorded: `claude plugin uninstall -s project` rewrites the checkout's committed `.claude/settings.json`, deleting the id from `enabledPlugins` and reordering top-level keys. - `SKILL.md`'s stale-records section names the block as the source rather than a candidate and points at the recipe; the `gotchas.md` pointer line drops "two questions still open". Frontmatter unchanged. - claude-ops `0.42.17` -> `0.42.18` with CHANGELOG entry. Docs only. ## Verification - `markdownlint-cli2` over the three touched markdown files: 0 issues. `typos`: clean. No em dashes in added lines. - `SKILL.md` frontmatter diffed byte-identical against `origin/main`. - Real state after the probes: `~/.claude/settings.json`, `installed_plugins.json`, `known_marketplaces.json` all hash-identical to the pre-probe copies once the two scratch records were uninstalled at project scope from inside the scratch directory; the scratch directory was deleted. ## Related - #3688 (the decision these probes feed; stays open) - #3786 (recorded the probes as open) - #3760 (the 2.1.261 re-verification pass this extends) 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_019gWgHogJFQeCne5U7vHKAE Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Sep 6, 2026
…3826) Closes #3759 ## Summary The `plugins` skill's `install_new` render rests on the claim that an unset `userConfig` key renders the literal `${user_config.<key>}` placeholder rather than the manifest default. The 2.1.261 re-verification could not confirm or deny it because its positive control failed: a key set through `--settings` `pluginConfigs` rendered literal. Issue #3759 asked for a probe allowed to write real user settings so the two explanations (`--settings` does not feed substitution, or substitution regressed) could be separated. ## Fix - Probe run 2026-09-06 on Claude Code 2.1.263 with a throwaway local marketplace and a throwaway plugin carrying two `userConfig` string keys, both with manifest defaults. `claude plugin install <id> --config setkey=REALVALUE` wrote the real user-settings entry. Rendered skill body: the set key substituted, the unset key rendered the literal placeholder (not its default), `${CLAUDE_PLUGIN_ROOT}` substituted. Claim holds; explanation (1) from the issue is the right one, and there is no regression. - Second run with `--settings '{"pluginConfigs":{"<id>":{"options":{"unsetkey":"VIA_SETTINGS_FLAG"}}}}'`: the flag-sourced value substituted alongside the user-settings one. So `--settings` does feed skill-content substitution; the 2.1.261 control failed because its payload put the key directly under the plugin id instead of under `options`. - `SKILL.md` replaces the "inconclusive" paragraph with the 2.1.263 stamp, the `options` shape, the `--config` recipe, and a note that the current plugins-reference page says the default "is used if specified" for an unset key while the render contradicts it for skill content. `install_new` needs no fallback beyond the existing "placeholder means `ask`" rule. Frontmatter unchanged. - `scope-semantics.md` records the `options` nesting in the `pluginConfigs` section and updates the preamble's not-re-run list. - claude-ops version bump with CHANGELOG entry. Docs only. ## Verification - `markdownlint-cli2` over the touched markdown: 0 issues. `typos`: clean. No em dashes in added lines. `SKILL.md` frontmatter diffed byte-identical against `origin/main`. - Real state after the probe: throwaway plugin uninstalled, marketplace removed; `~/.claude/settings.json`, `installed_plugins.json`, `known_marketplaces.json` all hash-identical to the pre-probe backups. The `pluginConfigs` entry the probe wrote was removed by the uninstall. ## Related - #3734 and #3760 (the 2.1.261 pass whose control failed) - #3763 (the cache-residue observation from the same marketplace round trip, own PR) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3734
Summary
The
claude-ops:pluginsskill groundssyncon empirically probed Claude Code facts, each stamped with the CLI version it was observed on and carrying a recheck trigger. The newest stamps said 2.1.240 (one 2.1.218) against a 2.1.261 CLI, so the triggers were armed and unserviced. This pass re-runs every probe the issue lists, or marks it not re-run with the reason, and corrects prose where behaviour changed.Fix
$TEMP, never at user scope:update -s projectwrites no committed settings; the install/uninstall/enable/disable project-scope write matrix, key-created-when-absent, and whole-file re-serialization;enable -s projectgates on the merged effective value;-s projectkeys on the literal cwd with no path flag; no CLI verb reaps an install record by path andpruneremains a dependency axis.claude plugin update <bare-name>now resolves at project scope on 2.1.261, so the gotcha is re-worded (the skill already always passes full ids, no code change). The--idsrefusing--allclaim is re-attributed tofleet-state.sh's own guard, which is what actually enforces it.pluginConfigsreads user,--settings, and managed only (project and local ignored since v2.1.207); monitors require a session restart;userConfighas noenumtype; renames gate on v2.1.193./reload-pluginsbare-vs---forcewarning (both need an interactive session); install-summary activation line (doc-confirmed only);prunev2.1.121 and--forcev2.1.163 gates (docs state neither).userConfigunset-key placeholder probe, the highest-value one, was inconclusive, not a confirmation: its positive control failed (a key explicitly set via--settingspluginConfigsalso rendered literal).SKILL.mdnow says so and treats the claim as still resting on the 2.1.218 probe. Filed as claude-ops:plugins: userConfig substitution probe failed its positive control on CC 2.1.261 (pluginConfigs via --settings rendered literal) #3759.scope-semantics.mdandgotchas.mdname the pass date, state that per-claim stamps govern, and list the not-re-run claims by name.0.42.6->0.42.7with CHANGELOG entry. Docs only; no script changed.Verification
markdownlint-cli2overplugins/claude-ops/**/*.md: 0 issues.node scripts/validate-plugin-contracts.mjs: validated. SKILL.md frontmatter unchanged.~/.claude/settings.jsonsemantically identical to the pre-probe backup (amarketplace add/removeround trip for the throwaway plugin flipped line endings only);installed_plugins.jsonhas the same record set (user-scope versions advanced from the routine autoUpdate sweep that eachclaude -pprobe session triggers). No throwaway marketplace or plugin remains registered.Related
🤖 Generated with Claude Code
https://claude.ai/code/session_019gWgHogJFQeCne5U7vHKAE