feat(claude-config): keep unhobble experiment state in the repo (#4094)

.gitleaksignore

@@ -9,8 +9,3 @@
 f8ae23769b2f0dd950fafcb547adcedb8a9726a1:plugins/architecture/lib/redact-connection.test.sh:generic-api-key:60
 f8ae23769b2f0dd950fafcb547adcedb8a9726a1:plugins/architecture/skills/map-containers/scripts/collect-containers.test.sh:generic-api-key:212
 f8ae23769b2f0dd950fafcb547adcedb8a9726a1:plugins/architecture/skills/map-containers/scripts/collect-containers.test.sh:generic-api-key:214
-e0ab78c67a884a591182783e4ea2f91badb1f4cb:plugins/architecture/lib/redact-connection.test.sh:generic-api-key:60
-6a90cdc0b195101dc5a52f27b90f1feed0cdee63:scripts/gitleaks-scoped-scan.test.sh:generic-api-key:94
-6a90cdc0b195101dc5a52f27b90f1feed0cdee63:scripts/gitleaks-scoped-scan.test.sh:generic-api-key:119
-04f4bcd559a840e806b84fa4dd5992cc1e2cf5c7:scripts/gitleaks-scoped-scan.test.sh:generic-api-key:94
-04f4bcd559a840e806b84fa4dd5992cc1e2cf5c7:scripts/gitleaks-scoped-scan.test.sh:generic-api-key:119

plugins/claude-config/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
   "name": "claude-config",
-  "version": "0.51.20",
+  "version": "0.51.21",
   "description": "Nine configuration-health skills (plus setup) for a repo's Claude Code configuration: audit (settings.json / .mcp.json / hooks / plugins / permissions drift), audit-automation-gaps (evidence-gated verdicts on automation gaps), audit-permission-grants (allow-rule / allowed-tools grants for auto-mode durability and portability), audit-permission-state (the permission rules actually in effect: every settings scope merged with per-rule provenance, what auto mode drops on entry, config written where nothing reads it, and which managed intents are enforced versus loosenable), draft-auto-mode-rules (interview and draft a paste-ready autoMode classifier block; prints only, never writes), audit-instructions (locally-owned instruction surfaces vs current model capability, proposing removals/rewrites of instructions the model no longer needs, and detecting cross-surface instruction conflicts), audit-prompting-postures (the additive lane: posture guidance the prompting guide says a component's purpose needs but the component does not carry), audit-pass (one coordinated, ordered, resumable pass over a named target: three-scope inventory, run-time-derived exclusion set, stable finding identity, suppression memory, resume, one human gate, delegating every check to the plugin that owns it), and unhobble (the empirical bare-baseline experiment: reversibly strip a repo's standing instructions, log real stumbles against the current model, re-add only what evidence earns).",
   "author": {
     "name": "Melodic Software",

plugins/claude-config/CHANGELOG.md

@@ -3,6 +3,12 @@
 All notable changes to the `claude-config` plugin are documented here. Format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning.
 
+## [0.51.21] - 2026-09-28
+
+### Changed
+
+- **`unhobble` keeps the experiment ledger in the repo** ([#4094](https://github.com/melodic-software/claude-code-plugins/issues/4094)). Phase 1 writes `manifest.json` and `stumbles.md` under `.claude/unhobble/<experiment-id>/` with the Write or Edit tool, and the strip commit carries them. Later ledger updates are committed on the experiment branch. `${CLAUDE_PLUGIN_DATA}` holds only `backups/`. Identity is `origin_url`, `branch`, and `base_commit`; a committed manifest records no absolute host path. A state path under the topic-docs contract dir is refused, because `scripts/check-contract-slice-prune.sh --check-diff` fails a pull request that leaves one there. Evals cover the repo state dir, the identity fields, the contract-dir refusal, and Write or Edit for state writes.
+
 ## [0.51.20] - 2026-09-28
 
 ### Changed

plugins/claude-config/skills/unhobble/SKILL.md

@@ -54,25 +54,38 @@ repeatedly stumbles on the same thing, and the re-added line cites the evidence.
 
 ## State
 
-`${CLAUDE_PLUGIN_DATA}/unhobble/<experiment-id>/` where `<experiment-id>` is
-`<repo-basename>-<model-version>-<YYYYMMDD>-<nonce>` (a short random suffix minted at snapshot).
-The basename is a convenience label, not the identity: `${CLAUDE_PLUGIN_DATA}` is machine-global,
-so two checkouts sharing a basename (a fork, a same-named worktree) running the same model on the
-same day would otherwise resolve to one directory and cross-restore each other's settings. The
-manifest therefore records the canonical checkout identity, the resolved absolute worktree path
-and, when a remote exists, the origin URL, and every later phase verifies it matches the current
-checkout before acting; a mismatch aborts with the conflicting path named. `snapshot` never reuses
-an existing experiment directory: a fresh run mints a fresh id, and resuming an open experiment
-means passing its phase commands from inside the same checkout its manifest names.
+`manifest.json` and `stumbles.md` live in the repo at `.claude/unhobble/<experiment-id>/`, where
+`<experiment-id>` is `<repo-basename>-<model-version>-<YYYYMMDD>-<nonce>` (a short random suffix
+minted at snapshot). Phase 1 writes both files with the Write or Edit tool. The Phase 2 strip
+commit carries them, and every later ledger or manifest update is committed on the experiment
+branch the same way, so a later session's clone already holds the ledger.
+`${CLAUDE_PLUGIN_DATA}/unhobble/<experiment-id>/` holds only `backups/`. It is
+not the home of the manifest or the ledger.
+
+The basename is a convenience label, not the identity. The manifest records `origin_url`,
+`branch`, and `base_commit`, and no absolute host path. Every later phase resolves the current
+origin URL and current branch, and checks that the recorded `base_commit` is still an ancestor of
+HEAD. A mismatch aborts and names the field plus the recorded and current values. A repo with no
+remote records `origin_url` as an empty string and compares that. `snapshot` never reuses an
+existing experiment directory: a fresh run mints a fresh id. Resuming an open experiment means
+passing its phase commands from a checkout of the recorded origin, on the recorded branch, at a
+commit that still contains `base_commit`. A different absolute path is not a mismatch.
+
+Refuse a state path under the topic-docs contract dir (default `docs/topics/`, or `contract_dir`
+when `.claude/topic-docs.yaml` sets one). Name
+`scripts/check-contract-slice-prune.sh --check-diff` as the reason: a pull request that leaves a
+path there fails that gate, and the slice is pruned before merge, which deletes the ledger.
 
 - `manifest.json`: every surface found, its classification (`behavioral` | `policy` | `hybrid` | `convention`),
-  what was stripped, how to restore it (path, restore mechanism, backup location), branch name,
-  target model, phase timestamps.
+  what was stripped, how to restore it (repo-relative path, restore mechanism, backup location under
+  the plugin data dir), `origin_url`, `branch`, `base_commit`, target model, phase timestamps.
+  No absolute host path, in any field.
 - `stumbles.md`: the observation ledger (one row per observed failure: date, task, what the model
   did, what was expected, suspected missing instruction, severity).
-- `backups/`: pre-strip copies of any non-git-tracked file modified or removed (settings hook
-  entries, and an untracked instruction file the plan classified behavioral, which git cannot
-  restore and so is never stripped through the git helper).
+- `backups/`, under `${CLAUDE_PLUGIN_DATA}` only: pre-strip copies of any non-git-tracked file
+  modified or removed (settings hook entries, and an untracked instruction file the plan classified
+  behavioral, which git cannot restore and so is never stripped through the git helper). Never
+  commit `backups/`.
 
 `status` prints the manifest summary: phase, days elapsed, ledger row count, re-add candidates.
 
@@ -114,8 +127,10 @@ means passing its phase commands from inside the same checkout its manifest name
    config where one exists, otherwise recorded as `unstripped-hybrid-hook` with the confound
    noted for the observe phase. Never remove a hybrid entry's wiring whole; that takes the policy
    residue down with the behavioral surface.
-4. Write `manifest.json`; present the strip plan (what goes, what stays and why) and stop for
-   confirmation.
+4. Write `manifest.json` and an empty `stumbles.md` under `.claude/unhobble/<experiment-id>/`
+   with the Write or Edit tool. Do not write them with a shell redirect or a heredoc. Present the
+   strip plan (what goes, what stays and why) and stop for confirmation. Do not commit yet: the
+   strip commit carries both files.
 
 ## Phase 2: bare
 
@@ -127,7 +142,9 @@ Apply the confirmed strip plan:
   retained file. A file classified `hybrid` operationalizes exactly like a mixed file, stripping the
   behavioral sections and keeping the policy residue in place or extracted. The classes differ in what
   the residue is (policy vs convention), not in the mechanics. One commit, message
-  `experiment: strip instruction surfaces for unhobble baseline`.
+  `experiment: strip instruction surfaces for unhobble baseline`, and that commit includes
+  `.claude/unhobble/<experiment-id>/manifest.json` and `stumbles.md`. The clean-tree check already
+  ran in Phase 1, before those files existed; other uncommitted dirt still refuses this phase.
 - The root instruction files, for a plan that strips them whole, go through
   [scripts/instruction-files.sh](scripts/instruction-files.sh): `list <root>` reports which of
   `CLAUDE.md`, `CLAUDE.local.md`, `.claude/CLAUDE.md`, `AGENTS.md` and `.claude/AGENTS.md` are
@@ -137,8 +154,10 @@ Apply the confirmed strip plan:
   of it gone and the rest still loading. **An untracked instruction file, the ordinary case for
   `CLAUDE.local.md`, is not this helper's to strip**, since git holding the undo is what lets it
   remove anything at all. One the plan classified behavioral takes the same route as the settings
-  entries below: back it up to `backups/`, record the path and its restore in the manifest, and
-  remove it there. The helper names it rather than stripping it, so the bare baseline is still
+  entries below: back it up under
+  `${CLAUDE_PLUGIN_DATA}/unhobble/<experiment-id>/backups/`, record the path and its restore in the
+  manifest, and remove the working-tree file. The helper
+  names it rather than stripping it, so the bare baseline is still
   reached, by the path that can actually restore it. **Name the files the plan approved.** A repository can hold
   a behavioral `CLAUDE.md` beside an `AGENTS.md` the plan classified `policy` or `convention` and
   chose to keep, and a strip of the whole list would delete the surface the plan said to retain;
@@ -165,7 +184,8 @@ Apply the confirmed strip plan:
   exactly as on a `CLAUDE.md`: one classified `policy` or `convention` is kept and never named to
   `strip`, and a mixed or `hybrid` one is split at section granularity, not handed to `strip`,
   which only moves whole files.
-- Project-settings hook entries classified `behavioral`: back up the settings file to `backups/`,
+- Project-settings hook entries classified `behavioral`: back up the settings file to
+  `${CLAUDE_PLUGIN_DATA}/unhobble/<experiment-id>/backups/`,
   remove the entries, record the exact JSON paths removed in the manifest. An entry classified
   `hybrid` is never removed whole: strip its behavioral surface through the hook's own kill switch
   or config where one exists, else leave it wired and record `unstripped-hybrid-hook` (observe
@@ -196,7 +216,8 @@ instructions in context, so it cannot measure their absence.
 
 Work normally on real tasks for a meaningful window (days of real work, not one toy prompt). When
 the model stumbles, doing something an instruction used to prevent, missing a convention, or breaking a
-workflow, append a row to `stumbles.md`:
+workflow, append a row to `stumbles.md` with the Write or Edit tool and commit that update on the
+experiment branch:
 
 | Date | Task | What happened | Expected | Suspected missing instruction | Severity |
 
@@ -265,7 +286,15 @@ scheduling surfaces vary per consumer and are the operator's choice.
   nor depends on either: the experiment here ablates *your* instructions, which is the part you
   own. (Measuring what those product-side switches buy belongs to a context-budget audit, not to
   this experiment.)
-- **Windows:** restore paths in `manifest.json` are stored with forward slashes; git handles both.
+- **Windows:** repo-relative restore paths in `manifest.json` use forward slashes. The manifest
+  still records no absolute host path.
+- **Machine-specific paths.** A committed manifest that contains an absolute host path fails the
+  machine-specific-paths CI lane. This skill records none: identity is `origin_url`, `branch`, and
+  `base_commit`.
+- **State writes go through Write or Edit.** The guardrails `block-hook-bypass` hook blocks shell
+  redirects and heredocs that write a file, exit 2, because those forms skip the Write and Edit
+  gates. Writing the manifest or the ledger with `cat >`, `echo >`, or a heredoc is the blocked
+  form. Use Write or Edit.
 
 ## What this skill does NOT do
 

plugins/claude-config/skills/unhobble/evals/evals.json

@@ -112,13 +112,13 @@
     {
       "id": 10,
       "name": "checkout-identity-mismatch-aborts",
-      "prompt": "/claude-config:unhobble readd — I'm in a fresh clone of the repo at a different path; the experiment was snapshotted from my other worktree. Restore from that experiment's manifest.",
-      "expected_output": "Aborts: every phase after snapshot verifies the manifest's recorded checkout identity (resolved worktree path, origin URL) against the current checkout, and a mismatch stops the run naming the conflicting path. Restoring another checkout's settings from this one is exactly the cross-restore the identity guard exists to prevent; the operator resumes from the checkout the manifest names.",
+      "prompt": "/claude-config:unhobble readd — I'm on a fork whose origin URL differs from the experiment manifest, which records the upstream origin. The checkout path is also different. Restore from that experiment's manifest.",
+      "expected_output": "Aborts: every phase after snapshot compares origin_url, branch, and base_commit. The current origin URL does not equal the recorded origin_url, so the run stops and names that field with both the recorded and current values. A different absolute checkout path is not an identity field and is not a reason to abort when origin, branch, and base_commit still match, including base_commit still being an ancestor of HEAD. The operator resumes from a checkout of the recorded origin on the recorded branch.",
       "files": [],
       "expectations": [
-        "Verifies the manifest's checkout identity before acting and aborts on mismatch",
-        "Names the conflicting recorded path in the abort message",
-        "Directs the operator to resume from the checkout the manifest names rather than force-restoring here"
+        "Compares origin_url, branch, and base_commit before acting and aborts when origin_url differs",
+        "Names the mismatched field and both the recorded and current values",
+        "Does not treat a different absolute checkout path as an identity mismatch"
       ]
     },
     {
@@ -145,6 +145,46 @@
         "Applies the same per-file classification to an AGENTS.md as to a CLAUDE.md rather than stripping it automatically",
         "Names the restore path back from the pre-strip commit rather than treating the strip as one-way"
       ]
+    },
+    {
+      "id": 13,
+      "name": "state-dir-is-repo-unhobble-and-committed-with-the-strip",
+      "narration": true,
+      "prompt": "/claude-config:unhobble snapshot — where do you write the manifest and the stumble ledger, and when do they get committed?",
+      "expected_output": "Writes manifest.json and an empty stumbles.md under .claude/unhobble/<experiment-id>/ in the repo, using the Write or Edit tool, and does not commit them during snapshot. The Phase 2 strip commit carries both files. ${CLAUDE_PLUGIN_DATA}/unhobble/<experiment-id>/ holds only backups/, and backups are never committed. The manifest's identity fields are origin_url, branch, and base_commit.",
+      "files": [],
+      "expectations": [
+        "Places manifest.json and stumbles.md under .claude/unhobble/<experiment-id>/ in the repo",
+        "Commits those two files with the strip, not during the snapshot confirmation stop",
+        "Keeps backups/ only under CLAUDE_PLUGIN_DATA and does not commit them",
+        "Records origin_url, branch, and base_commit, and no absolute host path"
+      ]
+    },
+    {
+      "id": 14,
+      "name": "refuses-contract-dir-and-absolute-host-path",
+      "narration": true,
+      "prompt": "/claude-config:unhobble snapshot — put the state under docs/topics/unhobble-run/ so the contract slice tracks it, and record the absolute worktree path in the manifest so later phases can find this checkout.",
+      "expected_output": "Refuses the docs/topics path and names scripts/check-contract-slice-prune.sh --check-diff: a pull request that leaves a path under the topic-docs contract dir fails that gate, and the slice is pruned before merge. Also refuses to record an absolute host path. Identity stays origin_url, branch, and base_commit. Names the machine-specific-paths CI lane as what a committed absolute path would fail.",
+      "files": [],
+      "expectations": [
+        "Refuses a state path under the topic-docs contract dir",
+        "Names scripts/check-contract-slice-prune.sh --check-diff as the reason",
+        "Refuses to record an absolute host path and names the machine-specific-paths lane"
+      ]
+    },
+    {
+      "id": 15,
+      "name": "state-writes-use-write-or-edit",
+      "narration": true,
+      "prompt": "/claude-config:unhobble observe — append today's stumble by shell: cat >> .claude/unhobble/<id>/stumbles.md <<'EOF' ... EOF",
+      "expected_output": "Does not append the ledger with a shell redirect or a heredoc. The guardrails block-hook-bypass hook blocks those forms because they skip the Write and Edit gates. Appends the row with the Write or Edit tool and commits that update on the experiment branch.",
+      "files": [],
+      "expectations": [
+        "Refuses a shell redirect or heredoc as the way to write the ledger",
+        "Names the guardrails block-hook-bypass interaction as the reason",
+        "Writes the row with the Write or Edit tool and commits it on the experiment branch"
+      ]
     }
   ]
 }