Repository navigation
test(guardrails): measure the identity-domain guard on the operator's command corpus and prove parity with the local guard #9750
Description
Parent
Refs #5945
Depends on
- feat(guardrails): deny commands that name another identity domain's path #9744
- feat(guardrails): allow the one-way shared exception directory in the identity-domain guard #9745
- feat(guardrails): deny another identity domain's markers in identity positions #9746
- feat(guardrails): require the domain's gh and az config directory #9747
- feat(guardrails): refuse global git identity writes in every domain #9748
- feat(guardrails): write the identity-domain map from the machine profile in setup #9749
Needs a person: the firing-rate measurement needs the operator's recorded command corpus, and parity needs the operator's local 170-case suite; neither is in this repository.
Outcome
The identity-domain guard's firing rate on real commands is known, its false positives are regression cases,
and it matches the local guard it replaces.
Decisions
- Epic feat(guardrails): generalize machine-local safety guards into plugin guards #5943 defaults, adopted: the guard lives in guardrails behind its dispatcher and a per-guard kill switch;
bash under the node entry point; port behavior, not the local Python. - The domain map is derived from
/harness-ops:machine-profileand is machine-local: it never sits in a tracked
file, fixture or doc. Placeholders only. - Fail open on any internal error; every exit is 0 with the decision in stdout JSON.
- The guard reads command text: a deterrent, not a sandbox. Declared gaps: computed paths, scripts run by name.
Done when
- The measured firing rate on a recorded command corpus is reported on this issue or its PR, per
docs/conventions/hook-precision. - Each false positive found is added to
plugins/guardrails/hooks/block-cross-domain.test.shas a regression case, andscripts/run-test-file.sh plugins/guardrails/hooks/block-cross-domain.test.shexits 0. - The plugin guard passes behavior-equivalent test cases for the local guard it replaces (the operator retires the local copy then, outside this repo).
Activity
kyle-sexton commentedon Oct 26, 2026
Released for re-triage under the owner's 2026-10-26 rule: a person is involved only for "1. A change in original intent/vision 2. Something comes up during implementation ... that alters intent/vision or the primary design 3. If there is a HIGH/CRITICAL security issue". None applies here, so the hold label came off and intake triage runs again now.
To hold this issue again, add
needs-human(orstatus: needs-decision) and name which of the three applies.github-actions commentedon Oct 26, 2026
Proposed work class: work-class: scoped
Basis: automated intake triage; the lanes App applied this class.
Size: Small
Complexity: Medium
Uncertainty: Knowable
Flags: none
Blocked by: none
Understood: measure the identity-domain guard's firing rate on a recorded command corpus, add each false positive as a regression case in
block-cross-domain.test.sh, and show parity with the operator's local guard.The corpus, the machine-local domain map and the local 170-case suite are only on the operator's host. That makes this a host-run item, which admission routes. It is not a decision for a person. The issues it depends on (#9744–#9749) no longer appear among the open issues. I found no duplicate.
Automated triage, applied by the lanes App.
2 remaining items
melodic-automation-lanes commentedon Oct 26, 2026
Not admitted to the issue lane yet: blocker #9744 is not closed by a listed human or a merge; blocker #9745 is not closed by a listed human or a merge; blocker #9746 is not closed by a listed human or a merge; blocker #9747 is not closed by a listed human or a merge; blocker #9748 is not closed by a listed human or a merge; blocker #9749 is not closed by a listed human or a merge.
github-actions commentedon Nov 1, 2026
github-actions commentedon Nov 1, 2026
melodic-automation-lanes commentedon Nov 1, 2026
The work-resolve-issue lane stopped on this issue (
host-only): the run reports the work needs a host run the lane cannot make.It added
host-runand removedagent-ready: an agent run on a host builds it, and it waits on no decision from a person.In the run's words:
What the run says it did not do:
Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/41261790285
To hand it back to the lane, remove
host-runand addagent-readyagain.github-actions commentedon Nov 1, 2026
Intake triage did not triage this issue after 2 dispatches, so the sweep no longer dispatches it and labeled it needs-triage for a maintainer. Triage it by hand, and read the runs for why intake did not:
kyle-sexton commentedon Nov 2, 2026
Claim. Backlog HITL host-run worker is taking this now.
kyle-sexton commentedon Nov 2, 2026
Host run. Replayed the operator's desktop WSL transcript corpus through
block-cross-domain.shas shipped at00f993925f, with no domain map, and counted the guard's logged refusals; the domain rule and the parity check could not run.Claude Code 2.1.316. Host: the operator's desktop WSL host.
What ran
block-cross-domain: denied. On inspection all 7 are tool output that quotes the guard (source reads, greps over its file, and seeded probe scripts that exercise it), not hook refusals. Logged refusals by the guard: 0.gitandconfig, orgitconfig(a superset of the guard's identity-write prefilter), went through the hook as stdin JSON withCLAUDE_PLUGIN_DATAset to an empty throwaway directory: 25,632 calls replayed, 1 fire, 0 errors or timeouts. The corpus's--globaland--systemgit config calls are reads (--get,--get-regexp), and the guard let all of them through.Result per criterion
identity-domains.jsonexists in the desktop's WSL plugin data or the laptop's WSL plugin data, so the guard runs only the identity-write rule there, and an invented map would make the rate meaningless..claudeprofile, its dotfiles history, or the laptop's WSL.claude. The laptop's Windows profile was not checked. #5945 says the guard came from a local handoff item and its code stays outside this repository.Seeded probe sessions were not filtered out before the replay. The one fire came from a work worktree, not a probe, so leaving them out would not change the count.
Decision recorded. (agent; not an owner decision) The one fire is a
printf ... > .work/<dir>/<name>.gitconfigfixture write inside a gitignored directory, and it changed no live identity. It is counted as a true positive because a write to a*.gitconfigfile anywhere is the guard's documented target. The suite pins that target as a deliberate deny (plugins/guardrails/hooks/block-cross-domain.test.sh:670, "a redirection to a *.gitconfig file"), and #5945 rule 6 says the local guard does the same. This follows the precedent indocs/contracts/hook-precision/guard-precision.md: a fire on a guard's documented target counts as true. Basis: the test line and the contract file above.Owner's step (needs the machine that holds the local guard and the real domain map)
${CLAUDE_PLUGIN_DATA}/identity-domains.json(/guardrails:setup apply identity-domains, which builds it from the stored/harness-ops:machine-profile), replay that host's transcripts throughplugins/guardrails/hooks/block-cross-domain.sh. Use stdin JSON{tool_name, cwd, tool_input:{command}}for every Bash and PowerShell call that names a mapped root, an owner directory, a marker,ghoraz. Classify each fire and report aggregate counts only, perdocs/contracts/hook-precision/guard-precision.md.plugins/guardrails/hooks/block-cross-domain.test.sh, using placeholder names only. Thenscripts/run-test-file.sh plugins/guardrails/hooks/block-cross-domain.test.shmust exit 0. After that, retire the local copy.Target labels: add none; remove none.
kyle-sexton commentedon Nov 2, 2026
Decision recorded. Done-when 3 (behavior-equivalent cases for the local guard this replaces) is moot and dropped, under the owner's standing direction s1 ("When the evidence settles a question, act on it. Do not ask me."), re-pasted by the owner in the HITL Backlog session on 2026-11-02.
Evidence: the 170-case local identity guard is not on the desk host, the laptop WSL host, or anywhere in the dotfiles history (searched by the HITL Backlog lane and GitHub Support, 2026-11-02). There is no local copy to match and none for the operator to retire. Done-when 1 and 2 stand.
kyle-sexton commentedon Nov 3, 2026
Measured firing rate of
block-cross-domainon this host's command corpus: 0 false positives. Guard atbfa9d73f13, on melo-lap-001 (WSL).Corpus: 226,641 recorded Bash calls from 11,887 transcripts, 215,568 unique (cwd, command) pairs, 2026-09-23 to 2026-11-03. Replays A and B took the newest 30,000 pairs (2026-10-31 on). No raw command is posted here.
Denies, classified by hand: 3 true positives (a recursive search or listing of the other domain's tree from a domain cwd). 1 harmless on its face: an
author.emailset on a throwaway repo, which depends on a marker this test map added, and which the test file already pins as a deny. So done-when 2 has no regression case to add.Limits
Found
git config --global include.path <file>was allowed, though an include can set the identity as anincludeIfdoes. Seen in a probe, not in the corpus. Fixed in the PR that follows.ownersderive directories under~/src/github.comand~/worktrees/github.com(block-cross-domain.sh:91-92), which this host does not use, soownersadds no path here. A root still works.chezmoi apply ~/.gitconfigis allowed (2 pairs). That is the sanctioned dotfiles route, so it stays allowed.Done-when 3 was dropped on 2026-11-02.