guardrails: cli-flag-verify PostToolUse hook flags pre-existing unrelated lines (not diff-scoped)

[kyle-sexton]

Category

behavioral / improvement

Observed during

PR babysitting on #528. Edited plugins/toolchain/skills/check/SKILL.md at line 65 (replacing a sed prefix-strip with ${DEFAULT_BRANCH#"$REMOTE/"} parameter expansion). The PostToolUse cli-flag-verify hook fired and reported:

UNKNOWN_FLAG: dotnet test --project (not found in 'dotnet test --help')

Problem

The flagged token lives on line 108 — unrelated prose about ecosystem gotchas (For ecosystem-specific gotchas (xUnit --nologo trap, dotnet test --project, etc.)...), 43 lines away from the actual edit. The hook scans the whole edited file, not the diff, so any edit to a file that already contains a flag the DB doesn't recognize surfaces a "possible hallucination" warning about a line the agent never touched.

Impact

• Noise on every unrelated edit to any file containing a pre-existing (real or false-positive) flag hit.
• Pressure to "fix" out-of-scope lines to silence the warning, or to add skip-list entries, neither of which the current edit warrants.
• Erodes signal: a genuine hallucinated flag introduced by the edit is harder to spot amid pre-existing-line noise.

Suggested direction

Scope the cli-flag-verify check to tokens introduced/changed by the edit's hunk (diff-scoped), rather than re-scanning the entire file body on each PostToolUse. Separately, triage may want to confirm whether dotnet test --project is a genuine false positive in the flag DB (it is documented for dotnet test), but the diff-scoping behavior is the primary report.

Related

• guardrails: cli-flag-verify.sh still reads fd0 unbounded (last Win32-stall stdin site) #446 (cli-flag-verify.sh fd0 stdin) — different concern (stdin stall), same script.

Filed per babysit self-observation. Not refiling known: #484, #465, #473, #499, #504, #511, #512, #524.

[kyle-sexton]

This was generated by AI during triage. · Category: behavioral (duplicate)

status: needs-decision — recommend close: duplicate of #543.

Same hook (cli-flag-verify PostToolUse), same root cause (whole-file scan instead of diff-scoped → warns on pre-existing untouched lines), same suggested fix (scope to the edit's hunk). This report and #543 were filed ~3 seconds apart by two separate babysit workers on different PRs (#528 here, #529 there) — a concurrent-duplicate, exactly the class that motivates a shared dogfood-filing dedupe path (#540).

This report's distinct data point (toolchain check/SKILL.md line 65 edit firing on line 108) has been preserved as a second example on #543, so nothing is lost on close. The dotnet test --project flag-DB false-positive question is captured on #543 as the secondary item.

Recommend the human close this as a duplicate of #543 (the lower-numbered primary). Not routed ready to avoid two lanes implementing the same diff-scoping fix.

[kyle-sexton]

Decision brief (2-minute read)

Prepared by an AI agent from a read-only checkout at 789ec6c — verify before acting. This complements, and does not override, the existing triage recommendation on this issue to close as a duplicate of #543.

Problem. cli-flag-verify (PostToolUse advisory) warns on CLI-flag tokens anywhere in the edited file, not just in the lines the edit touched, so an unrelated edit surfaces a "possible hallucination" about a pre-existing line the agent never wrote (here: a line-65 edit flagged dotnet test --project on line 108). This is one member of the #547 over-fire class.

Today (verified). The over-fire is real and structural, not a tuning issue. plugins/guardrails/hooks/cli-flag-verify.sh reads only tool_input.file_path (line 50, via hook::read_file_path) and never reads old_string/new_string; emit_fragments() (lines 97-117) then scans the entire file — grep over all inline-code spans and awk over all fenced blocks for .md, or grep over all non-comment lines for shell. There is no hunk restriction, so every pre-existing flag is re-verified on every edit. The hook is already code-span-scoped and subcommand-aware (its false-positive suite has ~31 stay-quiet cases) — the missing dimension is edit-scope. The Edit payload does carry new_string (the test helper's edit_json builds it), so diff-scoping is feasible without new inputs.

Options.

• A — RECOMMENDED. Diff-scope the check to the edit's changed content, as the first fix conforming to guardrails: hooks ship false-positive over-fires with no per-hook fixtures or shared precision convention (6-issue class: diff-scope, quoted-arg, boolean-intent, stdin, path-branch) #547's precision convention; close this as a duplicate of guardrails: cli-flag-verify PostToolUse hook scans whole file, warns on pre-existing lines outside the edited hunk #543 and carry the fix there. Scan new_string (the added/changed text) instead of the whole file; fall back to whole-content only for Write/new files, where all content is new. Land the line-65-firing-on-line-108 case as a MUST-stay-quiet fixture in cli-flag-verify.test.sh. Per the existing triage, guardrails: cli-flag-verify PostToolUse hook scans whole file, warns on pre-existing lines outside the edited hunk #543 is the lower-numbered primary and this report's data point is already preserved there, so close guardrails: cli-flag-verify PostToolUse hook flags pre-existing unrelated lines (not diff-scoped) #544 as dup and track the diff-scope work on guardrails: cli-flag-verify PostToolUse hook scans whole file, warns on pre-existing lines outside the edited hunk #543 under the guardrails: hooks ship false-positive over-fires with no per-hook fixtures or shared precision convention (6-issue class: diff-scope, quoted-arg, boolean-intent, stdin, path-branch) #547 umbrella. Tradeoffs: correct root-cause fix; a genuinely hallucinated flag introduced by the edit is still caught, and pre-existing-line noise stops. Future-state risk: low — diff-scoping is the convention guardrails: hooks ship false-positive over-fires with no per-hook fixtures or shared precision convention (6-issue class: diff-scope, quoted-arg, boolean-intent, stdin, path-branch) #547 recommends, so this fix seeds it rather than diverging. Reversibility: fully reversible (advisory hook, config-gated, revertable). Rationale: fixes the class, not the instance, and keeps one lane (guardrails: cli-flag-verify PostToolUse hook scans whole file, warns on pre-existing lines outside the edited hunk #543) implementing the diff-scoping so two workers don't duplicate it.
• B — Add the specific flag to a skip-list / mark dotnet test --project in the flag DB. Tradeoffs: silences this one token; the whole-file-scan defect remains and any other pre-existing flag re-fires. Band-aid, not a fix. Not recommended.
• C — Fix diff-scoping in isolation, independent of guardrails: hooks ship false-positive over-fires with no per-hook fixtures or shared precision convention (6-issue class: diff-scope, quoted-arg, boolean-intent, stdin, path-branch) #547. Tradeoffs: fixes this hook but teaches the next hook nothing and skips the convention/fixture-discipline the umbrella exists to establish; also risks re-opening the guardrails: cli-flag-verify PostToolUse hook scans whole file, warns on pre-existing lines outside the edited hunk #543/guardrails: cli-flag-verify PostToolUse hook flags pre-existing unrelated lines (not diff-scoped) #544 two-lane duplication. Not recommended.

Related.

• guardrails: cli-flag-verify PostToolUse hook scans whole file, warns on pre-existing lines outside the edited hunk #543 — the primary of this duplicate pair (same hook, same whole-file-scan root, same suggested fix); recommend closing guardrails: cli-flag-verify PostToolUse hook flags pre-existing unrelated lines (not diff-scoped) #544 into it.
• guardrails: hooks ship false-positive over-fires with no per-hook fixtures or shared precision convention (6-issue class: diff-scope, quoted-arg, boolean-intent, stdin, path-branch) #547 — umbrella for the 6-issue over-fire class; its recommended precision convention makes "diff-scope PostToolUse:Edit to the changed hunk" the documented rule this fix should follow.
• guardrails: cli-flag-verify.sh still reads fd0 unbounded (last Win32-stall stdin site) #446 — same script, different concern (unbounded fd0 stdin read); another guardrails: hooks ship false-positive over-fires with no per-hook fixtures or shared precision convention (6-issue class: diff-scope, quoted-arg, boolean-intent, stdin, path-branch) #547 member.

[kyle-sexton]

Control-tower close (closure audit 2026-07-20, Class B). Duplicate of #543 — both the triage comment and the decision brief on this issue recommend close-as-duplicate; the distinct data point is already preserved on #543. Two independent audit verdicts HIGH.