guardrails: hooks ship false-positive over-fires with no per-hook fixtures or shared precision convention (6-issue class: diff-scope, quoted-arg, boolean-intent, stdin, path-branch)

[kyle-sexton]

Lane: triage · Category: improvement (prevention umbrella) · Source: triage cycle 2026-07-19 — routing #543/#544/#545/#546 surfaced a class with no prevention layer.

Pattern — guardrails hooks recurrently ship false-positive over-fires, each caught in production and hand-filed

Open guardrails-hook false-positive / imprecision issues, one root class, zero prevention:

• guardrails: cli-flag-verify PostToolUse hook scans whole file, warns on pre-existing lines outside the edited hunk #543 / guardrails: cli-flag-verify PostToolUse hook flags pre-existing unrelated lines (not diff-scoped) #544 (duplicate pair) — cli-flag-verify scans the WHOLE file, warns on pre-existing lines outside the edited hunk (not diff-scoped).
• guardrails:flag-commit-pr-skill-bypass: advisory false-fires on canonical git commit -F - with inline heredoc Co-Authored-By trailer (requires BOTH markers, contradicting documented "either marker -> stay quiet" intent) #545 — flag-commit-pr-skill-bypass uses || where intent is && → fires on the canonical git commit -F - heredoc it's documented to stay quiet on.
• block-hook-bypass: false-fires when a Bash command contains both an echo token and a redirect (script-stdout capture, or quoted-string arguments) rather than an actual file-authoring bypass #546 — block-hook-bypass matches echo + > anywhere in a command string → blocks legitimate subprocess-stdout capture and any command whose quoted args merely mention the tokens.
• guardrails: cli-flag-verify.sh still reads fd0 unbounded (last Win32-stall stdin site) #446 — cli-flag-verify reads fd0 unbounded (stdin stall).
• guardrails: hardcoded-path-check false positives — repo-path branch flags all paths under home; percent-env interpolations flagged #389 — hardcoded-path-check repo-path branch flags ALL paths under home; percent-env interpolations flagged.
• guardrails: hardcoded-path-check — Windows-aware remediation, inline opt-out for machine-scoped files, document gitignore limitation #390 — hardcoded-path-check Windows-aware remediation / inline opt-out gaps.

Note: #544 and #543 were filed ~3s apart by two babysit workers — the same over-fire found twice, independently, in production.

Root cause

Two systemic gaps behind every instance:

1. No false-positive regression fixtures per hook. Hooks ship with no golden corpus of "commands/edits that MUST stay quiet" alongside "that MUST fire." So an over-fire is only discoverable by a worker tripping it live, then hand-filing — the pipeline pays at production time, repeatedly, for a defect a fixture would catch at hook-authoring time. (Directly analogous to source-control:babysit-prs: one shared authorship/finding/approval classifier + golden fixtures; gate script owns the merge-ready boolean (6-issue class) #534's shared-classifier + golden-fixtures argument for babysit, and quality-net: skill changes ship with zero regression detection — 120 evals.json unexecuted, skill-quality:check uninvoked (staged eval/gate/pass^k umbrella) #530's skill-regression net.)
2. No shared precision convention. The same imprecision recurs across independent hooks: whole-file instead of diff-scoped (guardrails: cli-flag-verify PostToolUse hook scans whole file, warns on pre-existing lines outside the edited hunk #543/guardrails: cli-flag-verify PostToolUse hook flags pre-existing unrelated lines (not diff-scoped) #544), substring co-occurrence instead of structural match (block-hook-bypass: false-fires when a Bash command contains both an echo token and a redirect (script-stdout capture, or quoted-string arguments) rather than an actual file-authoring bypass #546), quoted-argument blindness (block-hook-bypass: false-fires when a Bash command contains both an echo token and a redirect (script-stdout capture, or quoted-string arguments) rather than an actual file-authoring bypass #546), boolean-intent inversion (guardrails:flag-commit-pr-skill-bypass: advisory false-fires on canonical git commit -F - with inline heredoc Co-Authored-By trailer (requires BOTH markers, contradicting documented "either marker -> stay quiet" intent) #545), unbounded stdin (guardrails: cli-flag-verify.sh still reads fd0 unbounded (last Win32-stall stdin site) #446), path-branch over-broad matching (guardrails: hardcoded-path-check false positives — repo-path branch flags all paths under home; percent-env interpolations flagged #389). Each is a fresh hand-fix with no reusable discipline.

Observed vs expected

• Observed: guardrails hooks are authored ad hoc; false-positives are found by babysit workers during real passes and filed one at a time; fixing one hook teaches the next nothing.
• Expected: (a) a per-hook golden-fixture harness (true-positive MUST-fire + false-positive MUST-stay-quiet cases; every filed instance above becomes a fixture), run in CI on any plugins/guardrails/hooks/** change; (b) a documented hook-precision convention — diff-scope PostToolUse:Edit checks to the changed hunk (whole-file only for new files), match structural producers not token co-occurrence, ignore tokens inside quoted args, bound stdin reads, and prefer either/or marker logic where "already-canonical → stay quiet" is the intent. New hooks and the fixes for the members conform to it.

Category

improvement (prevention). Kills a recurring false-positive class + the babysit-worker-time it burns finding them live, instead of N hand-fixes plus indefinite recurrence.

Relations

Members #389 #390 #446 #543 #545 #546 (#544 dup of #543); #534 (same fixtures-as-prevention pattern for the babysit classifier); #530 (skill regression net — behavior-level sibling); #540 (shared dogfood-filing — the #543/#544 concurrent-duplicate is its motivating example); #531/#532 (sibling prevention umbrellas). Decompose candidate.

[kyle-sexton]

This was generated by AI during triage. · Category: improvement (prevention umbrella) · Priority: priority: high (kills a recurring false-positive class + the babysit-worker time spent finding over-fires live). Routed needs-human + wayfind: design. Decompose candidate.

Why needs-human

Two systemic deliverables, both design decisions the maintainer must shape (the body has the full analysis):

1. Per-hook golden-fixture harness — true-positive MUST-fire + false-positive MUST-stay-quiet corpus per hook, run in CI on any plugins/guardrails/hooks/** change; every filed member becomes a fixture. (Same fixtures-as-prevention pattern as source-control:babysit-prs: one shared authorship/finding/approval classifier + golden fixtures; gate script owns the merge-ready boolean (6-issue class) #534 / quality-net: skill changes ship with zero regression detection — 120 evals.json unexecuted, skill-quality:check uninvoked (staged eval/gate/pass^k umbrella) #530.)
2. Documented hook-precision convention — diff-scope PostToolUse:Edit to the changed hunk (whole-file only for new files), structural-producer match not token co-occurrence, quoted-arg-ignore, bounded stdin, either/or marker logic.

The harness shape + the convention content are the design; not a defaultable fork.

Members (cross-linked, mostly already routed ready this session)

#543 (diff-scope) · #544 (dup of #543) · #545 (||→&&) · #546 (echo+redirect) · #446 (stdin) · #389/#390 (path-check). Recommend the member fixes land AS the first fixtures of the harness rather than as isolated patches. Related: #534, #530 (sibling prevention), #540 (shared dogfood-filing — the #543/#544 concurrent-dup is its motivating example).

[kyle-sexton]

Dogfood example (work lane, this session): multiple implementation subagents reported the /commit and /pull-request advisory hooks firing as false positives on direct git commit -F - / gh pr create even when the canonical contract was fully met (heredoc message with required trailers; PR body with ordered sections + closing keyword + non-empty ## Related, confirmed by green pr-title/pr-issue-linkage). The advisory suggested routing through the skills when the raw path already satisfied them — recurring across the #434, #422, #413 workers. If the commit/PR skill-suggestion advisory isn't already in this issue's 6-hook class, it's a 7th sibling: same root (fires without checking whether the canonical contract is already satisfied).

[kyle-sexton]

Decision brief (2-minute read)

Prepared by an AI agent from a read-only checkout at 789ec6c — verify before acting. This is a decision brief, distinct from the triage routing note already on this issue; it does not supersede the wayfind: design / needs-human routing.

Problem. Guardrails hooks recurrently ship false-positive over-fires (#389/#390/#446/#543/#544/#545/#546), each found live by a babysit worker and hand-filed. The umbrella asks for two prevention layers — per-hook fixtures and a shared precision convention — so the class stops recurring.

Today (verified). One premise in the issue body is inaccurate and it changes the scope:

• A per-hook false-positive fixture harness already exists and is CI-gated. Every hook has a plugins/guardrails/hooks/<name>.test.sh; they share plugins/guardrails/hooks/guardrails-test-helpers.sh; scripts/run-plugin-tests.sh runs all plugins/**/*.test.sh; .github/workflows/ci.yml gates on it ("Run plugin contract tests"). MUST-stay-quiet negative cases are pervasive — cli-flag-verify.test.sh alone has ~31 (prose flag, shell/fenced comment, cross-span, table cell), secret-pattern-detection.test.sh ~19, hardcoded-path-check.test.sh ~17. So "no per-hook fixtures / no golden MUST-fire+MUST-stay-quiet corpus" is not the real gap.
• What is genuinely missing: (1) a documented precision convention — there is a cross-hook convention precedent at docs/conventions/hook-telemetry/, but no hook-precision sibling, so each hook reinvents precision ad hoc; (2) the discipline that binds every filed production false-positive to a new MUST-stay-quiet case in that hook's existing .test.sh (today the FP is filed as an issue, never as a fixture); (3) specific uncovered defects — chiefly diff-scoping: cli-flag-verify.sh reads only tool_input.file_path (line 50) and re-scans the whole file in emit_fragments() (lines 97-117), so its test suite cannot assert hunk-scoping because the hook never reads new_string.

Options.

• A — RECOMMENDED. Document a precision convention + adopt the FP-to-fixture discipline, reusing the existing harness; land the 6 members as the first conforming fixes. Add docs/conventions/hook-precision/ (mirroring the hook-telemetry convention's shape) enumerating the rules the members imply: diff-scope PostToolUse:Edit checks to the changed hunk (whole-file only for a new file); match structural producers, not token co-occurrence (block-hook-bypass: false-fires when a Bash command contains both an echo token and a redirect (script-stdout capture, or quoted-string arguments) rather than an actual file-authoring bypass #546); ignore tokens inside quoted args (block-hook-bypass: false-fires when a Bash command contains both an echo token and a redirect (script-stdout capture, or quoted-string arguments) rather than an actual file-authoring bypass #546); bound stdin reads (guardrails: cli-flag-verify.sh still reads fd0 unbounded (last Win32-stall stdin site) #446); either/or marker logic where "already-canonical → stay quiet" is the intent (guardrails:flag-commit-pr-skill-bypass: advisory false-fires on canonical git commit -F - with inline heredoc Co-Authored-By trailer (requires BOTH markers, contradicting documented "either marker -> stay quiet" intent) #545); path-branch narrowing (guardrails: hardcoded-path-check false positives — repo-path branch flags all paths under home; percent-env interpolations flagged #389/guardrails: hardcoded-path-check — Windows-aware remediation, inline opt-out for machine-scoped files, document gitignore limitation #390). Make "every filed FP becomes a MUST-stay-quiet case" a documented gate. Tradeoffs: scoped to a doc + discipline, not a build — the harness exists, so this is days not weeks. Future-state risk: low; the convention is additive and each new hook inherits it. Reversibility: fully reversible (doc + tests, no runtime behavior forced). Rationale: re-derived from the verified state — the umbrella's own goal (stop recurrence) is met by convention + fixture-discipline, and the "build a harness" scope in the body would duplicate what CI already runs.
• B — Fix the 6 members individually now, add a stay-quiet fixture per fix, skip the convention doc. Tradeoffs: fastest to green on the open members; no reusable discipline, so the next hook author has nothing to inherit and the class recurs — which is exactly what this umbrella indicts. Reversibility: n/a (just patches). Weaker on the stated goal.
• C — Treat as greenfield: build a fixture harness + convention from scratch. Tradeoffs: matches the issue body literally but duplicates run-plugin-tests.sh / guardrails-test-helpers.sh / the CI gate that already exist; wasted effort and a second thing to maintain. Not recommended.

Related.

• guardrails: cli-flag-verify PostToolUse hook scans whole file, warns on pre-existing lines outside the edited hunk #543 diff-scope (primary of the guardrails: cli-flag-verify PostToolUse hook scans whole file, warns on pre-existing lines outside the edited hunk #543/guardrails: cli-flag-verify PostToolUse hook flags pre-existing unrelated lines (not diff-scoped) #544 pair) · guardrails: cli-flag-verify PostToolUse hook flags pre-existing unrelated lines (not diff-scoped) #544 dup of guardrails: cli-flag-verify PostToolUse hook scans whole file, warns on pre-existing lines outside the edited hunk #543 · guardrails:flag-commit-pr-skill-bypass: advisory false-fires on canonical git commit -F - with inline heredoc Co-Authored-By trailer (requires BOTH markers, contradicting documented "either marker -> stay quiet" intent) #545 ||→&& boolean-intent · block-hook-bypass: false-fires when a Bash command contains both an echo token and a redirect (script-stdout capture, or quoted-string arguments) rather than an actual file-authoring bypass #546 echo+redirect quoted-arg/structural · guardrails: cli-flag-verify.sh still reads fd0 unbounded (last Win32-stall stdin site) #446 unbounded stdin · guardrails: hardcoded-path-check false positives — repo-path branch flags all paths under home; percent-env interpolations flagged #389/guardrails: hardcoded-path-check — Windows-aware remediation, inline opt-out for machine-scoped files, document gitignore limitation #390 path-branch — the 6 members; recommend they land as the first fixtures under Option A.
• source-control:babysit-prs: one shared authorship/finding/approval classifier + golden fixtures; gate script owns the merge-ready boolean (6-issue class) #534 — babysit merge-readiness umbrella: same fixtures-as-prevention philosophy (shared classifier + golden fixtures, each member becomes a fixture), different plugin.
• convention: false-green checks — surfaces report success while the capability is dead or findings invisible (liveness-assertion umbrella) #532 — false-green umbrella: the inverse failure mode (checks under-fire / hide findings vs these over-fire); shared answer is "document the contract as a convention, then test/enforce it" — its "liveness-assertion convention" is the mirror of this one's precision convention.
• work-items: shared dogfood-filing skill — cross-lane-identical self-observation rule must not be absorbed as three private copies #540 — shared dogfood-filing dedupe (the guardrails: cli-flag-verify PostToolUse hook scans whole file, warns on pre-existing lines outside the edited hunk #543/guardrails: cli-flag-verify PostToolUse hook flags pre-existing unrelated lines (not diff-scoped) #544 ~3s concurrent duplicate is its motivating example).

[kyle-sexton]

Operator decision (2026-07-19, decision-session batch 3; relayed).

ACCEPTED — the decision brief's RECOMMENDED option verbatim.

Re-intake: removing needs-human, adding priority: needs-triage for routing.

[kyle-sexton]

This was generated by AI during triage.

Routing status: ready per ratified decision. Operator ACCEPTED Option A verbatim: the per-hook fixture harness already exists and is CI-gated (*.test.sh per hook + guardrails-test-helpers.sh + run-plugin-tests.sh, gated in ci.yml) — the real gap is (1) a documented hook-precision convention sibling to docs/conventions/hook-telemetry/, and (2) the discipline that every filed production false-positive becomes a new MUST-stay-quiet fixture case in that hook's existing test file, not a fresh one-off. The 6 members (#543, #544, #545, #546, #446, #389, #390) land as the first conforming fixes under this convention, not as a from-scratch harness build (Option C explicitly rejected — would duplicate existing CI infra).

Priority: not in the explicit scaling-chain list — downgraded from priority: high to priority: medium. wayfind: design removed (approach settled).