fix(guardrails): diff-scope cli-flag-verify to the tool payload - #780
Conversation
The PostToolUse check re-read the whole edited file from disk, so any edit to a file already containing an unrecognized flag elsewhere re-fired the advisory about untouched lines. Scan the tool payload instead: an Edit's changed hunk, a Write's full content (the payload cannot distinguish new-file from overwrite). Repro-first stay-quiet and MUST-FIRE cases added; harness converted to payload-carrying invocations with disk content kept equal to payload pre-fix to prove the conversion behavior-preserving. Guardrails 0.9.3 -> 0.9.4. Closes #543 Closes #544 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 2m 50s —— View job Security Review — PR #780
Scope: diff-scope fix that shifts No security issues found.The three new data-flow paths introduced are all safe: 1. TOOL=$(printf '%s' "$INPUT" | jq -r '.tool_name // empty' 2>/dev/null | tr -d '\r')
case "$TOOL" in
Edit) ... ;;
Write) ... ;;
*) exit 0 ;;
esac
2. Payload content extraction → SCAN_CONTENT=$(printf '%s' "$INPUT" | jq -r '.tool_input.new_string // empty' 2>/dev/null | tr -d '\r')
3. Verifier invocation with payload-derived tokens The pipeline is
All three are passed as properly double-quoted arguments. No command injection path. Attack-surface reduction vs. the pre-fix code The old hook passed Advisory-only blast radius The hook exits 0 in every branch. Even a complete logic failure produces, at worst, a missed detection or a spurious advisory message — not a block, credential leak, or code execution. No findings to report. |
|
Claude finished @kyle-sexton's task in 2m 14s —— View job PR Review:
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: db5760e7b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The typos gate tokenizes the hyphenated prefix as a bare "mis". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 2m 8s —— View job PR #780 Review
SummaryThe core fix is correct and sound: moving from One residual (below) is not yet documented in-code or tested. Finding: Codex P2 — partial-hunk Edit misses flag-only replacements (undocumented residual)Status: Real, inherent to the approach, but should be documented alongside the fence-straddling note. When an Edit replaces only a flag token — This is not a regression (the pre-fix hook would have scanned the whole file and potentially fired on unrelated lines); it's a new false-negative class that is inherent to payload-only scoping. It belongs in the documented residuals block at line 118–122 alongside the fence-straddling trade-off: The test at line 157 only covers the full-command hunk shape ( # flag-only hunk: no binary context → silent skip (accepted residual)
OUT=$(run_edit 'faketool sub --real' '--fake'); RC=$?
assert_exit "diff-scope: flag-only hunk → exit 0" 0 "$RC"
assert_silent "diff-scope: flag-only hunk → no advisory (no bin context)" "$OUT"Without this test, a future attempt to "fix" the miss by reading from disk would break the diff-scope contract without an explicit signal that the silence is intentional. Consider adding the comment + test → Everything else looks correctDispatch block (lines 70–75): Empty-content guard (line 76): Three separate
Test helpers updated correctly:
CHANGELOG / version bump: VerdictApprove with the suggestion to add one comment and one test case to document the partial-hunk residual. The fix itself is correct, and the overall improvement (no more whole-file re-reads, no more pre-existing-flag false positives) is a clear win. The undocumented residual is a known limitation of payload-only scoping — the only question is whether future maintainers will know it's intentional. |
|
Claude finished @kyle-sexton's task in 2m 53s —— View job Security Review — PR #780
Scope: Two commits — No security issues found.I independently verified the four data-flow paths introduced or changed by this PR: 1. TOOL=$(printf '%s' "$INPUT" | jq -r '.tool_name // empty' 2>/dev/null | tr -d '\r')
case "$TOOL" in
Edit) ... ;;
Write) ... ;;
*) exit 0 ;;
esac
2. SCAN_CONTENT=$(printf '%s' "$INPUT" | jq -r '.tool_input.new_string // empty' 2>/dev/null | tr -d '\r')
3. Attack-surface reduction vs. the pre-fix code The old hook passed 4. Downstream verifier invocation Line 262:
All three are passed as separately double-quoted arguments. No command injection path. 5. Typo commit ( Changes Advisory-only blast radius The hook exits 0 in every branch. Even a complete logic failure produces, at worst, a missed detection or a spurious advisory message — no block, credential leak, or code execution. No findings to report. The diff-scope change is correct and strictly reduces the pre-existing attack surface. |
An Edit whose new_string is a bare flag fragment carries no binary in the changed region, so the payload-scoped scan found no command candidate and silently missed a genuinely swapped-in unknown flag — narrowing false positives must not weaken detection. When the hunk yields flag-like tokens but zero candidates, pull only the on-disk lines carrying those tokens (bounded) and re-scan, then filter candidates to flags present in the hunk so pre-existing unrelated flags never re-fire. Repro-first: the reconstruction MUST-FIRE case fails pre-fix; suite 45/45. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 3m 18s —— View job Security Review — PR #780 (commit
|
|
Claude finished @kyle-sexton's task in 3m 15s —— View job PR #780 Review — Commit
|
| Test | Shape | Expected |
|---|---|---|
run_edit 'faketool sub --fake' 'faketool sub --real' |
Unknown flag on disk, clean hunk | Silent (regression guard for original bug) |
run_edit 'faketool sub --real' 'faketool sub --fake' |
Unknown flag introduced by hunk | Fires |
run_edit 'faketool sub --fake' '--fake' |
Bare-flag hunk, flag already on disk (PostToolUse) | Fires (reconstruction path) |
run_edit 'faketool sub --real --otherbogus' '--real' |
Bare-flag hunk; unrelated flag shares disk line | Silent (diff-scope contract through reconstruction) |
The repro-first claim holds: test 3 is the MUST-FIRE that fails against the pre-reconstruction hook and passes after. Test 4 is the MUST-STAY-QUIET counterpart that directly encodes the filter invariant.
Everything Else
head -20bound on disk context — Limits blast radius for a hunk flag that appears on many disk lines (e.g., a common flag like--verbose). Correct.mapfile -t— Requires bash 4.0+, whichdeclare -Aalready mandates. No new dependency.- CHANGELOG / version — 0.9.3 → 0.9.4 is correct for a bug-fix that changes scanning behavior. Entry accurately describes both the diff-scope fix and bounded context reconstruction.
is_skipped "$bin" && continue— Pre-existing per-binary opt-out is respected through reconstruction sinceextract_candidatesalready gates on this.
Verdict
Approve. The bounded context reconstruction correctly addresses the P2 gap — a swapped-in unknown bare flag now fires — while preserving the diff-scope contract: a pre-existing unrelated flag sharing the disk line stays quiet. The implementation is tight, the tests are repro-first, and the accepted trade-offs are documented in-code.
Closes #543
Closes #544 (duplicate pair — same defect, fuller repro on #544)
The PostToolUse
cli-flag-verifycheck re-read the entire edited file from disk, so any edit to a file already containing an unrecognized flag elsewhere (dotnet test --projectin prose, 43 lines from the edit in #544's repro) re-fired the advisory about lines the edit never touched. The hook now scans the tool payload — an Edit'snew_stringhunk, a Write's fullcontent(a PostToolUse Write payload cannot distinguish new-file from overwrite, so whole-content is the closest the payload allows) — landing the hook-precision convention's diff-scoping rule (rule 1) on its first member.Repro-first per the convention: converted suite vs unmodified hook = PASS=39 FAIL=2 (exactly the two new diff-scope discriminators — pre-existing-flag stay-quiet fired, hunk-flag missed); fixed hook = PASS=41 FAIL=0. Harness converted to payload-carrying invocations with disk content kept equal to payload so the 39 pre-existing assertions prove the conversion behavior-preserving. shellcheck clean. Guardrails 0.9.3 → 0.9.4.
Documented residuals (in-code): markdown fence state derives from the hunk alone — a fence-straddling edit can mis-classify in either direction; the convention's accepted trade. Live matcher is
Write|Edit; other tools exit 0.Related
🤖 Generated with Claude Code