Skip to content

tracekit audit: a dedicated error for a call an agent's hook can't read #771

Closed

Description

@LoopGlitch26

tracekit audit --calls FILE --agent NAME replays each call as that agent's Tracekit hook would map it. When the mapping raises any ValueError, audit reports the call as one the hook blocks (TK-HOOK-BLOCKED). That is meant for calls a mapping can't read on purpose, but it catches more.

  • The replay loop: except ValueError: # a call the mapping can't read (Cline's): its live hook blocks it, then blocked = True (tracekit/audit.py L257-258). The try above it (L249-253) covers _blocked(...), json.dumps(...) and hook_tool(...) (agent_hooks.tool, imported at L21).
  • Cline's mapping raises ValueError on purpose, at six places in tracekit/agent_hooks.py: _cline_commands (L263, L268) and the read_files, editor, apply_patch and fetch_web_content branches of tool() (L478, L486, L498, L503). Both Cline hooks block these calls. The v1 hook's command names its event (_install_cline, L1555-1556), so entry blocks a pre-tool call whatever the fail mode (L1750). The v2 hook blocks on any error (tracekit/integrations/harness_hooks.py L88-91).
  • goose's read_image raises one by accident. urllib.parse.urlsplit(src.replace("\\", "/")) (tracekit/agent_hooks.py L437; urlsplit(url, scheme='', allow_fragments=True) raises ValueError("Invalid IPv6 URL"), Python 3.11 urllib/parse.py L470, L514) fails on a source like http://[::1/cat.png, and audit reports the call as TK-HOOK-BLOCKED. The v2 goose hook does block it. The v1 goose hook's command names no event (_command, L1312-1314), so under the default fail-open mode it prints [tracekit] goose hook error (Invalid IPv6 URL); allowing (fail-open) and the call runs.
  • Any other ValueError in that try, from a real bug, would also be reported as the hook blocking the call, which hides the bug.
  • docs/audit.md L108-109 says "with --agent cline or --agent devin, a call that agent's hook can't read is denied (TK-HOOK-BLOCKED)". goose isn't named. docs/coding-agents.md L194-198 lists three cases a v1 hook blocks whatever the fail mode (an OpenCode call, an Antigravity CLI pre-tool call, a payload nested too deep). A Cline pre-tool call its hook can't decide is a fourth: like Antigravity's, its command names the event. (The Cline column at L157 already says such a call is cancelled.)

To see it on main:

echo '{"tool":"read_image","args":{"source":"http://[::1/cat.png"}}' > calls.jsonl
tracekit audit --calls calls.jsonl --agent goose

The report shows TK-HOOK-BLOCKED: 1 (the agent's Tracekit hook denies this call before the policy decides it).

What's needed

  • In tracekit/agent_hooks.py, an exception for a call a mapping deliberately can't read, for example class UnreadableCall(ValueError). Subclassing ValueError changes nothing for code that catches ValueError or Exception today (the hooks catch Exception).
  • Cline's six raise ValueError(...) sites raise it instead, with the same messages.
  • goose's read_image branch catches urlsplit's ValueError and raises UnreadableCall with a plain message (for example "goose's read_image source isn't a URL Tracekit can read"), so audit and the v2 hook keep denying it, now on purpose.
  • audit() catches UnreadableCall instead of ValueError (tracekit/audit.py L257). Any other ValueError then ends the replay with its message (exit 2, L430) instead of turning into a deny.
  • docs/audit.md L108-109 names the agents whose hook denies a call it can't read: Cline, Devin (already there) and goose (a read_image source Tracekit can't read as a URL). For goose, say this is the v2 hook's answer: the v1 hook allows such a call under the default fail-open mode.
  • docs/coding-agents.md L195-198: add "a Cline pre-tool call its hook can't decide (its command names the event)" to the cases a v1 hook blocks whatever the fail mode.

Acceptance

  • tests/test_audit.py: --agent goose with the read_image line above and one plain call: the first is denied with TK-HOOK-BLOCKED, the second replays, exit 0.
  • tests/test_audit.py: a ValueError that isn't UnreadableCall from the mapping (patch audit.hook_tool to raise one) is not reported as TK-HOOK-BLOCKED: audit() raises it, and the CLI exits 2 with its message.
  • tests/test_agent_hooks.py: each of Cline's six cases, and the goose read_image source above, raises UnreadableCall. Keep http://[::1/cat.png (an unclosed bracket) as the goose input: it raises on every Python we support, while a bracketed host that isn't an IP (http://[x]/a.png) raises only on patch releases that have the newer bracket check.
  • test_a_call_cline_s_hook_cant_read_is_blocked_as_the_hook_blocks_it (tests/test_audit.py L433) and test_agent_denies_what_its_hook_denies_before_mapping (L475) pass unchanged.

Comment here to claim it. Commits need a DCO sign-off (git commit -s).

Activity

  1. khushi-git-7 commented on Nov 12, 2026

    @khushi-git-7
    Contributor

    I would like to take this one. Plan: an UnreadableCall(ValueError) in agent_hooks.py, raised at Cline's six sites and from goose's read_image when urlsplit fails; audit() catching only that, so any other ValueError ends the replay with exit 2; the two doc changes; and the tests listed under Acceptance.

  2. LoopGlitch26 commented on Nov 12, 2026

    @LoopGlitch26
    MemberAuthor

    Assigned to you. The plan reads right; keep goose's read_image case raising the new exception too, so audit still denies it, and let any other ValueError end the replay with exit 2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions