Skip to content

tracekit audit: any agent's calls from one plain file, and each known agent's sessions read without a folder #714

Closed

Description

@LoopGlitch26

tracekit audit replays the tool calls in Claude Code's transcripts through the policy engine (tracekit/audit.py), and #669 adds Codex's rollouts. Every other agent writes its sessions in a format of its own, so each needs a reader, and an agent with none (or a framework agent with only its own logs) can't be audited at all. docs/limits.md L869 says so.

Proposal

Two parts, both in tracekit/audit.py, so that any agent can be audited today and the common ones need no work from the user.

  1. One plain calls file that any agent or log can produce. tracekit audit --calls FILE (- for stdin) reads JSON lines:

    {"time": "2026-11-10T14:02:11Z", "session": "s1", "tool": "Bash", "args": {"command": "rm -rf build"}}

    tool (a string) and args (an object) are required. time (ISO-8601) and session are optional; a call without a time is kept, as now. A line without tool and args counts as malformed. With --agent NAME (one of agent_hooks.AGENT_NAMES), each (tool, args) first goes through agent_hooks.tool(NAME, tool, args), the mapping the live hooks use, so an agent's own names (run_shell_command, Shell, bash) are decided as its hook would decide them. Without it, names are taken as Tracekit's (Bash, Write, Edit, Read, WebFetch, mcp__<server>__<tool>). Each call is decided with evaluate() as the loop does today, so the report, --json and --share are unchanged.

  2. A table of readers, read without a ROOT. One entry per agent: its name, its default folder (with the env var that moves it) and a reader (root, since_ts, unreadable) -> iterable of (time, session, tool, args). Claude Code's reader is today's loop. tracekit audit: read Codex session rollouts #669's per-line detection can stay inside it, or Codex can get an entry of its own, whichever reads cleaner. With no ROOT and no --calls, audit reads every entry whose folder exists. Its first line names each agent with its calls and sessions, and --json gives each call an agent. A ROOT given on the command line keeps today's meaning: Claude Code's folder, unless --agent NAME names another reader. Readers for other agents follow in their own issues and add one entry each.

Acceptance

  • Tests in tests/test_audit.py, written as _call writes Claude Code lines:
    • a calls file with a denied call, a held call, a line without tool (counted malformed), a --since cut, and stdin;
    • --agent gemini with run_shell_command decides the same as agent_hooks.tool("gemini", …) does;
    • no ROOT, with HOME set to a temp dir that holds only a Claude Code folder, reads that folder, names it, and doesn't fail on the agents whose folders aren't there.
  • docs/audit.md documents the calls format, with one jq example that turns some log into it. docs/limits.md L869 says which agents have readers and that any other agent can use --calls.
  • Read-only like the rest of tracekit audit: no signer, no writes, no network.

Comment here to claim it. Commits need a DCO sign-off (git commit -s).

Activity

  1. LoopGlitch26 commented on Nov 11, 2026

    @LoopGlitch26
    MemberAuthor

    Readers that plug into this: #669 (Codex), #715 (Gemini CLI), #716 (Cursor), #717 (Cline).

  2. samueladebisi-dev commented on Nov 11, 2026

    @samueladebisi-dev
    Contributor

    I’d like to work on this. I’ll add --calls and the reader table with the tests from the acceptance list, and open a PR.

  3. LoopGlitch26 commented on Nov 11, 2026

    @LoopGlitch26
    MemberAuthor

    It's yours, @samueladebisi-dev. Two things to plan around: #669 (Codex's reader, with kausthubhk) may land first, and its per-line detection can stay inside Claude Code's reader as the issue says; and #715-#717 add one table entry each, so keep the reader signature exactly as written so they can start from it.

  4. LoopGlitch26 commented on Nov 11, 2026

    @LoopGlitch26
    MemberAuthor

    For the readers that plug in here (#715-#717): as #730 builds it, a reader yields (time, session, tool, args) for each call and None for a line it can't read, which audit counts as malformed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

enhancementNew feature or requesthelp wantedExtra attention is needed

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions