Skip to content

feat: add tracekit audit command to replay local transcripts - #491

Merged
LoopGlitch26 merged 10 commits into
Cygnux-Labs:mainfrom
HarshRajSinghania:feat/audit-replay
Nov 1, 2026
Merged

LoopGlitch26 merged 10 commits into
Cygnux-Labs:mainfrom
HarshRajSinghania:feat/audit-replay

Conversation

@HarshRajSinghania

Copy link
Copy Markdown
Contributor

Implements the read-only tracekit audit command from issue #490.

Replays Claude Code tool calls through the policy engine and reports what would have been held or denied. Local only, no signer, no writes.

Includes tests and docs. DCO signed.

Fixes #490.

Harsh Raj Singhania and others added 6 commits November 1, 2026 21:09
Implements the read-only audit command requested in Cygnux-Labs#490.
Reuses transcript reading and the policy2 engine.
Adds tests, docs/audit.md and a limits.md row.

Signed-off-by: Harsh Raj Singhania <harshrajsinghania@proton.me>
Add the read-only audit command that replays Claude Code transcripts through the policy engine.

Signed-off-by: Harsh Raj Singhania <harshrajsinghania@proton.me>
Cover held/denied/allowed calls, malformed lines, since filter and unreadable paths.

Signed-off-by: Harsh Raj Singhania <harshrajsinghania@proton.me>
Signed-off-by: Harsh Raj Singhania <harshrajsinghania@proton.me>
Add the audit subcommand and its dispatch.

Signed-off-by: Harsh Raj Singhania <harshrajsinghania@proton.me>
…--show redacts secrets; stronger tests; limits and support rows

Signed-off-by: LoopGlitch26 <research@cygnuxlabs.com>
@LoopGlitch26

Copy link
Copy Markdown
Member

@HarshRajSinghania thanks for picking this up. I pushed one commit on top (5415c1f) so it can merge:

  • Your last commit replaced tracekit/cli.py with a placeholder line. I restored it and added audit as a delegated, experimental subcommand: tracekit audit [ROOT] [--policy] [--since] [--show] [--json], with main() in audit.py.
  • The root is compared by its real path, so a root under a symlink (macOS /var, a linked ~/.claude) no longer skips every file.
  • --show now redacts secrets with privacy.redact_text. The issue said core.scrub, which doesn't redact; that was my mistake, and I've fixed the issue text.
  • The tests check exact counts, an unreadable file, a symlink out of the folder, --since with relative times (the fixed date would have failed after 9 November), redaction and the CLI. They also import on Python 3.9 now.
  • docs/limits.md has an Audit section, and docs/support.md a row.

CI is running, and a second review reads it next. If you'd rather change any of this, say so here.

LoopGlitch26 added 4 commits November 1, 2026 23:08
…t, escaped output, unreadable folders counted, wiki sidebar, limits rows

Signed-off-by: LoopGlitch26 <research@cygnuxlabs.com>
Signed-off-by: LoopGlitch26 <research@cygnuxlabs.com>
…st denied and held calls (--all for every call)

Signed-off-by: LoopGlitch26 <research@cygnuxlabs.com>
Signed-off-by: LoopGlitch26 <research@cygnuxlabs.com>
@LoopGlitch26
LoopGlitch26 merged commit 1832064 into Cygnux-Labs:main Nov 1, 2026
33 checks passed
@LoopGlitch26

Copy link
Copy Markdown
Member

Merged, thanks @HarshRajSinghania. It goes out in the next release.

What I changed on top, so you know what's in main:

  • --show redacts as the signer does (API keys, and every value in a .env write); core.scrub alone didn't.
  • A call the engine fails on counts as a deny (TK-ENGINE-ERROR), as the signer decides it.
  • Files are opened without following symlinks, and unreadable ones are counted, not fatal.
  • The report opens with one summary line, gives each rule's meaning, and lists only denied and held calls unless --all.
  • Control characters from a transcript are printed as text; a subagent's calls count under its session.
  • cli.py only delegates to tracekit/audit.py; the page is in the wiki list, support and limits.

If you'd like a next one, #495 extends audit to Codex session rollouts. The mapping from a rollout call to a tool use already exists in harness_hooks.py, so it's mostly reuse.

@LoopGlitch26

Copy link
Copy Markdown
Member

Thanks again for contributing to Tracekit. A request: if you use a coding agent (Claude Code, Codex, Cursor, Gemini CLI or OpenCode), would you run Tracekit on it for a week and tell us how it went?

pip install -U tracekit-ai
tracekit audit --since 7d      # read-only: what it would have held in your last week (Claude Code history)
tracekit init --dev --v2       # then record and guard from now on

After a week, tracekit report --share prints counts only (no commands, paths or content). Post it in #385 with anything that got in your way: a false alarm, a confusing message, something slow. If you find a problem, you're welcome to fix it too: say so on #385 and it's yours.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tracekit audit: replay your coding agents' past tool calls through the policy and report what it would have held

2 participants