Repository navigation
Description
The output of tracekit report --share or tracekit audit --share
Tracekit 1.5.3 field report (tracekit report --share): 7 days
Integrity: the store verified, 408 of 408 runs verify
Runs that verify, by agent: claude-code 408
Tool calls: 25,746 allow 17,244, flag 8,382, ask 76, deny 44
Holds: 70 expired 63, approved · ran 6, not answered yet 1 (6 of the agent's retries of a held call, counted with it)
Gaps (signed records that something may be missing): 3,910 reconcile.fabricated 3,703, tailer_lost 121, reconcile.hook_missing 42, decided_not_completed 41, reconcile.args_mismatch 2, reconcile.result_without_call 1
Denied and flagged calls, by rule (one call can match several rules):
deny TK-D002 34
deny TK-D003 3
deny TK-D010 3
deny TK-D007 2
deny TK-D008 2
deny TK-SHELL-PARSE 2 (not placed)
deny TK-F001 34 (also matched, not why)
deny TK-F012 33 (also matched, not why)
deny TK-F013 33 (also matched, not why)
deny TK-F003 4 (also matched, not why)
deny TK-F004 3 (also matched, not why)
deny TK-A046 2 (also matched, not why)
deny TK-F005 2 (also matched, not why)
deny TK-F007 1 (also matched, not why)
deny TK-F014 1 (also matched, not why)
flag TK-F006 2,821
flag TK-UNKNOWN-TOOL 2,634 (not placed)
flag TK-F002 2,066
flag TK-F012 1,831
flag TK-F011 458
flag TK-F003 249
flag TK-F005 214
flag TK-F008 142
flag TK-F004 131
flag TK-F001 127
flag TK-F013 121
flag TK-F015 42
flag TK-F009 41
flag TK-OVERSIZE 17 (not placed)
flag TK-F007 2
flag TK-F016 2
flag TK-F017 1
Which agent or framework, and what kind of work
Claude Code 2.1.323, mostly the desktop app's Code tab, with several sessions in parallel. The work was a mix: web app and website development, Python and Go backends, and open-source contributions (this one included: the OpenHands harness, #563/#662).
How long, and on what
7 days (6–13 Nov) on a macOS 27 laptop (Apple silicon). Tracekit 1.5.3, installed with uv tool install (Python 3.12), tracekit init --dev --v2.
Anything you'd like to tell us (optional)
What went well: all 408 runs verify, and the hook never got in the way of a normal call. The denies I saw were right: a force-push (TK-D003) was blocked, and I pushed it myself, as the message said to. tracekit audit --since 7d beforehand was a good way to see what to expect.
Holds mostly expired (63 of 70). In the desktop app, a held call returns at once with the approve command, and the agent usually reworded the call or moved on rather than waiting. So most holds were never answered. That isn't wrong, but a shorter "approve this, or tell me to skip it" in the agent's message might get more of them answered.
Gaps: 3,703 reconcile.fabricated. That's most of the gaps, and I don't think the agent fabricated anything. My guess is calls the hook didn't see: parallel sessions in the desktop app, or sub-agents (Task). I can help narrow it down if you tell me what to look at.
False alarms I hit:
- TK-SHELL-PARSE holds on ordinary commands that use shell arithmetic (
$((…))) or a variable for a path (cd $DIR && …). Writing the command out plainly got through, as the message suggests, so it's friction rather than a blocker. - TK-D007 ("touching Tracekit's own files or signer") denied a
python -c "import tracekit.signer…"run inside a dev checkout of the Tracekit repo. That's the repo's source, not the installed signer. It's probably rare outside Tracekit contributors, but working on Tracekit under Tracekit trips it.
Before you post
- I removed anything private (secrets, hostnames, customer or project names, paths) from this report.
Activity
LoopGlitch26 commentedon Nov 13, 2026
Thank you. This is the first outside week we have, and it's exactly the kind of report that changes what we work on.
The 3,703
reconcile.fabricatedgaps are ours, not your agent's. We measured the same thing on our own Claude Code store. Nearly all of them are tool calls whose transcript lines the tailer never recorded, so reconcile can't match them. It has three causes:tailer_lostfit this);Parallel sessions and Task sub-agents make all three more common, as you guessed. The fix is starting now; I'll link the PR here. Then a re-run of
tracekit audit/reporton your setup would be the best check of it, if you're willing.On the rest:
$((…))andcd $DIR && …held by TK-SHELL-PARSE: noted. They're on our list for the shell reader, which holds what it can't fully read on purpose, so this needs care rather than a quick loosening.python -c "import tracekit.signer…"in a Tracekit checkout: right, that's a false alarm for anyone working on Tracekit itself. I'll open it as an issue too.The flag counts (TK-F006, TK-UNKNOWN-TOOL, TK-F002) also tell us where flags are noisiest on real work, which helps.
LoopGlitch26 commentedon Nov 13, 2026
Two follow-ups:
tracekit.signeryou import is the code that runs and signs the dev signer. The policy also can't trust the working directory, because the agent chooses it. So it can't tell a contributor's quick check apart from an agent reaching into its own signer. Code that only reads the RPC tables,reconcile.compared/model_names, orload_policyis allowed. Anything more is denied on purpose, as theTK-D007row indocs/limits.mdsays. When working on Tracekit itself, run that code through the tests. So no issue for this one.The fix for the fabricated gaps is still in progress. I'll link the PR here.
LoopGlitch26 commentedon Nov 13, 2026
The fix for the fabricated gaps is #818.
aj-codespy commentedon Nov 13, 2026
@LoopGlitch26 Thanks for digging into it so quickly, and for opening #816.
Yes, happy to re-run it. Once #818 is merged I'll update, keep running Tracekit on the same setup (parallel desktop sessions and Task sub-agents), and post
tracekit audit/tracekit report --sharehere so you can compare thereconcile.fabricatedandtailer_lostcounts with this week's.On TK-D007: that makes sense. With an editable install that import really is the signer that runs, so I'll go through the tests for that.
Holds: with an Instead line, the one rule lets the agent take its own…
Claude Code: start a stopped transcript tailer again, park an idle one (
LoopGlitch26 commentedon Nov 13, 2026
The fix for the fabricated gaps is merged (#818). A transcript reader that stopped is now started again after one signed gap and reads on from where it stopped. One that went quiet pauses instead of giving up after 2 hours, and the next event starts it again. On a scripted run shaped like your setup (parallel sessions, a sub-agent, a workflow agent and a killed reader), false
reconcile.fabricateddropped from 44 to 0, and the planted real gaps were all still caught. What's not covered yet: calls Claude Code runs without a PreToolUse hook, and calls decided in another run of the same session. Your 42hook_missingare those.To re-run on the same setup, either wait for 1.7.0 (soon), or install main now:
then restart your agent sessions so the hooks pick it up.
One exact ask, since you offered. Your report had 2,634
TK-UNKNOWN-TOOLflags, and our own week has 130. Could you post the tool names behind them, names and counts only, with any private MCP server names replaced?That tells us which tools to map, so they get decided instead of flagged.
Holds: ask the person "approve or skip?" so held calls get answered (#…