Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/coding-agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -520,7 +520,9 @@ these rules don't see (a script that calls a payment SDK, another provider): [li

For Claude Code, the transcript tailer records every line the model writes (thinking, text, tool use) and every prompt
as a model exchange: a commitment to the whole line, and the line itself when the signer keeps content (the dev signer
does). `tracekit init --v2` also sets `"showThinkingSummaries": true` in the Claude Code settings it writes the hooks
does). Each message's token usage (input, output, cache-read and cache-write tokens) is signed in clear once, at its
final counts, as the tailer reads it: `tracekit tips` counts those signed figures before the transcript's own
([limits](limits.md#signer-and-capture)). `tracekit init --v2` also sets `"showThinkingSummaries": true` in the Claude Code settings it writes the hooks
to, or with the plugin enabled in the user's settings (unless they set it; `--no-thinking-summaries` leaves it alone;
`tracekit uninstall` removes it again while it's still `true`): without it Claude Code keeps the model's thinking
only sealed. For headless runs pass `claude -p --thinking-display summarized`. `tracekit analyze --v2`
Expand Down
4 changes: 3 additions & 1 deletion docs/limits.md

Large diffs are not rendered by default.

6 changes: 5 additions & 1 deletion docs/privacy.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,10 @@ Both settings are off by default, recorded in each run's `run.start`, and the ve
| `tool.result` | output | hash | redacted text |
| `tool.result` | ok, duration_ms | clear | clear |
| `model.message` | content (only with `reasoning_capture`) | hash | redacted text |
| `model.exchange` (proxy) | model, status, stop reason, tool_use ids and names, timing | clear | clear |
| `model.exchange` (proxy) | model, status, stop reason, tool_use ids and names, timing, token usage | clear | clear |
| `model.exchange` (v2: LLM gateway) | model, stop reason, tool_use ids and names, usage (input, output, cache-read and cache-write tokens), the record's signed time | clear | clear |
| `model.exchange` (v2: Claude Code transcript tailer) | model, the message id, usage (input, output, cache-read and cache-write tokens, once per message), each tool use's id and name, the ids of the tool results sent since the last response (`tool_results_sent`), the record's signed time | clear | clear |
| `model.exchange` (v2: Claude Code transcript tailer) | the transcript line: the model's thinking, text and tool uses, the line's own timestamp | salted commitment; the line is kept only in the [content vault](#the-content-vault), when the signer keeps content | same |
| `model.exchange` (proxy) | request body, response body | hash (after redaction) | redacted text |
| `model.exchange` (proxy) | HTTP headers, including `x-api-key` / `authorization` | **never recorded** | **never recorded** |
| every hook event | `transcript` mark: path, length, sha256 of the transcript | clear (hash only, never content) | same |
Expand All @@ -62,6 +65,7 @@ Redaction is pattern-based and will miss secrets it doesn't recognise. A command
- **Paths and commands are in clear** (after redaction): file names, URLs, search queries and whole command lines can themselves be sensitive.
- **Hashes of low-entropy content can be guessed**: a hashed "yes", a short prompt or a known file can be confirmed by hashing candidates. Hashes are unsalted so reviewers can match known files; treat them as confirmable. The v2 signer publishes salted commitments instead (below).
- **Sizes and timing**: content sizes, tool durations and timestamps reveal activity patterns even when content is hashed.
- **Token counts**: a model response's usage is in clear (the gateway's records, and the transcript tailer's for Claude Code), so its output tokens say how long each reply was, and its input and cache counts how large the context was, even where the text is only committed.
- **Correlation**: the same file hashed in two runs links them; tool_use ids link proxy and hook events by design.
- **Witnesses** learn when checkpoints happen and how many records exist, nothing else.

Expand Down
21 changes: 15 additions & 6 deletions docs/viewer.md
Original file line number Diff line number Diff line change
Expand Up @@ -360,12 +360,21 @@ to paste.
| 3 · HOLDS NOBODY ANSWERED | held calls answered and not (the agent's retry of a held call counted with it, as PERMISSIONS and the run review count them), in the main session and in subagents, how long they blocked the agent, and the rules behind the ones nobody answered | when they waited: remove `TRACEKIT_APPROVAL_WAIT` from the `env` block of the Claude Code settings that set it (`~/.claude/settings.json` or `$CLAUDE_CONFIG_DIR/settings.json`, the project's `.claude/settings.json` or `.claude/settings.local.json`; **COPY** gives the `jq` edit of the first), unset it in the agent's shell, then restart the agent; otherwise the rule's remedy and `tracekit approvals list` |

Token counts come from signed records where they carry them (a `model.exchange` response's `usage`: the LLM gateway's,
an SDK integration's). Claude Code's transcript tailer records no usage, so for Claude Code the viewer reads its
transcripts, read-only, as `tracekit audit` reads them: `--transcripts DIR`, by default with the dev store
`$CLAUDE_CONFIG_DIR/projects` or `~/.claude/projects`. A signed step the transcripts also hold counts once, from the
transcripts: it is matched step by step, by a tool_use id or by identical token counts within 5 minutes of one
transcript message no other step matched (the gateway signs each exchange under an id of its own, and a message
without tool calls has no tool_use id). The page's first line says how many signed steps that was, and the time span
an SDK integration's, and Claude Code's transcript tailer's, which signs each turn's usage as it reads the transcript),
else from Claude Code's transcripts, read-only, as `tracekit audit` reads them: `--transcripts DIR`, by default with
the dev store `$CLAUDE_CONFIG_DIR/projects` or `~/.claude/projects`. A signed step the transcripts also hold counts
once, at its signed counts, in the transcript's place (the transcript says which agent took the step, when, and how
long each cache write lasts, which no signed record holds): it is matched step by step, by its message id (the
tailer's records, taken only from a claude-code run whose records name a call of that session, signed from the
turn's first line to 5 minutes after its last), by a tool_use id (an LLM gateway's records, within 5 minutes), or by
identical token counts within 5 minutes of one transcript message no other step matched (the gateway signs each exchange
under an id of its own, and a message without tool calls has no tool_use id). Where the gateway and the tailer both
signed a turn, the gateway's counts are kept, and a gateway's record always counts: one that joins no turn counts on
its own. Other records of one turn that disagree, and any other record of a transcript's turn, are refused: the transcript's counts stand, and the page's first line says how many
([limits](limits.md#viewer)). The page's TOKENS IN and OUT count a turn the gateway and the tailer both signed once, the
same way. Each card's `from` names where its token counts came from, the context card's arithmetic says how many steps each
source gave, and each wait of the cache card has its own (`src`). The page's first line says how many signed steps
took a transcript step's place, and the time span
of the signed records it read: the viewer's live view, its newest 50,000 rows, which on a busy store is less than 7
days. Holds come only from signed records; a transcript adds how long each blocked the agent and
whether it ran in a subagent. Each card names every source it mixes, and the page's first line says what it read; the
Expand Down
115 changes: 115 additions & 0 deletions tests/test_tailer.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,26 @@
"content": [{"type": "thinking", "thinking": "list the files first", "signature": "EqQBsealed"}]}}
SAID = {"type": "assistant", "uuid": "u-6", "cwd": "/work/proj", "message": {"id": "msg_1", "model": "claude-x",
"content": [{"type": "text", "text": "Listing the files."}]}}
# a message's usage as Claude Code writes it on each of the message's lines, and as the tailer signs it
USAGE = {"input_tokens": 6, "output_tokens": 412, "cache_read_input_tokens": 150_000, "cache_creation_input_tokens": 1_200,
"cache_creation": {"ephemeral_5m_input_tokens": 0, "ephemeral_1h_input_tokens": 1_200},
"server_tool_use": {"web_search_requests": 0}, "service_tier": "standard"}
SIGNED = {"input_tokens": 6, "output_tokens": 412, "cache_read_tokens": 150_000, "cache_write_tokens": 1_200}
THINK = {"type": "thinking", "thinking": "plan the listing first", "signature": "EqQBsealed"}
TEXT = {"type": "text", "text": "Listing the files now."}


def turn(mid, *blocks, model="claude-opus-5-5"):
"""The lines Claude Code writes for message `mid`: one per content block, each with the message's usage and its
stop_reason (tool_use when it calls a tool)."""
stop = "tool_use" if any(b["type"] == "tool_use" for b in blocks) else "end_turn"
return [{"type": "assistant", "uuid": f"u-{mid}-{i}", "timestamp": "2026-10-01T08:15:42.123Z", "cwd": "/work/proj",
"message": {"id": mid, "model": model, "role": "assistant", "stop_reason": stop, "content": [b],
"usage": USAGE}} for i, b in enumerate(blocks)]


def call(tcid, args=ARGS):
return {"type": "tool_use", "id": tcid, "name": "Bash", "input": args}


def line(e):
Expand Down Expand Up @@ -376,6 +396,77 @@ def test_with_the_vault_each_line_is_kept_whole_and_opens_its_commitment(self):
{"transcript": "main", "entry": TOOL_USE}, {"transcript": "agent-a1.jsonl", "entry": sub}])
self.assertTrue(all("salt" in x for x in got)) # each opens its record's commitment

def usage(self):
"""[(exchange id, its tool uses' ids, usage)] of the run's model responses, in order."""
return [(x["exchange_id"], [t["id"] for t in x.get("tool_uses", ())], x.get("usage"))
for x in (e["data"] for e in self.events() if e["type"] == "model.exchange") if x["phase"] == "response"]

def test_a_messages_usage_is_signed_once_on_its_last_line(self):
"""#865: on a week of real work no record carried what a model turn cost, and tips read it from the transcript,
a file the agent writes, later. Claude Code writes a line per content block, each repeating the message's
usage: the tailer signs it once, on the message's last line (its tool use: the viewer's model line), with the
model; Claude Code's stand-in for a failed call (`<synthetic>`) is no turn."""
self.write(PROMPT, *turn("msg_1", THINK, TEXT, call("toolu_1")), *turn("msg_2", THINK, TEXT),
*turn("msg_3", TEXT, model="<synthetic>"))
self.tail()
self.assertEqual(self.usage(), [("msg_1", [], None), ("msg_1", [], None), ("msg_1", ["toolu_1"], SIGNED),
("msg_2", [], None), ("msg_2", [], SIGNED), ("msg_3", [], None)])
models = {e["data"]["model"] for e in self.events() if "usage" in e["data"]}
self.assertEqual(models, {"claude-opus-5-5"})
self.assertEqual(self.lost(), [])

def test_a_turn_streamed_over_several_lines_is_signed_at_its_final_counts(self):
"""The user run on a real session found 714 of 8,415 messages whose subagent lines differ: Claude Code wrote a
tool use while the turn still streamed (no stop_reason, 16 output tokens so far), ran it (its result line in
between), then the turn's last tool use with the final counts (stop_reason, 243). The turn is signed once, at
those; one that never got a stop_reason (an interrupted turn) is not signed at all."""
early, final = turn("msg_1", call("toolu_1"), call("toolu_2"))
early["message"] = dict(early["message"], stop_reason=None, usage=dict(USAGE, output_tokens=16))
result = {"type": "user", "uuid": "u-r1", "message": {"content": [
{"type": "tool_result", "tool_use_id": "toolu_1", "content": "ok"}]}}
[cut] = turn("msg_2", TEXT)
cut["message"] = dict(cut["message"], stop_reason=None)
self.write(early, result)
self.tail(lambda: self.write(final, cut, PROMPT))
self.assertEqual(self.usage(), [("msg_1", ["toolu_1"], None), ("msg_1", ["toolu_2"], SIGNED),
("msg_2", [], None)])

def test_a_message_read_over_two_wakes_and_its_subagents_copy_of_it_sign_its_usage_once(self):
"""A message's lines can come in over two reads (inotify wakes the tailer at each write), and a subagent's
transcript opens with a copy of the message that started it (its Agent call): its usage is signed once, by its
message id, across them, on its tool use, not on a thinking line the viewer would show as a model call without
one."""
first, rest = turn("msg_1", THINK, call("toolu_a"))
self.write(first)
self.tail(lambda: self.write(rest), lambda: self.sub(rest, *turn("msg_s1", TEXT)))
self.assertEqual(self.usage(), [("msg_1", [], None), ("msg_1", ["toolu_a"], SIGNED),
("msg_1", ["toolu_a"], None), ("msg_s1", [], SIGNED)])
self.assertEqual(self.lost(), [])

def test_a_turns_usage_is_signed_in_clear_and_its_text_only_as_a_commitment(self):
"""docs/privacy.md: the usage numbers and the model are in clear; the model's thinking and text, the call's
arguments and the line's own cwd and time are only in the commitment (and the vault, when it is on)."""
lines = turn("msg_1", THINK, TEXT, call("toolu_1"))
self.write(*lines)
self.tail()
[rec] = [e["data"] for e in self.events() if "usage" in e["data"]]
self.assertEqual((rec["model"], rec["usage"]), ("claude-opus-5-5", SIGNED))
self.assertEqual(rec["content_digest"], self.signer._commit("model.exchange:" + rec["salt_id"], event_hash(
{"transcript": "main", "entry": lines[-1]})))
with open(os.path.join(self.d, "data", "store", "records.jsonl")) as f:
stored = f.read()
for text in ("plan the listing", "Listing the files", "EqQBsealed", "/work/proj", "ls -la", "2026-10-01T08:15"):
self.assertNotIn(text, stored)

def test_usage_never_enters_the_provenance_index(self):
"""`from: untrusted` rules read the run's provenance index, where a value once observed as trusted stays
trusted: the usage a transcript (the agent's own file) reports is never indexed, trusted or not."""
self.write(*turn("msg_1", call("toolu_1")))
self.tail()
self.assertEqual([u for *_, u in self.usage()], [SIGNED])
prov = self.signer.log.runs[("default", self.out["run_id"])]["prov"]
self.assertEqual((prov["state"], prov["index"]), ("complete", {}))

def test_a_burst_of_lines_past_the_rate_limit_is_sent_whole(self):
signer = svc.SignerService(os.path.join(self.d, "burst"), grace_s=600, fsck_every_s=0,
limits=svc.Limits(events_per_s=2000, burst=50))
Expand Down Expand Up @@ -1102,6 +1193,30 @@ def test_a_line_already_read_changed_while_no_tailer_ran_is_lost(self):
self.assertIn("a line already read changed in place while no tailer ran", lost[1])
self.assertEqual(found, [("reconcile.fabricated", "toolu_2")]) # its line came after the change: not read

def test_a_tailer_started_again_signs_no_usage_the_stopped_one_signed(self):
"""#865: a turn is signed once across a run's tailers. The tailer read a message's first tool use, signed
its usage there and was killed; the run's next hook event started a new one, which read the message's second
tool use, and a subagent's transcript that opens with a copy of it, with no second usage: the ids the stopped
tailer signed are in its progress log."""
first, rest = turn("msg_1", call("toolu_0"), call("toolu_1"))
sub = os.path.join(self.d, "s1", "subagents", "agent-a1.jsonl")
self.write(PROMPT, first)
self.hook(start="startup")
self.tailer(then=Killed)
self.write(rest)
self.decide("toolu_1", "Bash", ARGS) # finds it stopped: a new tailer
self.assertEqual(len(self.started), 2)
self.write(rest, *turn("msg_s1", TEXT), path=sub)
self.tailer()
self.finish()
with open(os.path.join(self.d, "data", "store", "records.jsonl"), "rb") as f:
events = [json.loads(x)["event"] for x in f.read().splitlines()]
self.assertEqual([(e["data"]["exchange_id"], e["data"]["usage"]) for e in events
if e["type"] == "model.exchange" and "usage" in e["data"]],
[("msg_1", SIGNED), ("msg_s1", SIGNED)])
self.assertEqual(sum(e["type"] == "model.exchange" and e["data"].get("exchange_id") == "msg_1"
for e in events), 3) # every line was still recorded

def test_an_idle_tailer_parks_and_the_runs_next_event_starts_it_again_with_no_gap(self):
"""A run stays open while an approval waits, and a person may answer hours later. The tailer gave up after
IDLE_TAILER_S with no new line ("no new transcript line for 7200s"), and every call after was a
Expand Down
Loading
Loading