Skip to content

perf(stop): cost-tracker builds the subagent type map in two jq calls (#379) - #384

Merged
wasikarn merged 2 commits into
developfrom
claude/perf-379-cost-typemap
Oct 3, 2026
Merged

wasikarn merged 2 commits into
developfrom
claude/perf-379-cost-typemap

Conversation

@wasikarn

@wasikarn wasikarn commented Oct 3, 2026

Copy link
Copy Markdown
Owner

Refs #379 (item 2 of 3: cost-tracker subagent type map).

What

  • build_type_map forked jq 2-3 times and basename once per subagent file. It now runs two jq calls in all:
    1. one call reads every agent-<id>.meta.json (--rawfile each) and prints one line per file: its
      agentType when that is a non-empty one-line string, else "";
    2. one call builds the map from named args (--arg f<i>/i<i>/t<i>, never positional, so a type or id
      that starts with - stays a value).
      A "" line (no meta, a directory, an unreadable file, a non-string or multi-line agentType, bad JSON)
      takes the old per-file code unchanged, so every type is the one that code gives.
  • scan_transcript's id loop uses parameter expansion instead of basename.

Proof

Equivalence. The develop copy of the script was frozen and both copies ran on the same inputs. Each run
had a throwaway HOME, settle 0, and a date shim. Every run compared costs.jsonl, the marker file,
stdout, stderr and the exit code byte for byte. The one normalization: bash's own <script path>: line N:
prefix on stderr, because the two copies live at different paths.

corpus bash 3.2 bash 5
316 real sessions (all 276 with subagents, incl. 193/176/129 subagents; 5 with codex calls; 40 without subagents) 316 same, 1,531 rows, 0 diffs 316 same, 1,531 rows, 0 diffs
8 hand-made sessions (below) 8 same, 61 rows, 0 diffs 8 same, 61 rows, 0 diffs

All 4,003 local metas carry a plain string agentType, so real data never takes the fallback. The
hand-made sessions cover it: a missing meta, a directory, chmod 000, an agentType that is
null/false/""/number/object/array/NUL/newline/trailing newline/U+2028/invalid UTF-8/BOM, concatenated and
truncated JSON, toolUseId with and without a parent hit, --help as a type, names with a space and a
quote, a non-agent- file, both jq_failed sentinel paths, and codex calls in main and subagent files.
A positive control (the agent- strip removed) shows DIFF on 8 of 12 real sessions and 1 fixture session.

Tests. Phase 5 of test-cost-tracker.sh adds five subagents under one transcript: two plain types,
--help, a meta that holds only toolUseId (resolved through the parent's Agent call), and no meta. It
also adds two return windows. It asserts the exact (agent_type, input, returns, verify_tokens) per
subagent row. It passes on the develop copy and on the new code under bash 3.2 and 5. It kills all 8
mutants: types index off by one, f/t keys swapped, agent- prefix kept, fast path empty with no
fallback, rawfile index counting only present metas, ids passed positionally, and two in the scan loop
(.jsonl kept, directory kept).

Timing. Whole hook, 20 interleaved runs, settle 0, medians (p90 of runs):

session bash 3.2 bash 5
6 subagents (median session) 202 (210) -> 144 (148) ms 219 (232) -> 158 (163) ms
37 subagents (p90 session) 890 (911) -> 485 (490) ms 846 (890) -> 479 (493) ms
193 subagents (58 MB) 5,992 (6,150) -> 3,692 (3,716) ms 5,931 (6,442) -> 3,700 (3,913) ms

The hook is async, so this is CPU per Stop, not wait time.

Notes

  • $ARGS.named holding --rawfile values was checked on jq 1.8.2 (local) and runs in CI on the
    runner's jq. On a jq where it is missing, the fast path prints "" and every file takes the per-file
    path: slower, same output.
  • Ceiling (a ponytail: comment in the code): the map call takes about 400 bytes of args per file, so
    macOS ARG_MAX is near 2,000 subagents. raw_records' --argjson typemap already has the same ceiling.
  • Found on develop, left alone so the proof stays byte-identical: under bash 3.2 a session with no
    subagent files prints sub_files[@]: unbound variable (inside $(...), rows unaffected). Also,
    scan_transcript passes ids to jq positionally, so an agent--x.jsonl file fails the whole
    orchestrator scan (jq: Unknown option). Worth separate issues.

… (GH #379)

build_type_map forked jq 2-3 times and basename once per subagent file.
One jq now reads every meta (--rawfile each) and prints each agentType;
a file whose meta is missing, unreadable or any other shape takes the
old per-file path, so every type is the one that path gives. A second jq
builds the map from named args. scan_transcript's id loop drops basename
for parameter expansion.

Whole hook, 20 runs, medians (bash 3.2 / bash 5):
6 subagents 202 -> 144 / 219 -> 158 ms; 37: 890 -> 485 / 846 -> 479 ms;
193: 5992 -> 3692 / 5931 -> 3700 ms.
Byte-identical costs.jsonl, marker, stdout, stderr and rc against the
develop copy on 316 real sessions (1,531 rows) plus 8 hand-made fixture
sessions (61 rows), under both shells.

Test phase 5 pins five subagent types (meta, a --help type, toolUseId
via the parent, no meta) and their return windows; 8 mutants killed.
@wasikarn
wasikarn merged commit 39f1e75 into develop Oct 3, 2026
3 checks passed
@wasikarn
wasikarn deleted the claude/perf-379-cost-typemap branch October 3, 2026 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant