Bug: Claude Code adapter discover() times out with 156 project dirs / 1,697 sessions

[c-h-]

Bug: Claude Code adapter discover() times out with large session history

Problem

agentctl list -a shows Warning: Adapter(s) timed out after 5000ms: claude-code consistently. The Claude Code adapter's discover() method scans ALL project directories and session files in ~/.claude/projects/.

Evidence

~/.claude/projects/: 158 project directories
Total session .jsonl files: 1,701
Only 12/158 projects have sessions-index.json (most require full JSONL scan)

Scanning 158 dirs × ~11 sessions each, reading files, parsing JSON, and checking PIDs exceeds the 5s timeout.

Impact

• agentctl list never shows Claude Code sessions
• Can't monitor running sessions or detect stalled agents
• Arc's agentctl adapter also can't discover Claude Code sessions

---

Investigation Findings

Root Cause: We're scanning 1,701 files when Claude Code maintains a single-file index

~/.claude/history.jsonl is Claude Code's global session index — a 380KB file with one JSON line per user message. Each entry contains:

{"display": "<prompt text>", "timestamp": 1772479101988, "project": "/Users/ms/personal/some-repo", "sessionId": "f45049ba-919e-4522-855d-dd00767ae9a5"}

This is almost certainly what powers claude --resume's instant picker. Parsing this single file gives us every session's ID, project path, first/last activity time, and prompt text — in 0.002 seconds.

Bottleneck Profile

Operation	Time	Notes
stat all 1,701 .jsonl files	3.8s	Just stat calls, game over for 5s timeout
readHead 8KB per unindexed session	~1-2s est.	For 146/158 projects without sessions-index.json
parseSessionTail 64KB per session	additional	Called for every discovered session in discover()
lsof per running PID	~54ms each	Sequential, one subprocess per PID
Parse history.jsonl (380KB)	0.002s	Has all the metadata we need

Total wall time for current approach: 5-8+ seconds (exceeds 5s timeout)

What Claude Code Knows That We Don't Use

File/Dir	What it contains	Sessions	Size
~/.claude/history.jsonl	Global session index — every user message with sessionId, project, timestamp	All 157 unique sessions	380KB
~/.claude/projects/*/sessions-index.json	Per-project session index (deprecated?)	Only 12/158 projects have it	Stale (last modified Feb)
~/.claude/session-env/	Per-session environment snapshots	502 entries	—
~/.claude/todos/	Per-session todo/task state	620 entries	—

sessions-index.json is Dead

Only 12 of 158 project dirs have a sessions-index.json. The oldest dates to Jan 30, newest to Feb 3 — Claude Code appears to have stopped generating them. The adapter's fallback path (scan all .jsonl files) is now the default path for 93% of projects.

No --list-sessions CLI

claude --help shows no session listing command. The --resume [value] flag opens an interactive picker (with optional search), and --continue resumes the most recent session in the current directory. Both work instantly — confirming Claude Code uses a fast internal index (history.jsonl) rather than scanning.

getClaudePids() is Also Slow

For each Claude process found in ps aux, the adapter:

1. Calls lsof -p <pid> -Fn to get cwd (~54ms each, sequential)
2. Calls ps -p <pid> -o lstart= to get start time

With multiple Claude processes, this adds hundreds of ms. Could be batched into a single lsof -p pid1,pid2,... call.

---

Proposed Fix: Use history.jsonl as Primary Index

Fast Path (discover/list)

1. Parse ~/.claude/history.jsonl — group entries by sessionId, extract:

   • project (cwd/project path)
   • First and last timestamp (session created/modified)
   • First display entry (prompt text)
   • This gives us everything discover() returns EXCEPT: model, tokens, running status

2. For running status — only check sessions active in the last N hours:

   • Get running Claude PIDs (batch lsof into one call)
   • Cross-reference with recent sessions by project path match
   • Check persisted session metadata (existing mechanism)

3. For model/tokens — only parse JSONL tail for sessions the caller cares about (lazy, on-demand in status() and peek(), not during discover())

Expected Performance

Step	Time
Parse history.jsonl	~2ms
ps aux (one call)	~67ms
Batch lsof for N running PIDs	~54ms
Total	~125ms

That's a 40x improvement over the current 5+ second timeout.

Implementation Notes

• history.jsonl maps project → filesystem path (e.g. /Users/ms/code/mono), not the hashed dir name. This is cleaner than the current approach of decoding sessions-index.json paths.
• The file is append-only and grows slowly (1,022 lines over ~1 month of heavy use). No need for streaming/incremental parsing — just read the whole file.
• Project path in history.jsonl can be used to construct the JSONL session file path: ~/.claude/projects/${hashPath(project)}/${sessionId}.jsonl — where hashPath converts /Users/ms/code/mono → -Users-ms-code-mono.
• For --all mode (showing stopped sessions): filter history.jsonl by timestamp instead of scanning every project dir.

Alternative: Parallel Scan (Quick Win)

If the history.jsonl approach is too much refactoring, parallelizing the current scan is a quick win:

• Use Promise.all with concurrency limit (e.g., 20 at a time) for project dir scanning
• Batch lsof calls: lsof -p pid1,pid2,pid3,... instead of one per PID
• Skip parseSessionTail() during discover — defer to status()

This alone would likely bring discover() under 2s, but history.jsonl is the proper fix.

---

Environment

• agentctl 1.5.2
• 158 Claude Code project dirs, 1,701 session files
• history.jsonl: 1,022 lines, 380KB, 157 unique sessions
• macOS, Apple Silicon (M4 Ultra)

[c-h-]

Work in progress on branch fix/claude-code-discover-perf (not yet merged):

• Commit fc3386a: Uses ~/.claude/history.jsonl as primary index (0.002s parse time vs 5+ sec full scan)
• Batch lsof calls instead of one per PID

No PR opened yet. The fix is ready but needs to be pushed through the PR process.