Add checkpoint capture: whole-session segmented memories

[jamby77]

Problem

After #8 moved capture to SessionEnd, a latent cap surfaced: selectTranscript keeps only 8,000 chars, so a long session now captures ~5% of itself into a single memory. Distinct topics (PR review, debugging, a design discussion) collapse into one truncated summary, which also hurts recall — near-duplicate/bunched content always scores "low" at the confidence gate.

Change

A new Stop hook checkpoints the transcript incrementally: every ~8,000 chars it queues one segment to the existing ingest queue and advances a /tmp checkpoint file — doing zero model work. SessionEnd flushes only the post-checkpoint tail and spawns the existing detached drain. A long session becomes many distinct, sequential segments instead of one husk. The drain is unchanged (Track 1's empty-summary guard still applies); session consolidation and provider-aware thresholds are deliberately deferred.

Implementation (7 TDD tasks)

• nextChunk — pure decision (in turns, resuming in bytes), table-tested with no fs/Valkey/model.
• Checkpoint file I/O — atomic temp+rename; missing/corrupt → zero checkpoint.
• Shared parseTranscriptLine + byte-offset parseTurnsFrom (replaces the private parser in session-end, de-duplicated).
• The Stop hook — stat-gate → parse new bytes → nextChunk → queue + advance to endByte (never EOF).
• session-end flushes only the tail from the checkpoint offset, tags segment, cleans up.
• Registration across index.ts, register-hooks.ts, install-hooks.sh.
• Integration test — a seeded transcript yields sequential segments [0,1,2,…] + a tail against real Valkey.

Install-path refactor

Registering a fifth hook surfaced drift the branch inherited: all three install paths kept their own copy of the hook map, next to a literal Registered 5 hooks count and a hand-aligned summary listing — so adding a hook meant remembering several places.

src/hook-spec.ts is now the single typed source of truth (HookEvent/HookSpec/HookEntry/HookCommand — no more Record<string, unknown[]> and no casts). All three paths derive from HOOK_SPECS: index.ts and register-hooks.ts their settings map, hook count and summary listing (plus, in index.ts, the hook entries of BINARIES), and install-hooks.sh via requireing the module from its embedded Bun snippets. isOwnHookEntry replaces substring matching on "betterdb"/BIN_DIR when identifying our own entries to replace.

install-hooks.sh also gains two fixes. It wrote { ...existing, hooks }, replacing the entire hooks block — installing silently deleted every hook the plugin did not write, third-party entries on unregistered events included. It now merges per event and drops only its own entries, like the other two paths, and reinstalling stays idempotent. Its verification step counted Object.keys(settings.hooks), which reported success even with a hook missing; it now checks each spec's binary is present and exits non-zero naming what is absent.

The hook-migration strip is deleted (module + 89-line test file). It only ever covered Stop, and Stop is registered again, so both install paths' merges already replace the legacy session-end entry before the strip could. Verified rather than assumed: replaying a v0.4.x settings.json (legacy session-end on Stop, a third-party Stop hook, an unrelated PreCompact hook) through both paths with and without the strip produces byte-identical output — stale entry gone, third-party hooks and unrelated keys preserved.

Review & verification

Each task passed an independent spec+quality review (three needed a fix loop: a plan off-by-one, a missing byte-offset test, a dead tail-flush path — all fixed). A whole-branch review verified all four load-bearing invariants end-to-end — no model work on hooks, checkpoint advances to endByte not EOF, segment continuity across Stop firings + SessionEnd, tail parses from the offset — verdict ready to merge.

bun run typecheck clean · bun test tests/unit 164/164 · bun test tests/integration 0 fail (checkpoint test runs unskipped).

Build script

bun run build:hooks was referenced by CLAUDE.md and install-hooks.sh but not defined in package.json — the documented installer was broken independent of this branch. It is now scripts/build-hooks.ts, which compiles each hook entry point to dist/hooks/ and derives its build list from HOOK_SPECS, so a new hook needs no separate build wiring. The optional provider openai (an undeclared dynamic import) is marked --external so --compile does not fail on it. Verified: all five binaries compile and exit 0 on the empty-stdin hook contract, and the working tree stays clean (*.bun-build temp files are swept and ignored).

Stack

Top of a 2-PR stack, based on #8. Its diff is scoped to the checkpoint work only; merge #8 first.

---

Note

Medium Risk
Changes the core session-capture and Claude Code hook registration paths (including ~/.claude/settings.json), but behavior is heavily unit/integration tested and failures are designed to fail fast or re-queue rather than drop data silently.

Overview
Fixes long-session capture where a single ~8k selectTranscript cap dropped most of the conversation. A new Stop hook (stop-checkpoint) queues ~8k transcript segments to Valkey with no LLM work, tracking byte offset and segment index in /tmp checkpoint files. SessionEnd resumes from that offset, chunks the remainder like Stop (planTailSegments), flushes numbered segments, and still spawns the detached drainer when Stop already queued work.

src/hook-spec.ts becomes the single source for hook events (including Stop), per-hook timeouts, buildHookMap, and conservative isOwnHookEntry replacement across CLI install, register-hooks.ts, and install-hooks.sh (which now merges per event instead of overwriting the whole hooks block). Adds build:hooks / companion drain binary compile, locateDrainCommand, and flushSegments for partial-failure safety.

Ingest reliability: drainIngestQueue loops until empty; failed pops re-queue the full unprocessed batch. Valkey gets deadline-bounded connectValkey and shorter hook retries. Ollama summarization streams with keep_alive and one retry on cold-load timeout.

Docs add macOS dev guidance (docker-compose.mac.yml, native Ollama); extensive unit/integration tests cover checkpoint I/O, chunking, hook identity, and drain behavior.

^{Reviewed by Cursor Bugbot for commit 707fd81. Bugbot is set up for automated code reviews on this repo. Configure here.}

[KIvanow]

Approve. Builds cleanly on #8 (merge #8 first). The checkpoint design is careful — byte-offset checkpoints with atomic temp+rename, multi-byte-safe slicing at newline boundaries, segment continuity across Stop→SessionEnd, and dedup gating on the event-file fallback. Good test coverage, and the install-path refactor to a single HOOK_SPECS source of truth is a nice cleanup.

Verified locally on this branch (contains #8): bun run typecheck clean · bun test tests/unit 164/164 · build:hooks compiles the hook binaries and sweeps its temp files.

Two low-severity hook-matching edge cases, neither blocking:

• src/hook-spec.ts isOwnHookEntry (source-name match) — the doc comment calls this "deliberately conservative — a false positive deletes a third party's hook," but matching on generic source filenames like pre-tool.ts / post-tool.ts is arguably the opposite: a third-party hook command that happens to reference such a path would be silently deleted on install. Consider requiring a path marker and a source-name match, or namespacing the filenames (e.g. betterdb-post-tool.ts).

• src/hook-spec.ts isOwnHookEntry (cross-install-path gap) — installed-binary entries are recognized only via the caller's markers (BIN_DIR/distDir/"betterdb"), because the binary command is .../session-end and does not contain session-end.ts. So an entry written by install-hooks.sh (binaries under a project path with no "betterdb" in it) won't be recognized by a later index.ts CLI install (marker ~/.betterdb/bin) — it survives the merge and you get two hooks on the same event. Edge case (mixing install mechanisms), but eliminating exactly this drift was the goal of the refactor. Worth a test covering the install-path → CLI-install upgrade.

Also see the note left on #8 about processIngestQueue dropping already-popped items on a mid-batch failure — pre-existing, but segmented capture here makes it materially more likely to lose a session's later segments.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit ae6045e. Configure here.}