Tower architect identity hardening: role_id || 'main' silent fallback, missing UNIQUE constraint, fresh-launch crash-loop fallback gap

[amrmelsayed]

Problem

Hitting Ctrl-C twice in Claude Code terminates the CLI process (Claude Code's own behavior — two SIGINTs, it exits). The question is what Tower does in response, and the answer is the bug: Tower cannot tell a deliberate quit apart from a crash, and always tries to resume.

Confirmed directly in the shellper-backed exit handler (packages/codev/src/terminal/pty-session.ts:156-185):

client.on('exit', (exitInfo: { code: number; signal: string | null }) => {
  ...
  if (this._restartOnExit) {
    // unconditional — no branch on exitInfo.code or exitInfo.signal
    this.onPtyData('[Process exited — restarting...]');
    ... respawns ...

There is zero inspection of exitInfo.code/exitInfo.signal before deciding to restart. Whether Claude Code exits with code 0 (deliberate quit) or dies from an actual crash, Tower observes the identical event and reacts identically. restartOnExit is always true for architect sessions.

Worse: the restart is never a plain relaunch. It re-injects --resume <stored-session-id>, read from that architect's row in state.db (issue #832's design, meant to survive real crashes without losing conversation context). This resume-injection fires on every exit, deliberate or not — so whatever picks the wrong stored session isn't a rare edge case, it's exercised every time a user deliberately ends a session this way.

Reported symptom (Waleed, 2026-07-21, Tower UI)

Session restart when I do ctrl-c ctrl-c is picking random sessions to restart that do not match the architect I just restarted. I would rather that when I do ctrl-c ctrl-c it just restarts without any retry.

Structural causes found (grounded in source, not speculation)

1. Identity-resolution fallback silently defaults to 'main'. dbSession.role_id || 'main' appears at packages/codev/src/agent-farm/servers/tower-terminals.ts:653, 909, 1030. The adjacent comment says a v13 migration "should have" backfilled every legacy row's role_id" — if any row's role_idis null when Tower rebuilds that session's restart options (at reconciliation — Tower startup, or a reconnect sweep), the rebuilt session resolves identity as **main** regardless of which architect it actually was, and bakes in main's stored--resume` id.
2. role_id has no UNIQUE constraint anywhere in the schema. Explicitly called out in code comments: packages/codev/src/agent-farm/commands/workspace-recover.ts:121 ("has no UNIQUE constraint on role_id — the cautious read is 'alive'") and :343 ("role_id has no UNIQUE constraint in the schema, so collect every matching"). Duplicate/stale rows for the same architect name are structurally possible; any lookup expecting "the one row for this name" can land on the wrong one depending on insertion order — which looks like "random" to a user, since row order isn't observable.
3. A dated, real incident matching this exact failure shape already occurred. The pir-1198 builder thread documents that on 2026-07-19, one session's reconnect glitch got misclassified as dead, and killOrphanedShellpers (packages/codev/src/terminal/session-manager.ts:599-643 — scoped to all shellpers on the Tower instance, not to one terminal) then killed two unrelated, healthy architect processes as collateral damage. That specific trigger was patched, but the mechanism — one terminal's problem causing an unrelated terminal's session to die or rebuild — is proven to fire in this exact subsystem within the last few days.

What's NOT yet confirmed

A clean, single-keystroke path where Ctrl-C-Ctrl-C in terminal A directly restarts terminal B in normal steady-state operation (no recent Tower restart/reconcile). The generic per-session restart handler (session-manager.ts:setupAutoRestart) is correctly scoped per in-memory session with no shared-state bug found there. The likely trigger for the specific cross-architect mismatch is a reconciliation event (Tower restart/redeploy) landing around the same time as the Ctrl-C-Ctrl-C, at which point causes 1 and 2 above kick in. Worth confirming with Waleed whether Tower was restarted/redeployed around when he observed the mismatch.

Why PIR, not BUGFIX

This touches core, shared Tower session-restart machinery (pty-session.ts, session-manager.ts, tower-terminals.ts, tower-instances.ts, the terminal_sessions/architect schema) with several real design decisions, not a mechanical fix:

1. How should Codev distinguish "intentional quit" from "crash"? Recommended: fix the underlying cause (branch on exit code), not keystroke-listening. A precedent already exists in this exact file: session-manager.ts:905 gates the crash-loop counter with if (exit.code !== 0), and the adjacent comment states the semantic plainly — "Clean exits (code 0) never count: a user quitting a healthy session repeatedly must not trigger it." Codev already treats exit code 0 as "not a crash"; it just isn't wired to the restart-with-resume decision, only to the crash-loop-counting decision. Extending that same signal to gate resume-injection is the general, root-cause fix: it covers every voluntary-exit path uniformly (Ctrl-C-Ctrl-C, /exit, closing the terminal), not just the one specific keystroke sequence. Listening for the literal double-Ctrl-C pattern client-side would (a) miss every other intentional-quit mechanism, leaving them still wrongly auto-resumed, and (b) require adding keystroke-parsing to a layer (the shellper passthrough pipe) that is deliberately keystroke-agnostic today, to reconstruct information the exit code should already carry. Client-side keystroke detection should be treated as a fallback only, if empirical testing shows Claude Code's exit code does NOT reliably distinguish voluntary quit from every crash mode (segfault, OOM-kill, unhandled exception) — this needs verification (spawn a bare session, trigger each exit path, observe the actual code/signal) before committing to the exit-code branch as sufficient.
2. Should role_id gain a real UNIQUE constraint (scoped by workspace_path + type), with a migration to reconcile any existing duplicate/legacy rows? Or is a code-level fix (never trust an ambiguous match, log loudly and fail closed instead of silently defaulting to 'main') sufficient?
3. Should the 'main' fallback be removed entirely in favor of failing loudly (skip restart-with-resume, log an error) when an architect's role_id can't be resolved, rather than silently misattributing to main?
4. Should "restart without resume" be the default behavior for ANY exit, a per-exit-reason branch, or a user-facing toggle? This is the crux of the design: Multi-architect conversation resume: disambiguate via per-architect session UUID #832 introduced resume-on-restart specifically to survive real crashes without losing context — an implementation must not regress that case while fixing the deliberate-quit case.

Origin

Reported by Waleed Kadous (2026-07-21) via architect relay, investigated by the main architect session same day. Related: issue #1198 (killOrphanedShellpers root-cause, same subsystem, same week), issue #832 (introduced resume-on-restart), issue #1149 (crash-loop fallback), Spec 755 (role_id / multi-architect identity), Spec 786 (architect identity preservation across restart).

[amrmelsayed]

Empirical verification: exit code DOES reliably distinguish voluntary quit from crash (2026-07-21)

Ran the actual test rather than assuming. Spawned a real claude process inside a tmux pty (a genuine PTY, matching Tower's own shellper model — not a piped stdin, which would behave differently) and sent real Ctrl-C keystrokes.

Double Ctrl-C (rapid, real pty):

dead=1 status=0 signal=

Claude Code catches both SIGINTs itself and calls its own graceful shutdown with exit code 0, no signal. It is not killed by the raw signal — it exits intentionally.

Control: SIGKILL (simulated crash) on a fresh instance:

dead=1 status= signal=kill

No exit code at all; died via signal instead.

This confirms the recommended direction in the "Why PIR" section is sufficient, not just plausible: the exit code (0, no signal) for a deliberate double-Ctrl-C quit is unambiguously distinct from a real crash (signal-based death, or — untested here but standard Node behavior — a nonzero code from an uncaught exception). This is exactly the same signal session-manager.ts:905's exit.code !== 0 check already keys off for crash-loop counting. No client-side keystroke detection is needed; the fallback option in Q1 can be dropped from consideration unless plan-phase testing turns up a crash mode that also exits 0 (untested: uncaught exception, OOM-kill exit code specifically — worth a couple more empirical spot-checks at plan time, but the mechanism itself is now proven sound).

Method note for whoever plans this: tmux new-session + remain-on-exit on + pane_dead_status/pane_dead_signal is a clean, repeatable way to test any other exit path (e.g. /exit, closing the terminal) against this same distinction.

[amrmelsayed]

Clarification on the previous empirical-verification comment

The tmux-based test above was a throwaway external verification tool, not a suggestion for the codebase or the test suite. Codev's actual PTY layer is node-pty + its own shellper daemon (packages/codev/src/terminal/pty-session.ts, session-manager.ts) — tmux has no relationship to that stack and shouldn't be referenced as a method going forward. I only reached for it because it was the fastest already-installed way to (a) allocate a real PTY, (b) deliver an actual SIGINT the way a physical keypress does (a raw 0x03 byte over a plain pipe, with no TTY line discipline in between, would not trigger a signal at all), and (c) read the exact exit code/signal back out via pane_dead_status/pane_dead_signal.

For any further spot-checks at plan time (the uncaught-exception / OOM-kill cases flagged as still-untested above), the equivalent, codev-native way to reproduce this is a small node-pty script (spawn the CLI via pty.spawn, listen for the exit event, trigger the condition, read {exitCode, signal}) — that exercises the exact same code path pty-session.ts's own onExit/shellper-exit handlers consume, rather than a second, unrelated tool.

[waleedkadous]

But why is it resuming a random session, rather than the one that just got killed?

[waleedkadous]

Also this happens quite frequently and I can do nothing at that point other than kill the architect and retart it.

[Process exited — restarting...]
Error: Session ID 166e1c25-fd66-46ed-a81c-5dfcf2295efd is already in use.

[Process exited — restarting...]
Error: Session ID 166e1c25-fd66-46ed-a81c-5dfcf2295efd is already in use.

[Process exited — restarting...]
Error: Session ID 166e1c25-fd66-46ed-a81c-5dfcf2295efd is already in use.

[Process exited — restarting...]

[waleedkadous]

More issues around restarting sessions:

zbb is back and stable — new terminal (PID 4599), its Claude child has been running continuously instead of dying
every few seconds, /arch-init sent, and I briefed it: state file intact through the approved convergence plan, both
builders (#3123 behavioral-gate resurrection, #3127 golden-journey metrics) alive and not to be respawned, lane
map unchanged.

What actually happened, in order: the foreign wrong-auth claude you quit had claimed zbb's session ID, which
started the crash loop. Quitting it freed the session, but zbb's shellper had by then wedged into a state where its
children kept dying instantly and it silently deregistered from Tower — the same terminal-death pattern as
dreaming and realtime earlier. Killing the wedged shellper and re-adding the architect was the fix, same recipe as
dreaming.

That's four architect-terminal failures today, all post-restart. I'm now treating this as a real Tower/codev bug —
the respawn path doesn't mint-or-reclaim session IDs atomically, and a wedged shellper can crash-loop forever
without ever recovering or cleanly deregistering. I'll file the upstream issue and send the number to the codev
workspace architect for cross-reference.

[amrmelsayed]

Scope addition: two crash-loop-breaker blind spots (from the #1224 investigation, 2026-07-23)

Found while investigating #1224; deliberately kept OUT of PR #1225 (active-incident fix, already twice-consulted) because both belong to this issue's design territory — the crash-loop response policy and exit-code semantics. Adding them to this PIR's scope so they're decided together with the resume-injection gating:

1. crashLoopFallback is only armed on resume launches. Every wiring site gates it with if (resumed && storedSessionId && fallback) — a fresh-mint launch (--session-id <new>) gets no fallback at all. If a fresh launch crash-loops (broken auth, bad config, a collision on the freshly persisted id), the breaker sees the crashes but has no lever: the loop burns 2s restarts to maxRestarts=50, then tears down having never attempted recovery. Plan should consider arming a trivial fallback (mint another fresh id) on fresh launches, or explicitly documenting why not.

2. The failure counter only counts exit.code !== 0. Correct for its original purpose (a user quitting a healthy session repeatedly must not trigger a fallback — the same exit-0-means-voluntary semantic this issue is built on). But a failure mode that exits 0 is invisible to the breaker while restartCount still burns to 50 on every exit — the worst combination: no recovery attempt, budget exhausted anyway. Whether e.g. the #1224 collision exits 0 or nonzero is claude-version-dependent (see my 2.1.216 non-repro on #1224). Plan should consider: repeated instant exits (child never lived long enough to be usable) count as failures regardless of exit code — liveness duration, not exit code, may be the more robust crash-loop signal, and it composes with (rather than contradicts) the exit-0-gating this issue proposes for the resume-injection decision.

Post-#1225 note: the give-up path now reaps the shellper husk, so exhaustion no longer strands a zombie — these gaps are about never reaching pointless exhaustion in the first place.

[waleedkadous]

Re-scoped (per triage 2026-07-25): the deliberate-exit half of this issue — Ctrl-C-Ctrl-C indistinguishable from a crash, unconditional restart — is being fixed by PR #1244 (issue #1241): a shared isDeliberateExit() predicate (exit 0 AND no signal — node-pty reports signal deaths as code 0 + signal, so the naive code check would have been wrong) now gates all three restart surfaces (generated launch-loop scripts, setupAutoRestart, and the pty-session "restarting…" notice).

This issue now tracks the remaining defect, which deliberate-exit fixes do NOT close: on a genuine crash-restart, resume can pick the wrong architect's session via the role_id || 'main' identity fallback (tower-terminals.ts:653, 909, 1030) and the always-on --resume re-injection. That mis-resume fires on real crashes too, so it stays open on its own merits. Original forensics above remain the best starting point.

[amrmelsayed]

Re-triage against v3.2.4's #1241 / PR #1244 (2026-07-27): the exit-classification core of this issue SHIPPED — remainder is the identity-hardening cluster

What #1244 delivered (verified against its production diff):

• isDeliberateExit() in shellper-protocol.ts: code 0 AND no signal — including the node-pty wrinkle that partially invalidated this issue's original assumption (node-pty reports a SIGKILL'd child as {exitCode: 0, signal: 9}, so code alone is insufficient; the signal field disqualifies). This supersedes the empirical exit-code work in the comments above, consistently with it.
• SessionManager: deliberate exit → no restart at all, clean removal, session-clean-exit event (husk left for scrollback — collected later by Codev process fleet drives macOS memory-pressure kills: shellper husk sweep + fleet observability #1234's sweep).
• PtySession: deliberate exit on a restartOnExit session → clean end + honest message instead of the false '[restarting…]' notice; architect row cleared so workspace start can relaunch.
• All five builder launch-loop variants: clean exit → clear screen + wait-for-Enter gate (no auto-respawn); nonzero/signal-death (bash 128+N) → historical 2s auto-restart.

That resolves this issue's Q1 (intentional-vs-crash distinction) and Q4 (resume policy on quit — better than proposed: no restart at all), plus Waleed's original 'just don't retry' ask, plus most of breaker blind spot (b) from the scope-addition comment (a code-0-no-signal crash is now nearly definitionally impossible).

What REMAINS open (this issue's live scope now):

1. role_id || 'main' silent fallback (tower-terminals.ts:653/909/1030) — untouched by [Bugfix #1241] Auto-restart only on unnatural exits — a deliberate quit ends cleanly #1244. A null-role_id row still silently resolves identity to 'main' at reconciliation/restart, baking the wrong architect's --resume on the CRASH path (which still restarts, correctly). Fail-loud (skip resume + ERROR log) or repair, never silently misattribute.
2. No UNIQUE constraint on role_id (per workspace_path+type) — ambiguous-row hazard stands (workspace-recover.ts:121,343), needs constraint + duplicate-reconciling migration or documented code-level fail-closed.
3. Breaker blind spot (a): crashLoopFallback still armed only on resume launches — a fresh-mint launch that crash-loops has no fallback and burns 50 restarts.

Suggested re-scope: retitle to the identity-hardening remainder; with the design questions resolved by #1244, this is likely BUGFIX-sized now (schema migration is the one careful piece) rather than the PIR the original full scope warranted.