Repository navigation
Restart continuation revives runs that were cut hours ago and already superseded #326
Description
What: After bestia's 22:39 restart on 2026-10-14, #301's continuation revived old cancelled runs whose work had long been finished, and each one woke its parent with a stale result.
Why: The continuation seems to consider every cut run without a later turn, including runs cut in earlier restarts that their threads or parents already moved past.
So what: An agent narrows the continuation to runs cut by this restart (or still current), with a focused test; the planner staffs it.
Evidence (bestia, 2026-10-14 22:39 CEST restart, build 555f0a0)
Three parent wakes within minutes, each a revived old run reporting work that was already merged:
headless-cross-machine-control-2026-10-14: "This notification is for Sol's first roast of the design, which a host freeze cut early on. It no longer matters."chromeria-243-contextmd-52-finish-2026-10-14: "from the worker's round-7 run that was cut earlier; nothing changes."chromeria-256-orchestrator-r2-2026-10-14: a repair result from PR fix(server): replay stored events from an explicit cursor #267's15cebb6, merged long ago as96da60d.
Each costs a parent turn and a planner turn, and can make a coordinator act on stale state.
Acceptance
- A restart continues only the runs it cut (or runs still current for their thread), not runs cut by earlier restarts that a later run or a completed task has superseded.
- Skipped old cuts are recorded with a reason, per Restart continuation silently skips some cut turns; Codex threads were never continued #264's "continue or say why".
- Focused test with two restarts in sequence.
Non-goals
Changing the guarded queue resume tool; changing which providers continue.
Proof
Focused tests; both review/independent-sol and review/independent-grok on the exact head; the next restart produces no stale wakes.
Contribution to the Objective
Restarts stop costing agent turns, so the Promachos and delegated work keep going through deploys.
Elon record
Requirements and who asked: User wants agents to keep working across restarts at full throttle (2026-10-14); #264 required cuts to continue or say why.
Deleted: A parent-side filter for stale wakes; muting notifications.
Bottleneck: The continuation's selection of which cut runs to revive.
Checked myself: The three task_status results after the 22:39 restart, each self-described as stale.
Activity
lukemaj commentedon Oct 14, 2026
What: Read-only live evidence identifies historical result replay, rather than new child provider runs, in all three reported 22:39 wakes.
Why: The startup delegated-result query admits old untransferred cancelled follow-ups on legacy unbound children even when their delegated task already has a result.
So what: The implementation reproduces this source-selection defect across two SQLite/runtime restarts and repairs the existing recovery component before exact-head Sol/Grok review and authorized merge.
The three reported parent threads received transfers from
roast-164-design-sol-r1run 2 (cut 09:23:33Z),232-fix-round3-implrun 9 (cut 11:37:06Z), andrepair-267-r1run 6 (cut 09:40:32Z). Those transfers were created at 20:39:42–43Z. None of those children has a newly requested or started run at that boot. Their immutable first delegated results were already cancelled, cancelled and completed respectively. The parents' new turns were notification handling, not evidence of revived child execution. All queries used SQLitemode=ro; no live server was started or changed.Current
origin/main/base:555f0a00198b7aba75d8e8ce13af9c7d983712f4. Exclusive branch/worktree:fix/326-current-restart,/home/luke/dev/toolboxmd/chromeria-worktrees/326-current-restart. Main checkout and its user-owned cache remain untouched. No version bump, deploy, restart or memory write is authorized here.Upstream first: main is
e47bfe4617f6fdca2c6513bd0afe0f6b8d24688a. Searches of upstream PRs forrestart continuationand Issues forrestart supersededfound no same-cause merged repair. Open #15820 remains the earlier shared-session issue; #17589 concerns queued steering/continuation. This failure is in the fork's legacy delegated-follow-up result discovery, so neither supplies a port for the demonstrated cause.Requirements and who asked: The user requires no stale wakes from old restart cuts, a reason for each skipped cut, a two-restart regression, focused proof, exact-head independent Sol/Grok success and green CI before the delegated merge.
Deleted: No provider/queue policy change, no broad parent notification mute, no new design round, no native server restart, deployment or version bump.
Bottleneck: Reproduce the historical unbound-child state and distinguish old result discovery from actual continuation execution.
Checked myself: Read #326 and the #264/#301 recovery history; inspected source selection, outbox and transfer identities; queried the named children and their parent task/transfer rows read-only; created an exclusive worktree from freshly fetched origin/main and installed with the prescribed pnpm command.
lukemaj commentedon Oct 14, 2026
What: PR #329 contains the reproduced historical-result recovery repair at
eef68e9be143dfeab1c32f088e3be78765f8182f.Why: The reported children were not restarted; startup newly transferred old cancelled follow-ups of already-returned delegated tasks.
So what: Exact-head independent Sol/Grok review and complete CI gate the authorized merge; no deployment or Chromeria restart is part of this task.
Canonical continuation state:
555f0a00198b7aba75d8e8ce13af9c7d983712f4/ main; exclusive writer branch/worktreefix/326-current-restart,/home/luke/dev/toolboxmd/chromeria-worktrees/326-current-restart.eef68e9be143dfeab1c32f088e3be78765f8182f; source/index clean. RS/Restart continuation silently skips some cut turns; Codex threads were never continued #264 map part names PR fix(server): skip stale restart-cut results for settled tasks #329. No versions, dependencies or lockfiles changed.chromeria-326-sol-r1-eef68e9be1(Codex gpt-6.1-sol MAX) andchromeria-326-grok-r1-eef68e9be1(Grok grok-4.7 XHIGH). Both remain working; named and generic statuses pending on the full head. Neither verdict is yet claimed. Findings must be PR comments before success statuses./tmpwas deleted after no open reader was found; usage fell 62% to 15%. Subsequent evidence queries are direct SQLitemode=ro; retained reports are on disk in/home/luke/.cache/ch326./home/luke/.cache/ch326/baseline-worktreeremains detached at the base, dirty only in the task-owned copied regression. Its unique test is already preserved in the candidate; retire after terminal disposition. Reviewer worktrees are tool-owned/exclusive; preserve until each review stops. Main's unrelated user cache remains untouched.Requirements and who asked: The user requires no stale restart-cut wakes, a recorded reason per skipped old cut, two-restart proof, the specified exact-head reviewers and green CI before delegated merge.
Deleted: Actual provider revival as an unsupported diagnosis for these three cases, blanket parent notification filtering, provider/queue policy changes, new design round, versions and deployment.
Bottleneck: Independent verdicts and complete CI on the frozen candidate.
Checked myself: Matched live identities read-only, reproduced the exact final test on unchanged base, inspected candidate recovery/transaction/discovery paths, observed 168 focused passes, checked map/ownership and current refs, linked the PR and verified RAM-copy cleanup.
lukemaj commentedon Oct 14, 2026
What: PR #329 merged and #326 is closed; the reviewed tree is now main at
fbbe448316be1af24aeefb3ed124b152e8817253.Why: The three stale wakes were historical result replay from old cancelled follow-ups after their delegated tasks had already returned, rather than newly started child runs.
So what: The fix records
TASK_SETTLEDand excludes those resolved cuts from discovery; no restart or deployment was performed. Delivery is merged, with one managed reviewer-worktree cleanup exception below.Acceptance: the exact final two-restart SQLite regression fails on pinned base
555f0a00198b7aba75d8e8ce13af9c7d983712f4withexpected true to be falseat the stale-transfer assertion, and passes on reviewed headeef68e9be143dfeab1c32f088e3be78765f8182f. Author ran seven focused files/168 tests; Sol independently ran 200 focused tests and Grok 168. Sol findings and Grok findings were accepted without waived findings. Both required named success statuses and generic success were verified on the exact head, created bylukemajafter their comments. Complete CI run 38636834973 passed; the fresh ready-event native fingerprint run 38639960161 also passed before merge. Adjudication.Verified GitHub merged state at 21:25:57Z; fetched main and compared its complete tree to the reviewed head with zero diff. Issue closure verified. No version bump. Main checkout remained untouched. Native provider restart and next installed live observation remain unverified, as required by the no-restart/no-deploy boundary.
Cleanup: primary implementation, Sol review, and disposable baseline worktrees are removed; implementation and Sol local branches are removed; all task remote branches are absent. The 7.2 GB
/tmp/ch326-live.sqlitecopy was removed after confirming no open handles; evidence now lives on disk under~/.cache/ch326/, and subsequent live reads used SQLite mode=ro directly. No full live copy remains. PR linkage and unwatched state verified.Exception:
/home/luke/.t3/worktrees/chromeria/review-326-grok-r1, branchreview/326-grok-r1, exact reviewed head, clean source/index, remains because the T3-managed Grok provider PID 3997071 and MCP bridge PID 3997160 still own that exact cwd after completed-task cancellation. Both reviewer tasks are terminal with results acknowledged, and task_cancel was called on each. These processes were app-spawned, not PID-captured by this implementation, so no name/path-based kill was used. Owner: T3 provider lifecycle. Next action: remove this clean reviewer worktree and local branch after the managed provider releases its cwd. No production process was touched. This is a retained resource exception, not an implementation or merge blocker.Requirements and who asked: The user requested the #326 stale-cut fix, durable reasons, two-restart failing-before proof, exact-head Sol/Grok success and green CI, then delegated merge.
Deleted: No new design round, provider/queue policy, blanket result muting, version bump, restart, deployment or agent memory.
Bottleneck: The implementation and merge gates are resolved; only managed idle-reviewer resource retirement remains conditional on its owner releasing the cwd.
Checked myself: Read-only production identities and matching cancellation timestamps, exact final red test, 168 author focused passes, both complete reviewer findings and status timestamps, green CI, merged main tree equality, closed Issue, remote branch absence and task-owned cleanup after-state.