feat(fork): catch up to upstream's V2 orchestrator (185 commits, 69 conflicts)

[nohat]

Problem

fork/prod last synced upstream at 54084ae1e6 (2026-10-01). Upstream has since landed 185 commits, including de34391427 "new orchestrator" (V2, 1907 files). A trial merge of upstream/main into fork/prod (scratch worktree, 2026-10-05, nothing landed) conflicts in 69 files, and many more fork files auto-merge but now sit on code that no longer exists. Upstream moves about 45 commits a day, so the cost of waiting rises daily. This is the first real sync under #6, and it cannot be automated.

What the trial merge found

• Upstream deleted apps/server/src/orchestration/ and packages/contracts/src/orchestration.ts; V2 lives in orchestration-v2/ and orchestrationV2.ts. Git's rename detection dragged the fork's new drain/watchdog files into orchestration-v2/, where their imports point at deleted modules.
• Wire protocol is now ORCHESTRATION_PROTOCOL_VERSION 2; /ws answers mismatches with HTTP 426. Server, web, desktop, and iPad must ship in lockstep.
• V2 snapshots state.sqlite (about 1.08 GB live) into statev2.sqlite on first launch. The original stays for v1; later launches never refresh the copy.

Fork features: verdicts (read from the trial merge, not yet run)

Feature	Verdict
Drain mode (c45ea87d87)	Port. Choke point is orchestration-v2/ThreadMessageIntake.ts; running count from orchestration_v2_projection_runs; drain HTTP endpoints move out of deleted orchestration/http.ts; CLI into binCli.ts. The deploy depends on it.
Stall watchdog, stalledSince (911e4b6fe9, 442110291b)	Port, lower priority. Upstream has no silent-provider watchdog. Only #28 consumes stalledSince.
Claude silent-CLI fix (#1)	Mostly obsolete. ClaudeAdapterV2 reopens the query on model/policy change. One unbounded setPermissionMode remains.
OpenCode wave 0/1 + child-agent surface	Obsolete. OpenCodeAdapterV2 has native child threads.
OpenCode scoped agents (f602765a91)	Port (OpenCodeDriver.ts, v1 and v2 workspace paths).
#18 #19 #20 OpenCode lock/serve	Not resolved upstream. Re-diagnose on V2 (OpenCodeAdapterV2.ts:959 still connects per session).
Provider-agnostic subagents (#21)	Dispatch obsolete (delegate_task, t3_thread_launch). Pricing in capabilities output and the model-selection skill are still ours.
Papercuts	Textual plus one port. Papercuts.ts reads projection_thread_sessions/_activities, which are frozen copies in statev2.sqlite and fail silently stale. Rewrite to V2 tables. Keep the papercutCreate scope entry.
Direct APNs (#15)	Port. Not on fork/prod. Its migration id 055 collides with upstream's 055_OrchestrationV2; use a file store or separate ledger. Relay hook must be redone against V2 events.
Send-never-settles (#2)	Port onto launchThread/message.dispatch (no expiry upstream).
Send-blocked reason	Mechanical conflict.
Reconnect feedback (#7)	Port: WorkspaceEnvironment became EnvironmentConnectionSummary in client-runtime.
Thread sync stall bound, reload thread	Port/rewrite (V1 loader replaced; V2 sync is afterSequence). #3 may be mitigated by paged snapshots; needs a repro on V2.
Composer drafts (#8)	Web clean, mobile mechanical.
Model picker, pricing	Web clean; mobile port; pricing: mechanical (ModelListRow, providerIconUtils, ws.ts four small deltas).
iPad feed jump (#17)	Upstream shipped its own keyboard-controller patch (pingdotgg#14808). Take upstream's whole, re-run the replay, re-add the fork's hunks only if the jump reproduces. The fork's guard test forbids what upstream's patch does, so it must change with the decision.
#23 PostHog default	Unchanged upstream; still on by default.

Deploy and data effects (my live data)

• The cutover creates statev2.sqlite beside state.sqlite; nothing is written back to the original. Rollback to a v1 release reopens state.sqlite and loses threads and messages created after cutover. Migrated history is messages only (no checkpoints, tool calls, approvals, plans).
• fork-deploy-lib.ts countRunningSessions reads projection_thread_sessions in state.sqlite. After cutover that table is frozen, so the drain gate holds forever or reads zero. Fix before the first V2 deploy.
• restartAndProbe allows 90 s; the 1.08 GB copy is untimed, so a false "unhealthy" rollback is possible.
• fe417dff4c (t3 refuses to start when state names a live pid for the same base dir) must be checked against the swap ordering.
• Fork migrations: none on fork/prod; papercuts are JSON files and carry over.

Plan

1. Pre-flight (read-only, on a VACUUM INTO copy of live data): time the statev2 snapshot; record the result here.
2. Make fork-deploy V2-aware: running count from statev2.sqlite when present (or from the drain-status endpoint), longer first-start probe, and a recorded rollback note that post-cutover data does not return to v1.
3. Land the in-flight versioning work first (uncommitted in the primary checkout; a merge needs a clean tree, and it touches files upstream also changed).
4. Merge in a scratch worktree off fork/prod, git-imerge style if it fights back, with rerere on. Take upstream for pnpm-lock.yaml and regenerate; keep fork deltas small per file.
5. Port order: drain, papercut tables, send-never-settles and reconnect feedback, cache bound and resync, scoped agents, pricing/picker, watchdog last. Drop the obsolete rows. Each port keeps or rewrites its guard test.
6. Gate: the patches.tsv guards and delta-check from feat(fork): upstream sync job, fork-delta check, and agent guards #6 are the acceptance test for this merge. Targeted typecheck/tests for touched packages only.
7. Ship as fork 1.0.0: by the versioning bump rules, a data migration and wire break is a major. Server, web, desktop, and the iPad build go together. Then re-diagnose bug(mobile): iPad thread open hangs on 'Syncing messages...' until force-quit #3, fix(opencode): retry server startup on transient SQLite 'database is locked' #18-20 on V2.

Not in scope

Opening upstream PRs; resolving #14 (file chips) or the notification/discovery proposals, which now have a V2 baseline to re-map onto (environment-discovery.md D0 gets client-runtime/src/connection/routes.ts).

Related: #6, #5.

[nohat]

Technical report merged to fork/prod: docs/fork/orchestrator-v2.md (5343913 and a follow-up). It corrects and extends the table in this issue's body:

• New blocker (verified): the live state.sqlite already records migration id 55 PushDevices (2026-10-04 16:57). V2 uses id 55 for OrchestrationV2, and the migrator skips ids at or below the recorded maximum, so V2's tables would never be created. Fix: build statev2.sqlite out of band and delete that one ledger row on the new file only; never touch state.sqlite. Some process running the APNs branch wrote to the live home; cause unknown.
• Correction: ThreadMessageIntake is not a policy gate (scheduled tasks, restart continuation, and wakes bypass it); drain still gates there for user-started turns, steers allowed. The running count is orchestration_v2_projection_runs in preparing/starting/running plus non-held queued; waiting is reported, not blocking.
• Measured: state.sqlite 1.09 GB; consistent copy 26.4 s (VACUUM INTO), the server's own copy 5.1 s; use a 240 s first-boot probe.
• Dispositions changed: the Claude fix, OpenCode waves, and child-agent work are dropped; feat(server): cross-provider subagent dispatch and model-selection skill #21 dispatch is dropped but pricing and the skill stay; scoped agents, fix(opencode): retry server startup on transient SQLite 'database is locked' #18/fix(opencode): lock-specific error detail and serve-process observability on startup failure #20 retry, and the 3 s Claude bounds are ports; the watchdog is deferred; the version label follows versioning.md, not a fixed 1.0.0 or 2.0.0.
• Decisions on the research passes' questions are recorded in the report ("Decisions recorded as assumptions").

Dispatching the sync agent now with scope: pre-flight notes, patches.tsv and delta-check, the merge and ports in a scratch worktree, and a candidate branch sync/upstream-20261005. It does not land, build, or deploy.

[nohat]

Gates built (2026-10-05): docs/fork/patches.tsv (21 guarded features, 1 unguarded: fork-docs-agent-config) and scripts/fork/delta-check.ts with a unit test, on feat/sync-gates 63f8777. All 21 guards pass on fork/prod 29fbaf0. Merged into sync/upstream-20261005 ahead of the upstream merge. Upstream target: upstream/main 194c73f (one commit past the report's 3e6b450); trial merge-tree conflicts: 69 files.

[nohat]

Merge committed on sync/upstream-20261005 (15859ef): upstream/main 194c73f merged into fork/prod 29fbaf0; 69 conflicts (24 modify/delete taken as upstream deletions, 7 file-location, 37 content, 1 add/add). Lockfile and routeTree.gen.ts regenerated, both unchanged from the auto-merge. git merge-base --is-ancestor upstream/main verified.

delta-check fix from the first real run: a file reset to upstream counted as "absorbed"; now it is "lines-dropped" unless upstream already has every fork line (63f8777..bb64189 on feat/sync-gates).

Client ports committed: send-never-settles + send-blocked web (751abdc), mobile send-blocked (10de934), reconnect feedback with failureKind in EnvironmentConnectionSummary equality and server-update text (e3aad86), thread resync (6887ed1), mobile cache bound + stall pill wired to resync (5ef0277), keyboard-controller guard rewritten for upstream's 1.22.6 patch (20d586e). design:check passes. Server-side ports (drain/fork-deploy/cutover, papercuts, provider) in progress on helper branches.

[nohat]

Assumptions recorded during the catch-up (each also in its commit message on sync/upstream-20261005):

• Send: the 120 s client start timeout is dropped (V2 replies at commit); the 30 s local-dispatch expiry is kept via a wrapper around the send body (751abdc). The fork's "Sending..." label is not re-added; upstream now says "Submitting message".
• Resync keeps the fork's module-level registry (requestThreadResync) instead of a ThreadHistoryController.resync method: every caller is a synchronous UI handler (6887ed1).
• Mobile cache bound and stall pill ported as dispatched; bug(mobile): iPad thread open hangs on 'Syncing messages...' until force-quit #3 still needs a reproduction on V2 before it is called fixed (5ef0277).
• Reconnect: "unsupported" guidance shows the compatibility detail, which names the side to update (e3aad86).
• Drain: intake reads the drain service optionally; agent/system sends and createdBy: "user" server paths (usage-limit recovery, scheduled tasks) bypass the gate; a launch with no initial message is allowed; a release counts as V2 when its app.asar names statev2.sqlite; outbox effect rows are not added to the count (465b7f9, 73a42f0).
• Scoped agents: agents is a plain optional field without a decoding default (a default broke upstream's round-trip test) (dfff5a5).
• Orchestrator hints: pricing from the fork's attachModelPricing/readModelRates; recent use is one grouped query over orchestration_v2_projection_runs (4b618b5).
• OpenCode wave 0/1 residue (resume-compaction banner, usage-limit text, refresh-policy doc) reverted to upstream.
• bug(mobile): feed scrolls the reader away from their position while a turn is running #17: upstream's keyboard-controller 1.22.6 patch ships; guard rewritten for it (20d586e). The 19k-pixel simulator replay was not run.
• Papercut snapshot fields: sessionStatus is run=<status> session=<status>, lastProviderEventAt is the newest thread event including user events (07b503a).

Defects filed: #38 (mobile outbox never matches V2 error tags), #39 (stale .v2-import-* dirs upstream), #41 (codex binaryPath re-probe test fails).

[nohat]

Resumed related work here (2026-10-05). Candidate sync/upstream-20261005 is now 09f1496; it includes current fork/prod b0ad7ce, completed versioning, installed-guard source, and sync gates. It has been preserved and remains a candidate, not a deployment.

Server typecheck exposed missing dependencies in the prior port. Fixed production-only model-hints composition so reusable MCP registrations retain their optional fallback; supplied the drain service to the transfer-budget fixture; corrected the model-hints test codec/types. 12 affected MCP tests pass, and all six targeted package typechecks pass (server, web, mobile, client-runtime, contracts, scripts).

19 existing fork feature guards passed. New guard/versioning/sync checks also pass (8 Python hook tests, 37 selected TypeScript checks; unrelated cases skipped). Final delta check against fork/prod: 33 explicitly accepted losses, zero unapproved losses. Fresh review covered the new code and wiring.

The sync runner and bounded weekly cadence review are now built on this candidate (#6). A real local trial against upstream 194c73f measured 31 unresolved conflicts from current V1 fork/prod, mapped 16 features, then aborted and removed only its scratch worktree. Its hourly job is prepared but disabled until cutover.

Still pending: integrated V2 client/native verification, matched packaged builds, and first cutover/deploy. Production data was not opened or changed. #38/#39/#41 remain separate tracked defects; this continuation did not silently mark them fixed. The owner's unrelated dirty iOS/papercut edits remain intact.

[nohat]

Merged and built at David's request.

• Fast-forwarded fork/prod to 09f14960c1 and pushed it. Existing local iOS/doc edits were preserved and restored.
• Built fork version 1.0.0, macOS ARM64, including matching V2 server and web bundles. The contracts build gate passed after installing the new lockfile dependencies in the dedicated build worktree.
• Verified packaged macOS metadata and package version are 1.0.0, and the bundled server chunks contain the V2 database implementation and fork version. Completed release staged under releases/09f14960c1; ZIP retained locally.
• DMG creation failed in host hdiutil with No child processes/invalid plist output. ZIP packaging of the same already-built bundles succeeded. Build ran under the existing governor's 40% CPU cap, with ordinary scheduling priority to avoid background starvation.

No production deployment or live data migration was performed. current remains on 7345a87552; the upstream-sync schedule remains disabled. Integrated client/cutover verification and a matching native mobile build are still pending. This update does not close the tracking issues.