fix(server): keep Codex SQLite files in shared home

[joe-lloyd]

Problem

Codex auth overlays currently symlink SQLite databases and their WAL/SHM sidecars into each account-specific shadow home. On Windows, opening those databases through shadow symlink paths can repeatedly fail, and deleting the links does not stick because the next provider startup recreates them.

Fix

Set CODEX_SQLITE_HOME to the shared Codex home for auth-overlay instances, keep SQLite files out of shadow-home materialization, and remove stale SQLite symlinks left by earlier launches. Real shadow-local SQLite files are preserved.

Materialization now also reads the shadow home so pre-existing SQLite symlinks are cleaned up; a failure to list it surfaces as CodexShadowHomeFileSystemError rather than a raw platform error.

Verification

• pnpm exec vp test run apps/server/src/provider/Drivers/CodexHomeLayout.test.ts
• pnpm exec vp run --filter t3 typecheck
• focused vp lint and vp fmt --check
• Codex CLI 0.152.1 doctor --json with disposable homes confirmed that a shadow CODEX_HOME plus shared CODEX_SQLITE_HOME resolves every SQLite database path to the shared home

Implemented with GPT-5.6 Sol using the Codex harness in T3 Code.

---

Note

Medium Risk
Changes Codex auth-overlay filesystem layout and provider env for SQLite paths; existing shadow homes may lose stale SQLite symlinks on next startup.

Overview
Auth-overlay Codex instances now point SQLite at the shared home via CODEX_SQLITE_HOME, while each instance still uses its shadow CODEX_HOME for per-account auth.

Shadow-home materialization stops symlinking .sqlite files (and journal/shm/wal sidecars) from shared into shadow, and on each run removes stale SQLite symlinks already in the shadow home without touching real shadow-local database files. Listing the shadow directory is now part of that pass, with failures wrapped as CodexShadowHomeFileSystemError.

Tests cover no SQLite links after materialization, repeated materialization preserving separate shared/shadow DBs, stale-link cleanup, and conflict recovery after removing blocking files.

^{Reviewed by Cursor Bugbot for commit 10a6f43. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Keep Codex SQLite files in shared home for auth-overlay instances

• Auth-overlay Codex provider instances now set CODEX_SQLITE_HOME to the shared home path so SQLite state stays shared while account credentials remain private in CodexDriver.create (CodexDriver.ts).
• materializeCodexShadowHome in CodexHomeLayout.ts excludes .sqlite databases and their journal/WAL/shared-memory sidecars from the shared-entry symlink set, and removes stale matching shadow symlinks instead of replacing local files.
• Adds SQLITE_ENTRY_NAME_PATTERN to classify SQLite filenames, and wraps shadow-directory read failures as CodexShadowHomeFileSystemError.
• Behavioral Change: auth-overlay instances now share SQLite state via CODEX_SQLITE_HOME; direct-home instances are unaffected. Reviewers should verify SQLITE_ENTRY_NAME_PATTERN covers all expected sidecar suffixes and that consumers of CODEX_SQLITE_HOME handle the shared path correctly.

^{Macroscope summarized 10a6f43.}

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This changes existing Codex auth-overlay behavior by redirecting SQLite state to a shared home and deleting stale shadow-home links during startup. Because it affects account/session isolation and authentication-related filesystem semantics, human review is warranted.

Notes:

• Diff unchanged. Approvability was decided on eligibility alone.

^{You can add or adjust custom eligibility rules. Learn more.}

Dismissing prior approval to re-evaluate b93ef8f

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit b93ef8fc868b633428d2b675a03d2b72890bcb7d. Configure here.}

[t3dotgg]

Note

🤖 GPT-6 Astra (preview) responding on behalf of Theo

This note is part of an automated cleanup pass.

Preserve the repeat-pass cases from #7201 at head 979c4878. Set up the shadow home once, create a new SQLite database plus WAL/SHM files in both homes, then repeat setup twice. Both real file contents must survive and absent shadow sidecars must not become shared symlinks. Keep the unrelated config.toml conflict test, including recovery after removing the conflict. Include the Windows sidecar-only case on #5817, where no regular shadow database exists. These checks should use this PR's shared-home design, not the older ownership loop.

Dismissing prior approval to re-evaluate ed2d643

[joe-lloyd]

Carried over at 10a6f434, expressed through this PR's shared-home design rather than #7201's ownership loop.

Repeat-pass — keeps both homes' SQLite files intact across repeated materialization: materializes once, then creates queue_1.sqlite plus -wal/-shm as real files in both homes and materializes twice more. Asserts both sides' contents survive and that no shadow entry became a symlink. The future_runtime.sqlite family in the same test covers the absent-sidecar half — its base is a real file in the shadow home while the sidecars exist only in shared, and neither sidecar is linked in.

#5817 sidecar-only — clears stale SQLite links when the shadow home has no database of its own: no shadow database at all, just one stale state.sqlite-wal symlink and one real state.sqlite-shm. The link is removed, the real sidecar survives, and no base link is created. The stale link is deliberately left dangling (the shared WAL is already gone), because that path resolves through readLink rather than stat — if it didn't, a dangling link would surface as a materialization failure and disable the provider, which is the same shape as the original bug.

config.toml conflict — kept, and extended with the recovery leg: both copies are asserted unchanged after the conflict error, then the shadow copy is removed, materialization re-runs, and the link resolves to the shared file.

Both new tests fail against the pre-fix CodexHomeLayout.ts at the merge base, so they are regression coverage rather than restatements of the implementation. The file goes 8 → 10 tests.

• pnpm exec vp test run apps/server/src/provider/Drivers/CodexHomeLayout.test.ts — 10 passed
• focused vp lint / vp fmt --check, and pnpm exec vp run --filter t3 typecheck — clean

Separately re-verified the environment variable on Codex CLI 0.153.2 (the PR originally cited 0.152.1): with a shadow CODEX_HOME and CODEX_SQLITE_HOME set to the shared home, doctor --json reports sqlite home as the shared path and resolves all six databases there — goals_1, logs_2, memories_1, queue_1, state_5, thread_history_1.

The branch is rebased onto 5f878d2a8. CI on the new head is sitting at action_required and needs an approval to run.

Implemented with Claude Opus 5 using the Claude Code harness in T3 Code.

[ElliotDrel]

Windows regression confirmation: regular sidecars still block startup

This reproduces the same sidecar-only shape described in #5817 after the provider had previously been recovered.

• Setup: shared CODEX_HOME plus a second authenticated shadow home on Windows 11.
• Current conflict: goals_1.sqlite-shm; the server logged the exact already exists and is not a symlink failure repeatedly on September 9, 10, and 11.
• Shadow-home state: goals_1.sqlite remains a symlink to the shared home, while goals_1.sqlite-shm and goals_1.sqlite-wal are regular files.
• Impact: the shadow provider is disabled. The desktop backend later exits with an unhandled write EPIPE, leaving the local environment endpoint unresponsive and the desktop app unable to open normally.

For immediate recovery, I am disabling only the affected provider instance. I am not deleting authentication or SQLite state. This is a temporary mitigation, not a fix.

This is a useful end-to-end acceptance case for this PR: an existing sidecar-only shadow conflict must no longer disable the provider or cascade into a desktop-backend crash. The PR's CODEX_SQLITE_HOME design looks intended to prevent this, but it has not shipped in the stable desktop build that exhibits the failure.

[juliusmarminge]

Thanks for working on this. We merged the orchestrator V2 rewrite in #2829, and we are closing this PR as part of that transition.

The patch conflicts with the rewrite in apps/server/src/provider/Drivers/CodexDriver.ts. Even where the conflict is small enough to rebase, we are asking for fresh PRs against the new base so we can review and verify the behavior in V2.

Sorry for the extra work this creates. If the change is still needed on V2, please rebuild it on current main, verify it there, and open a new PR linking back here. We're closing the current implementation without assuming the underlying request is resolved.