Repository navigation
feat(overleaf): coordinator local half — apply, recover, adopt, kept copies (step 3b) - #630
Merged
Merged
Conversation
…s-20261022' and 'origin/claude/overleaf-checkout-barrier-20261022' into claude/overleaf-coordinator-local-20261022
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s-20261022' and 'origin/claude/overleaf-checkout-barrier-20261022' into claude/overleaf-coordinator-local-20261022
… test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s-20261022' and 'origin/claude/overleaf-checkout-barrier-20261022' into claude/overleaf-coordinator-local-20261022
… room first Codex round 1 on step 3b: - Kept copies are identified by their place in the retention folder, so every version a step kept of one path is recorded and restorable. - The bulk-change settler fails closed: an unreadable connection list or record, or a folder it cannot look up, refuses the bulk change. - Restore checks room for what it will keep before linking; confirm counts up to three kept versions per changed path. - The apply link carries the local-only set the review counted, and adoption uses exactly that. - A restored file takes part in the sync again (managed, not local-only), so the next review sends it; restores report an unchanged base generation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ked, provenance Codex round 2 on step 3b: - One kept record per (apply, path); every version a step kept (main, staged, provisional, a retried step's) is listed and restorable from disk. Retries can no longer outgrow the room check, which is now exact per changed path. - Every sync and restore rechecks older releasable copies and records them changed or ambiguous, so a late write is never released. - Each record keeps whether its bytes were unresolved Scient conflict material; restoring that version tracks the path's markers again. - Restore keeps a local-only file local-only, and checks the managed-path limit before writing. The limit error says which limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…admission Codex round 3 on step 3b: - Each kept record names every version the applier reported for its path (bounded; beyond it the record is ambiguous), so listing and restore no longer depend on the applier's step records on disk. A record with any further version is never releasable. - Rechecks compare a size-and-change-time stamp recorded when kept, with no hashing; listing re-reads only changed or extra versions and each current file once. - Managed paths after an apply are the reviewed result's paths, bounded by the review's file limit; paths an interrupted unit kept stay in the base. - Every operation takes the store's writable admission (process lease) before touching the private repository; the store exposes admitWritable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…inator-local-20261022
Contributor
Thread transfer impact✅ Thread transfer remains within every enforced ceiling.
Baseline: Scenario and decoded snapshot size10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.
Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed. |
…inator-local-20261022
The alignment brought `prefer-effect-fn` and `no-real-time-wait-in-tests`. `make` and `keptCopies` are now `Effect.fn` functions (traced spans, no behaviour change). The bulk-change test no longer sleeps 100 ms for the competing bulk change to reach the barrier: it waits until the checkout's epoch moves, which the bulk change does as it enters, before it waits for the apply. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ptured, bounded reads Codex round 4 on #630: - A kept copy's stamp was taken after the applier hashed it, so a late writer's same-size rewrite in between looked like the kept bytes forever. Adoption now reads the copy afresh (stat before and after a bounded read): only bytes that still match what the applier kept get a stamp; anything else is recorded changed (ambiguous when gone) and is never released. - A path an interrupted group left in neither the base nor the result (an undone rename's new name) fell out of capture, and the next review refused it as unreviewed. Capture now reads every forced-conflict path too; nothing stored grows. - The bulk-change settler checked for structural copies only before its inline recovery; one the recovery found was then erased by the bulk change. It now re-reads the adopted record and refuses while copies remain. - Restore reads the kept copy once, never past 50 MiB, and restores exactly the bytes it checked. Listing reads within one inventory's total; files past it or past the restore limit are listed `unread`, not gone. The bulk-change ordering test checks at adoption that the bulk change had entered the barrier, instead of polling for it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r-file limit Codex round 5 on #630: - An interrupted group is stored whole as a forced conflict, and one group can exceed the record's 2,000 paths (a 2,000-file folder replaced by a file). Adoption then failed forever, leaving the apply linked. The applier's unit computation is now shared (`interruptibleGroups`), and confirm checks every group an interruption could leave before linking or writing anything. The record holds up to 4,000 groups: one sync changes at most the 2,000 files it captured and the 2,000 of its result. - A kept file swapped at its path right after being opened still read as the unchanged copy. A read now also requires the path to name the file the descriptor read. - Listing and restore read the folder's file within the same per-file limit as kept copies; a restore never takes a file too large to read for an absent one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex round 6 on #630: restoring a kept copy that held conflict material tracks its path again, and with 2,000 marker paths already recorded that took the record past its limit only after the file was written, leaving the restore linked for good. Restore now checks room for the marker paths it would track, with its other room checks, before reading, linking or writing anything. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rced conflict Codex round 7 on #630: restore adoption kept the record's forced conflicts but dropped the group the applier reported as interrupted, so a restore another writer interfered with left nothing to review and the next review called the folder up to date. Restore adoption now keeps the recorded groups and adds the interrupted ones, without duplicates or truncation, and a restore checks room for the one group it may leave before writing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
yaacovcorcos
marked this pull request as ready for review
October 23, 2026 11:19
yaacovcorcos
added a commit
that referenced
this pull request
Oct 23, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
Step 3b of the Overleaf plan (#535; design in
docs/internals/overleaf/step-3b-design.mdthere): the coordinator's local half.OverleafCoordinatorprepares a review, applies a confirmed one to the project folder, recovers an interrupted apply, and adopts the result into the connection record. It also lists the copies Scient kept of replaced files and restores one on request.It never publishes. Step 4 continues from an adopted result, and step 5 connects it to the app behind sign-in. Until then nothing calls it, so users see no change.
How it works
Locks, in order. The connection's operation reservation comes first: it's held in memory, and a second operation is refused (
OverleafOperationActiveError), not queued. Then come the process's writable lease (admitWritable), the checkout barrier's application section (#626), the record's compare-and-set, and the file layer's own locks. Overleaf is fetched before the section is taken.Confirm:
Adoption is one replacement of the connection record, so it happens exactly once. It sets:
Recovery resumes the applier by the linked ID and adopts its result once. A link whose apply never recorded its plan is cleared as a replan.
Kept copies:
unread, never as gone.Bulk changes. The coordinator registers a settler with the barrier. Before a pull, branch switch or checkpoint restore of a checkout with an unfinished apply:
Record changes
Schema version 1 is unchanged, because nothing is stored in production yet.
attentionPathsbecomesforcedConflicts,markerPathsandstagingPaths.remoteCommit,localOnlyandpriorMarkers, so recovery can adopt without the review.kept,versions,stampandmarkers.interruptibleGroups, shared with its own planning).Tests
OverleafCoordinator.test.tshas 35 tests, using real folders, a real private Git repository and the real native applier, against a scripted Overleaf.vp test run src/scient/overleaf src/scient/workspace: 562 tests pass. Typecheck, lint and knip are clean. These need macOS native publication and are skipped on Linux CI.Review
Codex (gpt-6.1-sol, high reasoning) reviewed the 3b branch in eight rounds. Rounds 1–7 found the problems below, all fixed, each with a test that fails without its fix. Round 8 found nothing: merge-ready (89 focused tests, 30 independent probes and three real crash-and-recover runs passed).
prepareskipped the process lease; non-main versions weren't rechecked; a damaged step record hid versions; hashing had no overall budget.Since round 4 the branch has also taken main (including the upstream alignment #614). Its only clashes were re-indentation, and it now follows main's new lint rules.
Follows #624 and #626.
🤖 Generated with Claude Code