Skip to content

fix(lastcode): publish checkpoints when work merges mid-run - #294

Merged
lastobelus merged 5 commits into
lastcode/mainfrom
lastcode/recovery-publish-without-main
Oct 7, 2026
Merged

lastobelus merged 5 commits into
lastcode/mainfrom
lastcode/recovery-publish-without-main

Conversation

@lastobelus

@lastobelus lastobelus commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Merging a PR while a checkpoint was validating threw the run away. Repaired checkpoints published the tag and lastcode/main in one --atomic push, so a mid-run merge rejected the validated tag too. The repair then had to be reselected, with the merge folded in by hand, and fully revalidated. Ordinary checkpoints published their tag but failed the run at promotion, which alerted the maintenance thread. On 2026-10-05, #280 and #282 each cost a manual fold-in and a 10–15 minute revalidation.

The validated tag no longer depends on main:

  • Repaired checkpoints publish the tag and immutable source ref atomically, then promote separately, the same as ordinary checkpoints.
  • Promotion defers instead of failing when main has advanced to a descendant of the candidate's source or git's lease rejects the push because a descendant merge landed mid-push. A rewrite that drops the pinned source still fails promotion after tag publication, and the rejected-push path fetches the competing head before checking ancestry. The run succeeds and logs that promotion is left to the next run. Failure to acquire the promotion lock still fails the run after the tag publishes; the lock ref cannot prove its writer is active, and abandoned locks or authentication/transport errors must remain visible to maintenance.
  • The guarded merge already requests a service run, and the supervisor runs it straight after the current run. That run publishes a revision replaying only the merged work onto the new tag, then promotes it. If main still equals the source, it promotes the published tag directly.
  • Merges after recovery selection no longer invalidate it. Only a main that no longer descends from the selected source (for example, rewritten by another promotion) requires reselection.
  • Recovery cleanup follows promotion validation. A main rewrite or promotion failure keeps the repaired worktree and selection for inspection. Retry promotes the existing immutable tag without republishing; successful promotion or validated descendant deferral releases the repair.

You can merge at any time. The build can start from the checkpoint tag immediately, and merged work follows minutes later as a revision.

Runbook text in fork-conventions.md, release.md, nightly-workflow.md and the lastcode-pr skill is rewritten to match.

Validation: focused checkpoint and recovery tests, selected-recovery lifecycle tests, and carry lifecycle tests pass. Lifecycle regressions cover descendant merges after selection, during validation, and during the promotion push; unrelated remote-only rewrites retain the selected repair and fail visibly while preserving the published tag and rewritten main. They also verify blocked retries preserve that repair, restored-source retries promote the existing tag, and promotion-lock failures retain recovery until a successful retry. The scripts typecheck and scoped format/lint checks pass. GitHub CI and Codex review are required on the final head and base. Quick CI was skipped at the maintainer's request; GitHub CI is the gate.

Includes merged #293 as its current base.

Implemented with Claude Opus 5.5 in the Claude Code harness. Refreshed and repaired with GPT-6.1-Sol in the Codex harness.

🤖 Generated with Claude Code

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-07T03:42:07.378191Z e70ba03 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8875b8b4da

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/lastcode-checkpoint.ts Outdated
@lastobelus
lastobelus force-pushed the lastcode/main branch 2 times, most recently from 4c10f7f to c69c2b8 Compare October 6, 2026 05:15
lastobelus added a commit that referenced this pull request Oct 7, 2026
The promotion lock reference cannot prove that its writer is active. Propagate
lock acquisition failures after tag publication so abandoned locks,
authentication failures, and transport errors still alert maintenance. Keep
successful deferral when a concurrent merge advances main, and preserve the
published tag for a later promotion attempt.

Carry-Group: tooling
Carry-Fix: #294
Carry-Observation: Review found that treating every lock acquisition error as an active merge silently suppressed persistent promotion failures.

Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
@lastobelus
lastobelus force-pushed the lastcode/recovery-publish-without-main branch from 8875b8b to c27cd52 Compare October 7, 2026 02:44
@lastobelus

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c27cd522cd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/lastcode-checkpoint.ts
lastobelus added a commit that referenced this pull request Oct 7, 2026
The promotion lock reference cannot prove that its writer is active. Propagate
lock acquisition failures after tag publication so abandoned locks,
authentication failures, and transport errors still alert maintenance. Keep
successful deferral when a concurrent merge advances main, and preserve the
published tag for a later promotion attempt.

Carry-Group: tooling
Carry-Fix: #294
Carry-Observation: Review found that treating every lock acquisition error as an active merge silently suppressed persistent promotion failures.

Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
lastobelus added a commit that referenced this pull request Oct 7, 2026
Capture the validated recovery selection in a constant before the deferred
promotion callback. This preserves the non-optional selection type without
changing checkpoint publication or promotion behavior.

Carry-Group: tooling
Carry-Fix: #294
Carry-Observation: GitHub CI found TS18048 when the promotion callback captured the mutable optional selection.

Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
lastobelus added a commit that referenced this pull request Oct 7, 2026
Require the competing main to contain the pinned checkpoint source before
reporting successful deferred promotion. Fetch the competing head after a
rejected push so commits created elsewhere can be checked. Unrelated rewrites
preserve the published tag and rewritten main while failing promotion visibly.

Carry-Group: tooling
Carry-Fix: #294
Carry-Observation: Review found that unrelated main rewrites could be mistaken for merged work and suppress the stale-promotion alert.

Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
@lastobelus
lastobelus force-pushed the lastcode/recovery-publish-without-main branch from c27cd52 to d04eb68 Compare October 7, 2026 03:12
lastobelus added a commit that referenced this pull request Oct 7, 2026
The promotion lock reference cannot prove that its writer is active. Propagate
lock acquisition failures after tag publication so abandoned locks,
authentication failures, and transport errors still alert maintenance. Keep
successful deferral when a concurrent merge advances main, and preserve the
published tag for a later promotion attempt.

Carry-Group: tooling
Carry-Fix: #294
Carry-Observation: Review found that treating every lock acquisition error as an active merge silently suppressed persistent promotion failures.

Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
lastobelus added a commit that referenced this pull request Oct 7, 2026
Capture the validated recovery selection in a constant before the deferred
promotion callback. This preserves the non-optional selection type without
changing checkpoint publication or promotion behavior.

Carry-Group: tooling
Carry-Fix: #294
Carry-Observation: GitHub CI found TS18048 when the promotion callback captured the mutable optional selection.

Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
lastobelus added a commit that referenced this pull request Oct 7, 2026
Require the competing main to contain the pinned checkpoint source before
reporting successful deferred promotion. Fetch the competing head after a
rejected push so commits created elsewhere can be checked. Unrelated rewrites
preserve the published tag and rewritten main while failing promotion visibly.

Carry-Group: tooling
Carry-Fix: #294
Carry-Observation: Review found that unrelated main rewrites could be mistaken for merged work and suppress the stale-promotion alert.

Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
@lastobelus
lastobelus force-pushed the lastcode/recovery-publish-without-main branch from d04eb68 to 8d001f7 Compare October 7, 2026 03:15
@lastobelus

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 8d001f7b59

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

lastobelus and others added 4 commits October 6, 2026 20:26
A merge that landed while a checkpoint was validating discarded the run: repaired checkpoints published tag and main in one atomic push, so the validated tag was rejected and the repair had to be reselected and revalidated by hand; ordinary checkpoints published their tag but failed the run at promotion. Publish the validated tag and source ref without touching main, and have promotion defer, instead of failing, when main advanced or a guarded merge holds the main write lock. The merge's own service request then publishes a revision that replays it onto the new tag and promotes that. Merges after recovery selection no longer invalidate it; only a main that no longer descends from the selected source does.

Carry-Group: tooling
Carry-Observation: On 2026-10-05 the merges of #280 and #282 each forced a manual fold-in and full revalidation of a repaired checkpoint.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The promotion lock reference cannot prove that its writer is active. Propagate
lock acquisition failures after tag publication so abandoned locks,
authentication failures, and transport errors still alert maintenance. Keep
successful deferral when a concurrent merge advances main, and preserve the
published tag for a later promotion attempt.

Carry-Group: tooling
Carry-Fix: #294
Carry-Observation: Review found that treating every lock acquisition error as an active merge silently suppressed persistent promotion failures.

Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Capture the validated recovery selection in a constant before the deferred
promotion callback. This preserves the non-optional selection type without
changing checkpoint publication or promotion behavior.

Carry-Group: tooling
Carry-Fix: #294
Carry-Observation: GitHub CI found TS18048 when the promotion callback captured the mutable optional selection.

Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Require the competing main to contain the pinned checkpoint source before
reporting successful deferred promotion. Fetch the competing head after a
rejected push so commits created elsewhere can be checked. Unrelated rewrites
preserve the published tag and rewritten main while failing promotion visibly.

Carry-Group: tooling
Carry-Fix: #294
Carry-Observation: Review found that unrelated main rewrites could be mistaken for merged work and suppress the stale-promotion alert.

Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
@lastobelus
lastobelus force-pushed the lastcode/recovery-publish-without-main branch from 8d001f7 to 082c8f6 Compare October 7, 2026 03:26
@lastobelus

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 082c8f616a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/lastcode-checkpoint.ts Outdated
@lastobelus

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: e70ba03bd3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@lastobelus
lastobelus merged commit 71b8174 into lastcode/main Oct 7, 2026
17 checks passed
@lastobelus
lastobelus deleted the lastcode/recovery-publish-without-main branch October 7, 2026 03:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant