Skip to content

review: readiness gate force-rebases speculatively — unrelated merges cancel and restart CI on a phase-locked cohort of PRs #9497

Description

@JSONbored

Summary

prReadyForReview force-rebases a PR before it knows whether that PR is worth touching — before the CI-pending wait, before the gate verdict, before the disposition plan, and before the merge-train check. The trigger is an absolute "default branch is N commits ahead" threshold with no cap, no cooldown, and no merge-readiness condition, so an unrelated PR merging causes CI to be cancelled and restarted on a cohort of other PRs.

This is the reported production behaviour: "whenever a separate unrelated PR is merged, it's retriggering CI on some or all PRs again, rebasing them before it's even necessary."

Mechanism (verified at HEAD, 776c414)

src/queue/processors.ts:4402-4420 — readiness step 1b:

if (typeof aheadBy === "number" && aheadBy >= settings.staleBaseAheadByThreshold) {
  const reason = `default branch is ${aheadBy} commits ahead of this PR's head (threshold ${settings.staleBaseAheadByThreshold}); update-branch before review`;

forceUpdateBranch (:4364-4393) → updatePullRequestBranch (src/github/pr-actions.ts:181-198, PUT …/update-branch).

Ordering is the core defect. This runs at step 1b, ahead of the CI-pending wait at step 2 (:4424-4447). So a PR whose CI is actively running is rebased, and .github/workflows/ci.yml:16-22 uses concurrency: group: ci-${{ github.ref }}-…-pr with cancel-in-progress: true — the push cancels the in-flight run and starts a fresh one. It also precedes the gate verdict and the disposition plan, so a PR about to be auto-closed, sitting on red CI, held for manual review, or merge-train-blocked gets rebased first and the CI run is then thrown away.

The codebase already acknowledges the risk in a neighbouring module — src/services/agent-action-executor.ts:1274-1275: "the worst case is a premature rebase that fires a fresh synchronize and gets re-reviewed on the next pass (#2424)."

The correctly-gated sibling shows what this should look like. maybeForceFreshRebase (processors.ts:4807-4897) fires only when planHasImminentMerge, only when mergeableState === "clean", and is capped at MAX_FRESH_REBASE_FORCES = 3 per PR per 24h keyed on PR number, not head SHA (:4784-4794, whose comment explains that SHA-keying would make the cap unreachable). Step 1b has none of those three guards.

Where the threshold actually comes from (both prior analyses got this wrong)

staleBaseAheadByThreshold: 15 is set in the global private config — /opt/loopover/loopover-config/.loopover.yml:51 (source: selfhost_private_global). It is not a code default, and it is not a DB row: repository_settings contains only JSONbored/awesome-claude and JSONbored/metagraphed, both with stale_base_ahead_by_threshold = NULL, and there is no row for JSONbored/loopover at all.

Implication for the quick fix: because the value lives in the global layer, clearing it there affects all three repos. Scoping it per-repo requires a per-repo private config entry (or a DB row), not a one-line global edit.

Why the threshold makes it recur on the same PRs

The default branch only advances when a PR merges, so "≥15 commits ahead" means "15 unrelated PRs merged since your head". A rebase resets the whole woken cohort to zero-behind simultaneously, so they stay phase-locked and re-cross 15 together — forever.

Production, JSONbored/loopover, 7 days to 2026-07-27:

  • Cohort {8301, 8302, 8337, 8340, 8342} was force-rebased 3 times in 2.5 hours (13:11, 14:06, 15:40 on 07-24) = three full CI re-runs each.
  • Merge counts between those bursts: 15, 10, 5, 14.
  • Largest single fan-out: 7 PRs rebased in 12 seconds (14:06:23.344Z → 14:06:35.203Z), 44s after test(engine): make governor action-mode's coverage visible to Codecov #8452 merged.
  • 625 merges produced rebases in only 34 cases (5.4%) — consistent with "roughly every 15th merge", not with every merge.

The sibling-merge wake (maybeEnqueueSiblingRegateForMergedPr, processors.ts:1402-1437, called at :7442) is the delivery vehicle, not the cause: it wakes up to MERGE_WAKE_MAX_PRS = 15 siblings per merge, staggered index * 10s (src/settings/agent-sweep.ts:37, :1432) — which is exactly the ~10s stagger visible in the audit ledger. Each woken sibling then runs the readiness gate, where this threshold lives.

Note the other rebase path, step 1a mergeable_state === "behind" (:4398-4401), is dead on this repo: branch protection returns required_status_checks.strict: false with no contexts, and GitHub only reports behind under strict.

Requirements

  1. A contributor PR's head must not be mutated unless that PR is the one about to merge.
  2. A PR with CI in flight must never have that run cancelled speculatively.
  3. Repeated rebasing of the same PR must be bounded regardless of head-SHA churn.

Deliverables

  • Move the staleness rebase from the readiness gate to the merge boundary. Delete the step-1b block at processors.ts:4402-4420 and fold the aheadBy >= threshold condition into maybeForceFreshRebase's call site (processors.ts:3896-3900), beside planHasImminentMerge. This inherits the imminent-merge condition, the mergeableState === "clean" condition, and the existing 3-per-PR-per-24h PR-number-keyed cap in one move.
  • Ensure the rebase runs behind the merge-train gate, not in front of it: maybeForceFreshRebase currently executes in processors.ts before executeAgentMaintenanceActions, so a merge-train-blocked PR is rebased anyway.
  • Interim mitigation while the above is implemented: decide whether staleBaseAheadByThreshold should be cleared in the global private config (affects all three repos) or overridden per-repo. On a repo with strict: false and no required contexts, being behind blocks nothing — GitHub will merge a 50-commit-behind PR. If freshness-at-merge is genuinely wanted, requireFreshRebaseWindowMinutes is the already-correctly-gated setting for it (currently unset; fresh-rebase-forced:* has 0 keys in prod, confirming that path never fires).

Tests (must fail against current main)

  • A PR with CI in flight is not rebased by the readiness gate.
  • A PR whose plan is close / hold / manual-review is not rebased.
  • A PR that is merge-train-blocked is not rebased.
  • A PR about to merge, behind by ≥ threshold, is rebased — and at most 3 times per 24h regardless of how many times its head changes.

Expected outcome

Rebases happen to the PR that is about to merge, once, when it is needed — instead of to a phase-locked cohort of unrelated PRs every fifteenth merge.

Metadata

Metadata

Assignees

Labels

gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.maintainer-onlyOwner-only work — yields no Gittensor points.orbGittensory Orb related - maintainer self-hosting analytics.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions