Skip to content

chore(relay): move US cells c32 and c33 to the general same-cap list after promotion - #25367

Merged
Jinwoo-H merged 1 commit into
mainfrom
relay-c32-c33-same-cap-general
Oct 5, 2026
Merged

Jinwoo-H merged 1 commit into
mainfrom
relay-c32-c33-same-cap-general

Conversation

@Jinwoo-H

@Jinwoo-H Jinwoo-H commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 2 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​26 $\color{#cf222e}{\Huge{\mathbf{−}}}$​14 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​12
Prod 2 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​5 $\color{#cf222e}{\Huge{\mathbf{−}}}$​6 $\color{#cf222e}{\Huge{\mathbf{−}}}$​1

ELI5

The two new US relay servers, C32 and C33, started serving regular traffic on 2026-10-01. Our rollout tooling still lists them as "spare only". Because of that mismatch, one rollout mode could quietly switch a live server back to spare-only, which pushes its users off it. This updates the list to match production.

Merge order: this must land before #25336 (the C34 Asia spare). #25336 adds C34 next to C32/C33 in the same list. Landing this first leaves that PR one small conflict to rebase (C34 alone joins C17/C18), rather than carrying the stale entries forward.

What Changed

Before:

  • The same-cap roll job's SAME_CAP_MIGRATION_ONLY_CELLS still lists C32 and C33.
  • The live selector (generation 344, read-only inspect run 37248143823) has both cells in general. They were promoted by Asia-admission runs 36918046200 and 36919480902.
  • canary-apply and batch-apply on either cell fail closed: the admission precheck expects migration-only, sees general, and stops before any change.
  • rollback mode prechecks general-or-migration-only, so it passes. It then runs isolate, which commits general → migration-only plus the roll stamp. Only after that does the changed == false check for migration-only cells fail (cloud-deploy-relay-production-same-cap-job.yml around line 531). The result is a live general cell demoted, with its hosts re-placed off it on their next dial. Recovering means re-promoting it through the Asia-admission canary.

After:

  • C32 and C33 are in the general part of SAME_CAP_CELLS, and only C17 and C18 remain migration-only. A roll of C32 or C33 now isolates and restores the cell (selector delta 2), the same as every other general cell, and returns it to general.
  • FLEET_POOL_CELL_IDS is unchanged. That list tracks the 16-connection Asia pools, and C32/C33 run the US default of 10. The workflow guide said this would be decided at promotion; it now records the decision.

Mechanism, one file at a time:

  • relay-production-same-cap-wave.mjs: the migration-only list sets each cell's entry admission and selector delta. It also forbids mixing the two classes in one wave.
  • Tests:
    • The list test now pins only C17 and C18.
    • New cases show C32 and C33 enter as general with delta 2 and can batch beside C26.
    • A wave mixing C32 with C17 is refused.
    • The census test no longer needs its unpromotedSources exception.
  • docs/relay-workflows.md records the promotion.

Why

This mirrors #24318 (C31) and #22439 (C30), the follow-ups each earlier promoted cell got. A review of #25336 found it (review-c34-pr.md, finding F1). No Terraform values, workflows, or other cells change.

Linked Issue

N/A, maintainer follow-up to the C32/C33 promotions.

Visual Proof

N/A. Release tooling only, no UI.

Testing

  • Automated tests added/updated
  • cd cloud && node --test dev/scripts/*.test.mjs: 825 tests, 824 pass. The one failure is relay-load-control-peer.test.mjs, which this PR does not touch. It cannot load tweetnacl in a lane worktree without installed dependencies.

Review

Agent skill upstream boundary

  • Not applicable

Notes

Security, cross-platform, SSH, mobile: not applicable. Until this merges, do not dispatch the same-cap job in rollback mode on C32 or C33.

Checklist

  • This PR is small and focused
  • I explained what changed and why
  • N/A for visual proof with reason
  • Self-reviewed for correctness, security, and performance
  • Cross-platform, SSH/remote, and path/shortcut impact considered (N/A)
  • pnpm lint, pnpm typecheck, pnpm test, and pnpm build pass (cloud scripts are outside these; CI will cover)

…after promotion

Both were promoted to general on 2026-10-01 (selector 344 shows them general),
but the same-cap job still classed them migration-only. Rollback mode on either
would isolate (general -> migration-only) before its no-op check failed, demoting
a live cell. They stay out of the fleet pool list (pool 10, not 16).

Claude-Session: 1145a80d-dec4-4a9b-9373-bbbb876b9041

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — the four-file promotion of US relay cells C32/C33 from the same-cap migration-only list to the general list, and the tests/docs that record it.

  • List change: SAME_CAP_MIGRATION_ONLY_CELLS drops to [c17, c18], and C32/C33 join the explicit general tail of SAME_CAP_CELLS, flipping entryAdmission to general and selectorWaveDelta to 2.
  • Wave tests: pin migration-only to C17/C18, add C32/C33 general/delta-2 assertions, a c26,c32,c33 general batch, and a c32,c17 mixed-wave refusal.
  • Census invariant: removes the unpromotedSources exception, so SAME_CAP_CELLS minus REHOME_SOURCE_CELLS now equals SAME_CAP_MIGRATION_ONLY_CELLS exactly.
  • Docs: record the 2026-10-01 promotion and why C32/C33 stay out of the fleet pool list (US default pool of 10).

I traced the changed list against its consumers: entryAdmission/selectorWaveDelta drive the same-cap job's drain, isolate, and restore branches; the job's c32|c33 cell-shape case (cap 3000, us-central1, pool default 10) is unaffected and still correct; FLEET_POOL_CELL_IDS correctly stays C27-C31; and no other migration-only list in the tree hardcodes C32/C33. Ran the three same-cap test files (node --test): 51/51 pass.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

@coderabbitai

coderabbitai Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 9bff33b6-d67b-4ea8-b8be-0ec89d5038f0
📥 Commits

Reviewing files that changed from the base of the PR and between d9846e6 and d7db11d.

📒 Files selected for processing (4)
  • cloud/dev/scripts/relay-production-same-cap-wave.mjs
  • cloud/dev/scripts/relay-production-same-cap-wave.test.mjs
  • cloud/dev/scripts/relay-same-cap-script-census.test.mjs
  • cloud/docs/relay-workflows.md

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 2 remain after this review.


📝 Walkthrough

Walkthrough

The same-cap script now classifies C32 and C33 as general cells instead of migration-only cells. Tests cover their general admission, selector delta, and batching rules. The census test and workflow documentation reflect the updated classification.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to d7db1

The changed checks and workflow agree with the reported C32/C33 promotion. No actionable defect is established; verify the production selector state before dispatching the updated job.

Security Architecture Review

Security architecture risk: 🔵 Low · up to d7db1

The change puts C32 and C33 on the normal isolate-and-restore rollout path without widening the approved cell set or granting new permissions. Risk is low, but depends on both cells having completed promotion: rollback accepts migration-only cells and would now restore these two as general.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The PR-specific lifecycle change affects C32 and C33 and their connected hosts in production. Operations reach the director’s shared admission selector and the selected cell’s drain endpoint. The existing deploy identity already has broad admin-route authority; this reclassification does not grant that authority or widen the approved target set.

Trust Boundaries and Controls

  • observed — Workflow dispatch supplies operator-selected arguments through validation, with main-branch and production-environment gates. The preparation CLI checks approved membership and exact director and cell origins before attaching its admin bearer token. The selector endpoint requires an admin token verified against Google issuer, audience, signature, and configured service-account identity. Effective production access policies were not inspected.

Resilience and Maintainability Implications

  • observed — Existing containment controls remain relevant to the newly general cells: concurrent selector changes are rejected, ambiguous client responses are checked against attempt state and exact selector membership, and failed mutations trigger best-effort re-isolation. Recovery distinguishes a still-draining stranded cell from an already restarted cell. The workflow explicitly warns that timeout cancellation skips its failsafe; this limitation predates the PR.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the main change: moving US cells C32 and C33 to the general same-cap list after promotion.
Description check ✅ Passed The description covers the change, reason, mechanism, testing, and visual-proof status. It explains that the author is a maintainer and that no linked issue is needed. It also reports the test failure…
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 3…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant