Repository navigation
fix(cloud): validate protocol-0 same-cap cell plans without rehome trust lines - #18818
Conversation
There was a problem hiding this comment.
ℹ️ No critical issues — one minor suggestion inline.
Reviewed changes
- Protocol-aware plan validation —
validate-relay-capacity-plan.mjsgains a--regional-rehome-protocol 0|1flag: protocol 1 keeps the exact rehome-trust expectations, protocol 0 requires both rehome lines to be absent and fails closed on any rehome drift (unexpectedRehome). - Workflow plumbing — the same-cap job now passes the protocol on both validator invocations (the resume drift check and the apply-step validation), so the plan check and the runtime-status check agree by construction.
- Tests — a unit matrix for the protocol rules plus a census test that derives each same-cap cell's protocol from
relay_region_rehome_source_cell_idsinproduction.tfvars, cross-checks it against the job's region mapping, and validates a correct roll plan at each cell's protocol while asserting the opposite protocol rejects it.
DeepSeek Pro (free via Pullfrog for OSS) | 𝕏
| !values['rehome-director-service-account'] || | ||
| !values['rehome-audience']) | ||
| !values['rehome-audience'] || | ||
| !['0', '1'].includes(values['regional-rehome-protocol'])) |
There was a problem hiding this comment.
A bad value like --regional-rehome-protocol 2 falls into this clause and reports "requires rollback image and rehome trust config", even though those flags were supplied — the accurate "invalid regional rehome protocol" message in rehomeProtocol is only reached on a direct validateCapacityPlan call, never through the CLI. Consider a dedicated protocol check with its own message for the parse path.
📝 WalkthroughWalkthroughThe change adds Merge Risk: 🟡 Moderate · up to This change enables protocol-0 cells without rehome trust configuration, but validation can still accept some retained rehome-variable lines. That can leave unintended rehome settings in production startup scripts, so the matcher and regression coverage should be corrected before merge. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 30.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 4 files. (1 skipped: 1 unsupported.)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Team
Run ID: ce35d41c-212e-4589-95a6-5a78f03f8918
📒 Files selected for processing (5)
.github/workflows/cloud-deploy-relay-production-same-cap-job.ymlcloud/dev/scripts/relay-regional-rehome-workflow.test.mjscloud/dev/scripts/relay-same-cap-script-census.test.mjscloud/dev/scripts/validate-relay-capacity-plan.mjscloud/dev/scripts/validate-relay-capacity-plan.test.mjs
Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review.
| const unexpectedRehome = | ||
| config.mode === 'same-cap-cell' && | ||
| !rehomeTrusted && | ||
| lines.some((line) => REHOME_CONFIG.test(line)) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Detect all rehome-variable assignments for protocol 0.
REHOME_CONFIG matches only canonical assignments with a non-empty value. If both scripts retain printf 'ORCA_RELAY_REHOME_AUDIENCE=%s\n' '', or another noncanonical assignment form, unexpectedRehome is false and the normalized comparison still passes after image replacement. This accepts a protocol-0 script that contains a rehome line.
Use a presence matcher that detects both rehome variable names regardless of assignment value or rendering. Add a regression test where the same unmatched assignment exists before and after the replacement.
…ust lines (stablyai#18818) Mirror-Of: 9f2a9a2

What happened
Run 33957151726 rolled the first Asia same-cap cell,
production-gce-c27(asia-east2). It failed after the reversible isolate/drain and before any apply, in "Apply only the selected same-cap template and MIG":The Terraform plan was correct (template replaced to the target image, MIG updated in place). The validator was wrong. The failsafe held: c27 stayed migration-only with rehome disabled, and nothing was applied.
Root cause
In
same-cap-cellmode,requireDesiredStartupScriptunconditionally required exactly oneORCA_RELAY_REHOME_DIRECTOR_SERVICE_ACCOUNTline and oneORCA_RELAY_REHOME_AUDIENCEline with exact values.relay-gce-startup.sh.tftplemits those two lines only whenrehome_source_enabled, which iscontains(var.relay_region_rehome_source_cell_ids, each.key). That list inenvironments/production.tfvarsholds the 16 US cells (c7-c10, c13-c16, c19-c26). The Asia cells c27-c29 are not rehome sources, so their startup scripts have no rehome lines and their runtime reportsregionalRehomeProtocol: 0. The expectation was written when every same-cap cell was a US rehome source.Fix
validate-relay-capacity-plan.mjstakes a new--regional-rehome-protocol 0|1, required insame-cap-cellmode and rejected in every other mode.The job passes
--regional-rehome-protocol "${DESIRED_REHOME_PROTOCOL}"on bothsame-cap-cellinvocations, the resume drift check and the apply-step plan validation. That is the same value already asserted against the live runtime byverify-relay-capacity-transition.mjsand by the preflightruntime-statuscheck, so the plan check and the runtime check now agree by construction instead of by coincidence.normalizedStartupScript's strip-rehome behaviour is unchanged; for protocol 0 it is a harmless no-op since neither side has those lines.Unchanged
cloud-deploy-relay-production-capacity-job.yml(US-only,--mode bootstrap-cell).same-cap-imagemode, used by the staging c4 recovery and staging capacity proof workflows.rehome-protocol == '1', so it correctly skips for Asia.Tests
validate-relay-capacity-plan.test.mjs:0,'',2,yesand a missing value are all rejected as an invalid protocolsame-cap-cellmode and rejected inbootstrap-cellmoderelay-same-cap-script-census.test.mjs:same-cap-cellvalidator invocations in the job carry--regional-rehome-protocol "${DESIRED_REHOME_PROTOCOL}"SAME_CAP_CELLS, the protocol derived fromrelay_region_rehome_source_cell_idsinproduction.tfvarsmatches the region the job's own cell-shape block resolves, and the validator accepts a minimal correct roll plan at that protocol while rejecting the same plan at the other onerelay-regional-rehome-workflow.test.mjs: the pinned resume-call assertion now requires the protocol flag betweenhost-drainand thejq -e '.changes == 2'pipe.Full
cloudsuite: 529 passing, 0 failing.pnpm typecheckincloudclean.