[Fix] Skip candidate-only runtime tool checks when verifying the baseline release - #125
Merged
Merged
Conversation
…line release The deployment acceptance gate validates upgrade and rollback against the previous published release, but verify_stack asserted the runtime tools (gh, opencode) that #114 newly added to the candidate's control-plane images. The baseline predates #114, so the gate failed on every publish — and because publishing is what advances the baseline, it could never self-heal. Baseline verification now proves only that the previous release still runs; the tooling contract applies to the candidate alone.
This was referenced Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Every develop publish since #114 fails at the Deployment acceptance gate, which blocks image publishing and nightly deploys (#115/#117/#119 are all stuck undeployed).
The gate's upgrade/rollback validation starts the previous published release (
develop-3f602c68) as a baseline, andverify_stackassertscommand -v gh/command -v opencodein the api/web/bullmq containers — checks #114 added alongside shipping those tools in the control-plane images. The baseline predates #114, so those binaries legitimately don't exist there and the gate fails one second after the baseline stack reports healthy (see the develop run: all containers Healthy at 19:52:16, failure at 19:52:17, right at the tool checks).This can never self-heal: the baseline only advances when a publish succeeds, and no publish can succeed while the baseline lacks the tools.
Fix
verify_stacktakes a release argument (candidatedefault,baselineat the two baseline call sites invalidate_upgrade_and_rollback). Baseline verification now proves only that the previous release still boots, migrates, and serves health checks; the runtime-tooling contract is asserted against the candidate alone — which is the image the contract is about.Testing
bash -nsyntax check; the change is two call-site arguments and a guarded block.develop-3f602c68→ candidate upgrade → rollback). I'll watch that run.Note: this is independent of #118 (GHCR login), which addressed the earlier
unauthorizedbaseline pull that #119 also carried a fix for.