Skip to content

[Fix] Skip candidate-only runtime tool checks when verifying the baseline release - #125

Merged
mrubens merged 1 commit into
developfrom
fix/smoke-baseline-tool-checks
Jul 10, 2026
Merged

[Fix] Skip candidate-only runtime tool checks when verifying the baseline release#125
mrubens merged 1 commit into
developfrom
fix/smoke-baseline-tool-checks

Conversation

@mrubens

@mrubens mrubens commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Problem

Every develop publish since #114 fails at the Deployment acceptance gate, which blocks image publishing and nightly deploys (#115/#117/#119 are all stuck undeployed).

The gate's upgrade/rollback validation starts the previous published release (develop-3f602c68) as a baseline, and verify_stack asserts command -v gh / command -v opencode in the api/web/bullmq containers — checks #114 added alongside shipping those tools in the control-plane images. The baseline predates #114, so those binaries legitimately don't exist there and the gate fails one second after the baseline stack reports healthy (see the develop run: all containers Healthy at 19:52:16, failure at 19:52:17, right at the tool checks).

This can never self-heal: the baseline only advances when a publish succeeds, and no publish can succeed while the baseline lacks the tools.

Fix

verify_stack takes a release argument (candidate default, baseline at the two baseline call sites in validate_upgrade_and_rollback). Baseline verification now proves only that the previous release still boots, migrates, and serves health checks; the runtime-tooling contract is asserted against the candidate alone — which is the image the contract is about.

Testing

  • bash -n syntax check; the change is two call-site arguments and a guarded block.
  • The acceptance job itself only runs in the publish workflow (develop/main pushes), so the real verification is the first develop publish after this merges — it exercises the exact failing path (baseline develop-3f602c68 → candidate upgrade → rollback). I'll watch that run.

Note: this is independent of #118 (GHCR login), which addressed the earlier unauthorized baseline pull that #119 also carried a fix for.

…line release

The deployment acceptance gate validates upgrade and rollback against the
previous published release, but verify_stack asserted the runtime tools
(gh, opencode) that #114 newly added to the candidate's control-plane
images. The baseline predates #114, so the gate failed on every publish —
and because publishing is what advances the baseline, it could never
self-heal. Baseline verification now proves only that the previous release
still runs; the tooling contract applies to the candidate alone.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants