Skip to content

CD failed on main: run 35257430439 #4648

Closed

Description

@github-actions

CD failed on main: run 35257430439 for commit 877ba7bb908b3cacbdb6801707eb75fd8715b5c8 (entered via push).

Triage the failed job in the linked run - which of the two things CD delivers is broken depends on WHICH job went red:

  • an IMAGE job / promote / verify-images: no complete image set was published. Publication is all-or-nothing, so every self-updating install stays on the PREVIOUS image until a CD run promotes the full set.
  • preflight / notify-platform-update: the images may be fine, but the release EVENT did not reach the control instance, so no node repo re-bakes against this commit until its own schedule poll (Release broadcast has never delivered: notify-platform-update succeeds, Hosting/PlatformBuilds stays empty, no satellite dispatch since 08-22 #2235). Read the job's ::error:: line - it names what to provision or what refused the delivery.
  • satellite-compat (a matrix leg) / satellite-compat-image: the set IS published and the release event WAS sent - what failed is the per-build measurement that a satellite's NodeTypes still compile against it (the counterweight to the daily rebuild, 2026-09-12). A red leg means THIS BUILD BROKE that satellite: its types compiled against the set the fleet is on and do not against this one (a satellite that was already broken, or an unestablished baseline, is an advisory on the run page, never a red). The lines under Compatibility below name the types. No image and no publication is missing.

The hourly reconciler will re-attempt main's HEAD on its own (bounded at 3 attempts per commit, logged here); you do NOT need to force a merge to trigger delivery. This issue CLOSES ITSELF when a CD run reports a successful heal (#3176) - a later failure files a fresh issue rather than appending here, so this one's title and age stay true to its own cause.

Activity

  1. added
    ci-failureA main-branch CD run failed - no deployable image was published
    on Sep 17, 2026
  2. github-actions commented on Sep 17, 2026

    @github-actions
    ContributorAuthor

    CD failed on main: run 35261268480 for commit e527101202356b981ae947b306a93d078daca83b (entered via push).

    Triage the failed job in the linked run - which of the two things CD delivers is broken depends on WHICH job went red:

    • an IMAGE job / promote / verify-images: no complete image set was published. Publication is all-or-nothing, so every self-updating install stays on the PREVIOUS image until a CD run promotes the full set.
    • preflight / notify-platform-update: the images may be fine, but the release EVENT did not reach the control instance, so no node repo re-bakes against this commit until its own schedule poll (Release broadcast has never delivered: notify-platform-update succeeds, Hosting/PlatformBuilds stays empty, no satellite dispatch since 08-22 #2235). Read the job's ::error:: line - it names what to provision or what refused the delivery.
    • satellite-compat (a matrix leg) / satellite-compat-image: the set IS published and the release event WAS sent - what failed is the per-build measurement that a satellite's NodeTypes still compile against it (the counterweight to the daily rebuild, 2026-09-12). A red leg means THIS BUILD BROKE that satellite: its types compiled against the set the fleet is on and do not against this one (a satellite that was already broken, or an unestablished baseline, is an advisory on the run page, never a red). The lines under Compatibility below name the types. No image and no publication is missing.

    The hourly reconciler will re-attempt main's HEAD on its own (bounded at 3 attempts per commit, logged here); you do NOT need to force a merge to trigger delivery. This issue CLOSES ITSELF when a CD run reports a successful heal (#3176) - a later failure files a fresh issue rather than appending here, so this one's title and age stay true to its own cause.

  3. github-actions commented on Sep 17, 2026

    @github-actions
    ContributorAuthor

    🔁 CD reconcile attempt 1/3 for bf0960c (run).

    main's HEAD is missing part of its image set in ACR (that run's annotations name which repositories), so this run is publishing the full set for bf0960c. The commit whose CD run originally failed is NOT re-published — it is superseded by HEAD on purpose: the version tag comes from the BUILDING run's number, so re-publishing older code would mint a higher -ci.<n> for it and roll every install backwards.

  4. github-actions commented on Sep 17, 2026

    @github-actions
    ContributorAuthor

    ✅ Healed 3a48c3f (run) — main's HEAD now has the complete image set (portal 3.0.0-ci.8867), verified against ACR. Installs pick it up on the build-completion event, or within one safety-net check interval if that event never reaches them (#2494).

    Closing this issue, because the delivery hole it records is filled. A later failure files a FRESH ci-failure issue rather than appending here (#3176), so an alert's title names its own cause and its age is the outage's age. Reopening is never needed — and closing also hands the reconciler a fresh heal budget.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ci-failureA main-branch CD run failed - no deployable image was published

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions