Skip to content

ship-drift reports '0 agents merged-but-unshipped' while all 7 dev-lead stubs are 29-181 commits behind — it compares the 'next' channel, which no stub pins (the #1592 detector is blind to #1592) #1049

Description

@don-petry

canary-rollout drift's ship-drift report is meant to be the detector for petry-projects/.github-private#1592 ("merged ≠ shipped"). It currently reports the fleet as fully shipped while every dev-lead caller stub is 29–181 commits behind, because it measures the wrong tag.

Observed

Run 33454620632 (canary-rollout drift), 2026-09-01:

== agent_ref ship-drift: next channel vs host main (agent_ref-consumed paths) ==
  ci-failure-analyst: shipped — next (33451103a2cf) == petry-projects/.github-private main HEAD
  dev-lead: shipped — next (33451103a2cf) == petry-projects/.github-private main HEAD
ship-drift summary: 0 agent(s) with merged-but-unshipped agent_ref changes

0 agent(s). Meanwhile, what the stubs actually pin:

repo dev-lead.yml pins resolves to behind main
.github dev-lead/v1-ring0 6d362318 (2026-07-10) 181
TalkTerm dev-lead/v1-ring1 6d362318 181
bmad-bgreat-suite dev-lead/v1-ring1 6d362318 181
.github-private dev-lead/v1-stable 40f6c232 (2026-08-18) 29
markets dev-lead/v1-stable 40f6c232 29
google-app-scripts dev-lead/v1-stable 40f6c232 29
ContentTwin dev-lead/v1-stable 40f6c232 29

Seven repos, none current, report clean.

Cause

scripts/canary-rollout.sh ~2578:

a_next="$(channel_commit "$a" next)"
a_main="$(_gh_head_sha "$a_host" "$a_defbranch")"
...
if [ "$a_next" = "$a_main" ]; then
  echo "  $a: shipped — next (${a_next:0:12}) == $a_host main HEAD"
  continue
fi

It compares the next channel to host main. No caller stub pins next — every stub pins a ring/stable tier. So the check answers "has autocut cut a candidate?" and then labels the answer shipped.

The block's own header comment states the intended semantics, and the implementation does not match it:

# pinned channel has NOT yet shipped — the exact failure mode where a PR merges, CI stays
# green, the issue closes, and the change reaches nobody.

That is exactly #1592. The detector for it is measuring next.

Why this is worth fixing rather than tolerating

next being current is the weakest possible signal — it only proves autocut ran. The whole point of #1592 is that autocut working and the fleet being current are independent: autocut has been cutting dev-lead releases normally (up to v139.4.0) the entire time the fleet sat on July code. So this report is not merely incomplete — it is reliably green precisely in the failure mode it exists to catch, and it has been green throughout the ~3 weeks the fleet has been stale.

It also actively misleads recovery work: the natural read of 0 agent(s) with merged-but-unshipped agent_ref changes is "nothing to do here."

Acceptance criteria

  • Ship-drift resolves, per agent, the tag each caller stub actually pins — read the uses: ref (and agent_ref:) from each ring member's stub — and compares that commit against host main over the agent_ref-consumed paths.
  • Report per repo, not just per agent: different members legitimately sit on different tiers (.github on ring0, TalkTerm/bmad on ring1, the rest on stable), so one row per agent cannot express the state.
  • Distinguish expected ring lag from stuck. A stable-tier repo trailing next is normal canary behaviour; the same repo trailing by 181 commits since 2026-07-10 is not. Age or ring-distance thresholds, not raw inequality — otherwise the report is green-when-broken (today) or noisy-always (naive fix), and both get ignored.
  • Flag a pin that no promotion can advance — a stub pinned to a tag outside the agent's current major line (v1-* while the agent is on major 139) is unreachable by promotion and should be loud. That single check would have caught #1592 on day one.
  • Keep the existing next vs main check as a separate, differently-named line (it is a genuine autocut-health signal), so the two are not conflated.
  • Regression test on the current real shape: 7 stubs on v1-*, agent on major 139, next == main → must not report 0 agent(s).

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugBug reports

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions