Repository navigation
[#1086 slice 1] Gate counts a failure against a candidate only if the run executed it (referenced_workflows attribution) #1120
Copy link
Copy link
Open
Labels
bugBug reportsBug reports
Description
Story
As the canary release gate,
I want to count a failure against a candidate only if that run actually executed the candidate,
so that an unrelated failure in a repo running an older channel can no longer HALT a release as a "REGRESSION".
Slice 1 of #1086 (its AC3, plus the attribution primitive the other slices reuse). #1086's single pass timed out at the 2100s action budget on 2026-09-13 (run 34735591102).
Live evidence: this defect is blocking dev-lead right now
candidate (v139-next) = ae4afa5957d1 cut=2026-09-12T22:30:07Z
next->ring0 BLOCKED dwell=5h/4h sample=171/3 cum_fail=1
##[error]triage=REGRESSION — candidate changed the reusable and a run failed since cut. HALT + hold; recommend rollback.
The single counted failure is petry-projects/.github run 34735591102, dev-lead's own timed-out pass on #1086. It did not execute the candidate:
$ gh api repos/petry-projects/.github/actions/runs/34735591102 --jq '.referenced_workflows[]'
path: .../dev-lead-reusable.yml@dev-lead/v139-ring0 sha: 09d6ddcb (v139.14.0, not ae4afa59)
No repo in the fleet pins dev-lead/v139-next, so this candidate has zero executing runs, yet it is BLOCKED as a regression.
Acceptance Criteria
- Attribution primitive. Add a helper that returns the reusable commit a run executed, read from the workflow-run API's
referenced_workflows[].sha, matched by the agent's reusable path. This needs no log download. Cache it per run, as_run_signaturedoes. _cumulative_healthcounts a failure or startup_failure only when the run executed the candidate commit. Failures from runs that executed a different commit are excluded fromcum_failand reported separately (e.g.other_version=N), not silently dropped.- Fail closed on unknown attribution. If
referenced_workflowsis absent or unreadable for a failed run, count the failure as today. An unattributable failure must never unblock a gate. - Triage (
classify_failure) sees only candidate-attributed failures, so a failure from another version can never produceREGRESSION. - Regression test built on the real case above: candidate
ae4afa59, one failure whosereferenced_workflows.shais09d6ddcb→cum_fail=0,other_version=1, not BLOCKED on that failure. Plus: a failure that did execute the candidate still blocks; a failure with noreferenced_workflowsstill blocks. shellcheck --severity=warning -xclean; tests registered in the existing bats job.
Dev Notes
- Out of scope: sample and dwell coverage and the
NO_COVERAGEstate (slice 2), and the registry's dev-leadnexttopology (slice 3). - Keep it small. The parent timed out. Touch
_cumulative_health, the new helper, and tests only. - Sequencing: canary-rollout ring promotion can skip the v-scoped channel tag, breaking every stub deployed to that tier #1065 and canary-rollout evaluates only the newest candidate — every new cut cancels a pending ring1→stable human confirmation, so dev-lead stable is frozen at 09-07 #1118 also edit
scripts/canary-rollout.shandtests/canary_rollout.bats. Start this slice only after both have merged, to avoid repeated merge-from-main on the same file (see.github-private#1620).
References
- Gate soak is vacuous: dwell/sample count runs from repos that are not executing the candidate — and dev-lead's 'next' tier is structurally unoccupiable (SC2 forbids its only member) #1086 (parent), Define the release soak standard (ring-promotion gate): time floor + traffic-scaled sample + health #548, Enhancement: canary gate should assess correctness, not just run-reliability #668,
.github-private#1620
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugBug reportsBug reports
Activity