You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Phase D] Post-promotion watch: restore the previous default and pause further rollouts on a regression beyond tolerance #2129
As a maintainer trusting automated promotion,
I want the same metrics monitored after a model is promoted, with automatic rollback on a regression,
so that a wrongly promoted model is reverted within one monitoring interval.
Acceptance Criteria
A scheduled watch evaluates, for each (family, tier) whose head was changed by the pipeline, the canary metrics (cost USD, latency p50/p95, refusal and error rate, spike cap) against the policy tolerances, using the previous default's records as the baseline.
On a regression beyond tolerance, the watch opens and merges a PR that restores the previous default as the chain head, moves the channel back with scripts/cut-release.sh --promote to the prior release, pauses further rollouts for that (family, tier), and records the reason on the onboarding issue with the for-human-analysis label.
The watch's evaluation is a pure function with bats covering no regression, cost regression, latency regression, refusal spike and INSUFFICIENT data (which never triggers a rollback).
The monitoring interval is read from the policy file, and the kill switch stops the watch from acting.
The monitoring-interval value is a policy key; if the policy story did not define one, add it there via this PR's policy change and record it in the ADR.
Pure-logic-plus-bats (ADR-0004).
Uses the same merge- and tag-push-capable credential as auto-promotion (epic untracked prerequisites).
Activity