escalate-workflow-outage counts manual dispatches, so a maintainer's own retries escalate as an outage

[JSONbored]

Problem

scripts/escalate-workflow-outage.ts (#10146) counts the leading non-successes at the head of a workflow's run history and escalates at 3. That signal cannot tell a standing outage from a maintainer retrying by hand, because it does not look at how each run was triggered.

Confirmed live on #10171. Six consecutive publish-miner.yml failures escalated to an issue; every one was a manual workflow_dispatch against main, by JSONbored, failing ETARGET on @loopover/contract@^3.19.0 because that version was not yet published. The next run after contract published succeeded with no code change. The workflow was never broken.

Why the distinction matters

The premise the escalation rests on is stated in its own comment: "a deterministic failure fails identically every time." That holds — but so does a human retrying a publish before its dependency is live, and only the first one is an outage nobody is watching. mcp-release-please.yml's reconciliation job already sequences contract → engine → dependents with dispatch_and_wait precisely so the automated path cannot hit this; a hand dispatch bypasses it.

An alert that fires on maintainer-driven retries is the unread-noise failure mode #9951 filed this machinery to escape, just relocated.

Suggested scope

Restrict the failure-streak count to runs the automation dispatched, rather than every run at the head of the history. The GitHub API exposes what is needed on each run (event, triggering_actor, and head_branch — the automation dispatches --ref <tag_name>, a hand dispatch here used main), so no new state is required.

Worth deciding explicitly whether a manual run should be excluded from the count or merely not reset it — excluding is the safer default, since a maintainer's own failed dispatch is already visible to the maintainer who ran it.

Deliverables

• The streak count ignores manually-dispatched runs, so a hand retry cannot escalate.
• A genuine automation-dispatched streak still escalates at 3, unchanged.
• Regression test covering a history that mixes both, pinned to the workflow outage: publish-miner.yml has failed on consecutive runs #10171 shape (manual failures followed by an automated success).

[linear-code]

JSO-7140

[JSONbored]

Correction — the signal this issue proposed does not exist

I filed this asserting that event, triggering_actor and head_branch could separate an automation dispatch from a manual one. They cannot. Verified against the real run history for both publish-miner.yml and publish-contract.yml:

2026-07-31T13:05:09Z  success  workflow_dispatch  actor=JSONbored  branch=main
2026-07-31T11:13:59Z  success  workflow_dispatch  actor=JSONbored  branch=main
2026-07-31T10:51:26Z  failure  workflow_dispatch  actor=JSONbored  branch=main
2026-07-31T10:45:43Z  failure  workflow_dispatch  actor=JSONbored  branch=main
...

Every run is identical on all three fields, whether dispatched by hand or by mcp-release-please.yml. The parent run's actor propagates through gh workflow run, and the reconcile path dispatches against main like a human does. The workflow_dispatch inputs are not exposed on the run object either — GET /actions/runs/:id returns no inputs key, so released_by_release_please is not readable after the fact.

So the deliverables as originally written are unimplementable. Re-scoping below.

What would actually work

Option A — stamp the dispatch origin into run-name

run-name: is rendered by GitHub into display_title, which is on the runs API. This repo already proves the pattern: .github/workflows/visual-capture-fallback.yml:73 sets a run-name from its inputs precisely so a later reader can recover them, and its own comment says so.

Add a dispatched_by input to the publish workflows, have both automation paths in mcp-release-please.yml set it (the direct per-package dispatch and dispatch_and_wait in the reconcile job — note the reconcile path passes no inputs today, so released_by_release_please is false there and is NOT a usable stand-in for "automation"), then filter the streak on display_title in scripts/escalate-workflow-outage.ts.

Cost: five publish workflows plus mcp-release-please.yml. Honest and durable.

Option B — count what the automation actually observed, not the API history

dispatch_and_wait_with_retry already runs up to 3 attempts and only calls the escalation after all of them fail. Those three are, by construction, automation-dispatched and automation-observed. Escalating on that fact needs no run-history query and no classification at all — the ambiguity disappears rather than being worked around.

Caveat: selfhost.yml is the other caller and is not dispatch-based — it fails across ordinary post-merge runs, which is the #9951/#10138 case this script exists for. So the history-based path has to stay for it, and this becomes two modes in one script.

Option C — do nothing to the counting, and accept the noise

The rule fired correctly on its own terms in #10171: six genuinely consecutive deterministic failures. The cost of a false escalation is one issue a maintainer closes.

Recommendation

Option B, scoped to the dispatch-based callers only, with selfhost.yml left on the existing history path. It removes the ambiguity at the source instead of reconstructing intent from metadata, and it deletes an API call rather than adding a stamping convention across six files. Option A is the fallback if a single uniform mechanism is preferred over two modes.

Worth a maintainer decision before implementation, since B changes what "consecutive" means for the publish workflows.

[JSONbored]

Correction: the three fields this issue proposes cannot discriminate. Verified against live runs.

I wrote the "Suggested scope" above asserting that event, triggering_actor and head_branch
are enough to tell a hand dispatch from an automated one. That is wrong, and an implementer
following it would reach a dead end. Measured against the real API rather than assumed:

The automation and the human produce byte-identical runs

GET /repos/JSONbored/loopover/actions/workflows/publish-miner.yml/runs, grouped:

workflow	runs sampled	distinct (event, head_branch, triggering_actor)
publish-miner.yml	100	1 — workflow_dispatch / main / JSONbored
publish-mcp.yml	100	1 — workflow_dispatch / main / JSONbored
publish-contract.yml	9	1 — workflow_dispatch / main / JSONbored

Not a sampling artifact — here is one run proved automated from the dispatcher's own log:

reconcile-stale-releases  2026-07-31T14:10:02Z  Dispatched publish-miner.yml as run 30637452300, waiting for it to complete...
reconcile-stale-releases  2026-07-31T14:12:37Z  publish-miner.yml (run 30637452300) concluded: success

Run 30637452300 is publish-miner run #484, created 14:10:02Z. The API reports it as
event=workflow_dispatch, head_branch=main, triggering_actor=JSONbored, display_title="Publish Miner Package"
— identical on every field to the six manual failures at 10:16–10:51Z that produced #10171.

Why

There are two automation dispatch paths, and only one of them is distinguishable:

1. release-please path (mcp-release-please.yml:181-205) — gh workflow run --ref <tag_name> -f released_by_release_please=true
   under secrets.GITHUB_TOKEN. Lands as github-actions[bot] on a tag ref. Distinguishable — and
   not the path that calls the escalation.
2. reconcile path (dispatch_and_wait, mcp-release-please.yml:285) — a bare gh workflow run "$workflow",
   no --ref, no inputs, under secrets.RELEASE_PLEASE_TOKEN (a PAT, so the actor resolves to its
   owner, JSONbored, and the ref defaults to main). This is the path wrapping
   escalate_persistent_publish_failure, and it is indistinguishable from a laptop by construction.

released_by_release_please is not a stand-in either, on two counts: the reconcile path deliberately
leaves it false (it does need to create the release), and GET /actions/runs/:id returns no inputs
key at all, so a dispatch input is unreadable after the fact regardless.

What actually works

Provenance has to be stamped at dispatch into a field the runs API returns. run-name: is
rendered into display_title, which the runs API does return, and this repo already relies on exactly
that trick — .github/workflows/visual-capture-fallback.yml:73 stamps pr=<n> sha=<sha> into run-name
and actions-fallback.ts's parseFallbackRunCorrelation reads it back out.

So the revised scope is:

• Add an informational dispatched_by_automation input to the five publish-*.yml workflows, stamped
  into run-name:. It must be a new input — released_by_release_please carries real behaviour
  (skip re-creating the release) and cannot be overloaded.
• Pass it from both dispatch sites in mcp-release-please.yml.
• In escalate-workflow-outage.ts, treat a run as automation-dispatched unless it is a
  workflow_dispatch lacking the stamp. Non-dispatch events (selfhost.yml is push-triggered)
  keep counting exactly as they do today, so that caller's behaviour is unchanged.

One consequence worth accepting deliberately: runs already in the history carry no stamp, so they read
as unattributable and are excluded. The publish escalation therefore stays quiet until three stamped
automated failures accumulate. That is the fail-safe direction — under-alerting briefly beats
re-creating the false alarm this issue exists to remove — but it should be stated in the code rather
than discovered.

The three deliverables in the issue body are unchanged; only the mechanism is.