Skip to content

fix(ci): re-pin the watchdog and file its tracking issue in medley - #319

Merged
kyle-sexton merged 3 commits into
mainfrom
fix/273-watchdog-tracking-issue-repository
Aug 4, 2026
Merged

fix(ci): re-pin the watchdog and file its tracking issue in medley#319
kyle-sexton merged 3 commits into
mainfrom
fix/273-watchdog-tracking-issue-repository

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

standards-sync-stuck-automerge-alert has failed 254 of 254 runs since
2026-07-22. Not mostly — every one. Detection was never the problem; the
issue-filing identity was. The reusable's second token mint carried no
repositories input, so actions/create-github-app-token defaulted to the
calling repository and 404'd on
GET /repos/melodic-software/standards/installation. standards is the sync
source, deliberately not a target, and therefore outside the sync App's
8-repo selected access. A watchdog that dies before it can report is worse than
no watchdog: its silence reads as "all clear".

ci-workflows#346 merged the fix — a required tracking-issue-repository
input — at ed6d410c1fe10d9deb40241bbe46428b31fafd57. That merge changed
nothing here on its own
, because this caller pins by SHA. This PR is what
actually closes the defect.

The destination is medley, and the reason is disclosure, not convenience

The alert body builds a per-PR table embedding each reported PR's repository
name, number, and URL. standards is PUBLIC; four of the eight scanned
targets are PRIVATE (dotfiles, github-iac, medley, provisioning).
Filing the issue here would republish private repository names and PR URLs in
public, every hour, for as long as the condition held. medley is private, is
inside the App's selected access, and is where the reusable's own header
already imagines this alert living.

medley is also one of the scanned targets — it may end up reporting on its
own sync PRs. That is incidental, not the reason it was chosen, and the file
header now says so rather than implying the destination sits outside the scan.

Why the App installation is not widened instead

Adding standards to the sync App's installation would "fix" the 404 and break
everything else: sync.yml's attest job requires the installation's selected
set to equal the derived target set, so any addition fails the sync for
every target. distribution/sync-manifest.yml states the position directly —
standards is "manifest source, not a target". Closed on design grounds. This
is the reason issue #274's suggested fix is not the one taken; see Related.

Three files, and why the second two are not optional

File Change
.github/workflows/standards-sync-stuck-automerge-alert.yml Re-pin 8202e03fed6d410c; pass tracking-issue-repository: medley; correct the now-false header prose.
components/runner-policy/policy.json New approved-contract entry for ed6d410c, allowedInputs grown by tracking-issue-repository.
components/runner-policy/README.md The claim "the caller passes only runner" becomes false with this change.

The runner-policy contract entry is a hard gate, not bookkeeping: without
it, npm run lint:runner-policy rejects both the unknown SHA and the input the
caller now passes. The entry is added, not replaced — this map retains
superseded revisions (six standards-sync.yml SHAs, and the alert's own
43bc8d0f and 8202e03f, are all still present) so consumers still pinning an
older SHA keep a contract. 8202e03f additionally remains live for sync.yml
under a different workflow key.

allowedInputs is exactly the reusable's five workflow_call inputs at
ed6d410c — no input the caller passes is missing, and no input the reusable
does not define is claimed.

The value must be a bare name

tracking-issue-repository: medley, never melodic-software/medley. The
reusable's first step — before the checkout, so a malformed destination fails
early rather than confusingly late — rejects anything outside
^[A-Za-z0-9._-]+$, plus the literal . and ... An owner/repo value mints
a token fine and then breaks downstream, which is precisely the failure the
guard exists to pre-empt.

No caller-side permission change is needed: the reusable mints its own
issue-scoped token, so this workflow's ambient GITHUB_TOKEN still needs only
contents: read.

Test plan

Actual output from this branch, not a description of what would run:

bash distribution/sync-manifest.sh validate  -> Manifest valid: 34 components, 8 targets
npm run lint:runner-policy                   -> Runner policy passed.
npm run test:runner-policy                   -> tests 242 | pass 242 | fail 0
npm run lint:md                              -> 105 files | 0 issues in 0 files
npm run lint:hooks                           -> All good
npm run lint:concurrency-policy              -> Concurrency policy passed.
npm run lint:dependabot-policy               -> Dependabot policy passed.
npm run test:concurrency-policy              -> pass 24 | fail 0
npm run test:dependabot-policy               -> pass 35 | fail 0
npm run test:packages                        -> pass 14 | fail 0
components/pin-comment-convention/scan-workflow-files.sh    -> exit 0
harness/shell/run-tests.sh .../pin-comment-convention.test.sh -> 1 passed, 0 failed
actionlint <the changed workflow>            -> exit 0, no output

Verified from primary sources rather than assumed:

  • The pinned SHA is Link checker report #346's merge commit.
    gh pr view 346 --repo melodic-software/ci-workflowsstate MERGED,
    mergeCommit.oid ed6d410c1fe10d9deb40241bbe46428b31fafd57,
    mergedAt 2026-08-04T02:07:30Z; reachable from ci-workflows origin/main.
  • medley is in the App's selected access. sync.yml run 30870837177,
    attest job: EXPECTED_REPOSITORIES lists all eight targets including
    melodic-software/medley, and the job passed — attest requires equality,
    so this is proof of the whole set, and proof standards is outside it.
  • The App can author issues. gh api orgs/melodic-software/installations
    melodic-standards-sync carries issues: write,
    repository_selection: selected, unsuspended. The mint is satisfiable.
  • The input contract. Read from the reusable at ed6d410c, not from a
    summary: tracking-issue-repository is type: string, required: true, and
    the first-step guard's accepted/rejected forms are as described above.
  • Repository visibility. gh repo view per repo: standards PUBLIC;
    dotfiles, github-iac, medley, provisioning PRIVATE.

An independent fresh-context agent re-derived all of the above with the
author's reasoning withheld, and found a defect that is fixed in this
branch
: the header first claimed the destination was "neither this repo nor
any scanned target", which is false of medley. Commit 9d3fced corrects it.
Its one remaining note is recorded below.

Known limitation of the pin-comment date

The comment reads # ed6d410 2026-08-04, the commit's UTC date, matching
mergedAt. The convention in components/pin-comment-convention/ fixes the
form (YYYY-MM-DD) but never says whose date, and the repo's own precedent is
genuinely split — 31a5b76 2026-07-21 used UTC, f2d5e06 2026-07-14 used the
commit's local-offset date. The machine check accepts either, so no claim of
"matches precedent" would be honest in either direction. UTC is chosen as the
observer-independent timestamp. Pinning this down belongs in the convention
component, not in this PR.

Related

Closes #273.

#274 is deliberately left open, not closed by this PR, and that is a
judgement call worth stating rather than burying:

  • It reports the same failure as standards-sync stuck-automerge watchdog has never run: App token mint 404s on every scheduled run #273 from a second angle, and this PR does
    resolve that symptom.
  • Its suggested fix — "re-add standards to the App's selected-repositories
    list" (items 1 and 2) — is rejected on design grounds, not merely
    unimplemented. The attest equality check makes that change break the sync
    for all eight targets. Closing fix(ci): standards-sync-stuck alert fails hourly — App token minting 404s on /standards/installation #274 with a keyword would record it as
    "fixed as suggested", which is the opposite of what happened.
  • Its item 3 — make the alert fail loudly on a token-minting error, so the
    next occurrence is visible without someone reading run history
    — is a
    genuine, still-unaddressed residual. This PR removes the specific cause of
    the current 404; it does not add alerting on the alarm's own failure. That
    residual is exactly the reason the defect went 254 runs unnoticed, and it
    deserves to survive as tracked work rather than being auto-closed.

Recommended follow-up for a human: rescope #274 to its item 3, and close it as a
duplicate of #273 only if that residual is retracted or refiled elsewhere.

Delivers ci-workflows#346 to this consumer. Does not touch
distribution/sync-manifest.yml — standards#315 landed there at
2026-08-04T02:07:15Z and this branch is built on that main.

After merge, the next 17 * * * * run is the first that can succeed. Nothing
else is required to stop the hourly failures.

What to expect after merge — green, and quiet

Two effects, worth stating so neither reads as a regression:

  • The watchdog will run green and report nothing. chore(distribution): re-disarm fleet automerge until the watchdog is fixed #315 set automerge: false
    on all eight targets, so the never-armed half is inert by design, and the
    armed-but-BLOCKED half has nothing to find — gh pr list --author app/melodic-standards-sync --state open returns no open sync PR anywhere in
    the fleet right now. A green empty scan is the correct outcome, not evidence
    the fix did not take. The first real exercise of the issue-filing path comes
    when automerge: true is restored and a PR actually wedges.
  • The runner-policy component is sync-managed, so merging this cascades
    policy.json to all eight targets as ordinary sync PRs. Adding a contract
    entry (rather than replacing one) is what keeps that cascade safe: a target
    still pinning 8202e03f keeps its contract and its lint stays green.

Because the empty-scan case cannot distinguish "fixed" from "still broken by
some later step", the honest verification after merge is a workflow_dispatch
run confirming the token mint step itself now succeeds — the step that has
been 404ing — rather than inferring it from a green run.

Co-authored-by: Claude Opus 5 (1M context) noreply@anthropic.com

kyle-sexton and others added 2 commits August 3, 2026 22:26
…ley as its tracking-issue repo

The watchdog has failed 254 of 254 runs since 2026-07-22 — never once
succeeded. Detection was never the problem; the issue-filing identity was.
The reusable's second token mint had no `repositories` input, so it defaulted
to the calling repository, `melodic-software/standards` — the sync source,
deliberately outside the sync App's 8-repo selected access — and 404'd on
`GET /repos/melodic-software/standards/installation` before the scan could
report anything.

ci-workflows#346 added the required `tracking-issue-repository` input that
fixes it, but this caller pins by SHA, so that merge changed nothing here.
This is the change that closes the defect.

`medley` is the destination: private, inside the App's selected access, and
not the public repo whose alert body would otherwise republish four private
targets' repository names and PR URLs.

The runner-policy contract gains an entry for the new SHA rather than
replacing the old one — older entries stay for consumers still pinning them,
and `8202e03f` remains live for `sync.yml`'s separate workflow key. Its
`allowedInputs` is the first in this family to actually grow.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
`medley` is one of the eight scanned targets — the header's own motivating
incident, medley#1619, is a medley sync PR. The destination sentence carried
the old wording's shape over and inherited a claim that was true of the
calling repo but is false of medley. What is actually true is that the issue
goes to ONE named repository rather than being scattered across the targets;
the privacy rationale for choosing a private one is unaffected.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cursor

cursor Bot commented Aug 4, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@claude

claude Bot commented Aug 4, 2026

Copy link
Copy Markdown

Claude finished @kyle-sexton's task in 3m 6s —— View job


Code review

Reviewed the three changed files for correctness and alignment with CLAUDE.md/REVIEW.md (code-review lane: correctness, design, conventions, error handling, observability, tests, documentation — security lane N/A findings; this repo has no claude-security-review.yml, but nothing in this diff touches auth, secrets, or externally-built commands, so there's nothing for that scope to pick up either).

No blocking or nit findings. This is a clean, internally consistent fix. What I checked and confirmed:

  • Pin + input correctnessstandards-sync-stuck-automerge-alert.yml#L34-L39: the SHA bump and the new tracking-issue-repository: medley input are exactly what the PR body claims. The pin comment # ed6d410 2026-08-04 satisfies components/pin-comment-convention's fallback form (7-hex prefix of the full SHA, ISO date).
  • Header accuracy#L7-L18: cross-checked the "four of eight scanned targets are private" and "medley is itself one of the scanned targets" claims against distribution/sync-manifest.yml — exactly 8 melodic-software/* targets are listed, medley is one of them. Checks out, including the self-correction in 9d3fced (destination is inside the scanned set, not outside it).
  • policy.json entry#L475-L489: new SHA-keyed entry follows the sibling entries' exact shape (routing, runnerInput, allowedInputs, allowedSecrets), is additive (older SHAs for this workflow key are retained, matching the stated policy of keeping superseded contracts live for laggard consumers), and its allowedInputs/allowedSecrets match the workflow's actual with:/secrets: blocks 1:1.
  • README.md narrative#L556-L564: swept the file for the retired "the caller passes only runner" phrasing — no stale copies remain elsewhere in the doc, so this isn't a partial edit.
  • Design record — the "why not widen the App installation instead" section correctly matches sync.yml's attest job's equality requirement on the selected-repos set, so the rejected alternative is grounded rather than asserted.

Nothing to add beyond what the PR body already documents (including the deliberately-left-open #274 rationale and the pin-comment-date limitation, which are appropriately scoped as follow-up rather than blockers here).

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

Claude has reviewed this PR 1 time. The lane skips further automatic reviews after 5; deleting this comment resets the count.

The parenthetical's antecedent could be read as `medley` — the noun the
sentence opens on — inverting the claim into "medley is not App-installed",
which is the opposite of why it was chosen. Name the calling repo outright.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@kyle-sexton
kyle-sexton merged commit fe3fc91 into main Aug 4, 2026
44 checks passed
@kyle-sexton
kyle-sexton deleted the fix/273-watchdog-tracking-issue-repository branch August 4, 2026 03:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

standards-sync stuck-automerge watchdog has never run: App token mint 404s on every scheduled run

1 participant