Skip to content

The CD ledger closes itself, and the parser-delta gate reaches the merge queue - #3549

Merged
meshweaver-cloud[bot] merged 1 commit into
mainfrom
fix/3176-3508-ci-gates
Sep 7, 2026
Merged

meshweaver-cloud[bot] merged 1 commit into
mainfrom
fix/3176-3508-ci-gates

Conversation

@rbuergi

@rbuergi rbuergi commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Closes #3176
Closes #3508

Two CI signals that could not mean what they claimed. Both are placement, not logic — and both are the same shape: a mechanism that cannot express the state it exists to report.


#3176 — the CD alert could never be resolved, so it stopped being an alert

verify-images reports a successful heal onto whichever ci-failure issue is open, and the comment ended "Close this issue if nothing else is outstanding" — advice, to nobody in particular, from a bot — and closed nothing.

Nobody ever did. Because gate and alert-on-failure both append to whichever ci-failure issue is open, the first one ever filed absorbed every attempt and every heal from then on: #3176 reached 324 comments across four days and a dozen unrelated causes, under a title describing one commit's incomplete image set that had stopped being true on its first day. The maintainer's own root-cause comment on it had to open by restating the issue's premise, because the premise had expired.

That takes away both of the things an alert says:

property what it should mean what an immortal ledger made it mean
existence delivery is broken right now nothing — the issue is always open
age how long the outage has lasted how long ago the first ever outage was

The heal now closes it. Nothing is lost — the thread and every comment stay exactly where they were, searchable — and the next failure files a fresh issue (both gate and alert-on-failure create one when none is open), so an alert's title names its own cause and its age is its own outage's age. Closing also hands the reconciler a fresh heal budget, the same effect the 🛑 Automatic healing STOPPED comment asks a human for.

🚨 The comment is fatal, the close is a warning, and the asymmetry is AGENTS.md's own test — what does a failure here hide? A lost heal comment hides the delivery record. A failed close hides nothing: the issue stays open (the behaviour that shipped for months) and the very next successful heal closes it, so a transient 403 is self-correcting. A fatal close would instead red verify-images, which delivery-verdict and alert-on-failure both read as a broken delivery — an alert about the alerting, on a run that delivered.

How it is falsified

The step gained id: heal and takes every value through env:, because .github/scripts/test-cd-steps.py extracts it from the workflow and executes it against a stubbed gh. Its cases assert on the recorded gh invocations, never on stdout — a step that prints the word closing and calls no gh issue close is the defect, and the two are identical in a log.

arm assertion
an open ledger gh issue comment 3176 and gh issue close 3176 were both called
no open ledger neither was called — a silent no-op, not a close of nothing
the close 403s the run stays green, the heal comment is still written, and a ::warning:: says so
the step stops calling gh issue close the harness refuses to run, ::error:: + exit 1

Negative control, run: deleting the gh issue close line makes the harness exit 1 naming it; restoring it returns exit 0, all cases passed … steps release + decide + verdict + heal (extracted, not copied).


#3508 — a gate added to main reached neither build of the one PR it was written for

A pull request has two builds, and a gate added afterwards misses both:

  1. its own pull_request run executes the workflow file as it stood on the merge ref when it ran — a step added to main later is simply not in that file, and re-running re-runs the same file;
  2. its merge_group run — the only build that sees the change at the moment it lands — skipped the whole cross-repo-pair job, whose if: is pull_request only (correctly: its other input is the PR body).

Measured on the queue run for #3546 (gh-readonly-queue/main/pr-3546-…, run 34093021863):

job on merge_group
Public surface (binary compatibility) (record-signatures) completed/success
Cross-repo pair (public surface) completed/skipped

Exit (1) is structural and cannot be closed. Exit (2) is placement. So the parser-delta control moves into record-signatures, which already runs on pull_request and merge_group — and carries no fork or dependabot clause, so the move closes those two exits as well.

The strict/declaring split is written on the EVENT — the only exemption AGENTS.md sanctions, and the same shape the two gates beside it already use for --pr:

event form why
pull_request --pr-body-file supplied the body is where an intended decrease is declared
merge_group no body — STRICT: any decrease is refused a queue entry has no single pull request (its ref can carry several), and nothing should shrink the detector at the moment it lands

Every build gets exactly one of the two; there is no path on which neither runs, which is the property #3508 is about. No new skip that reads as a pass.

🚦 The one trade, stated rather than hidden: a declared decrease is honoured on the pull request and refused in the queue, so a genuine parser tightening lands only if it does not shrink a published counter. That is deliberate — the queue is where the change becomes main, and a declaration read there would have to be attributed across a multi-entry group carrying several pull requests' diffs at once. It is written into Doc/Architecture/CrossRepoPairGate beside the fix.

How it is falsified

The acceptance test the issue names is the point of the issue, so each row below executed the step's own run: block extracted from dotnet-test.yml, not a restatement of it:

arm result
clean tree, EVENT=merge_group exit 0 on the byte-identical proof, both SHA-256 digests printed
a planted blindness in the detector's git-tree reader (MeshWeaver.Layout/ skipped), merge_group exit 1 — publicTypesAtBase 1975 → 1710 (−265), and −1817 / −11 / −46 on the other three
the same plant, pull_request, empty body exit 1, identical verdict
the same plant, pull_request, body declaring all four counters exit 0 — ↓ DECLARED in the pull-request body, so the declaring arm is not lost
plant reverted exit 0 again

The extraction also prints where the step now lives, which is the other half of the claim:

step lives in job `record-signatures`; that job's if:
  "github.event_name == 'pull_request' || github.event_name == 'merge_group'"
Event is 'merge_group', which carries no single pull request:
  the STRICT rule applies — any decrease is refused, declared or not.

Cost: 3 s when the detector changed, and a printed proof when it did not.


Also verified locally

check-workflow-timeouts.py (71 jobs, 0 violations, no new job) · check-workflow-shell.py + its self-test · actionlint -shellcheck= -pyflakes= (the flags CI uses) — clean on both workflows · check-parser-delta.py --self-test (13 cases) · check-type-forwards.py --self-test (76) · check-cross-repo-pair.py --self-test (28) · check-pr-secret-preflight.py (no secret added) · MeshWeaver.Documentation.Test WhatsNewEntryIntegrity + DocumentationLinkIntegrity — 0 Warning(s), 0 Error(s), 2 passed.

No .cs changed.

Pairs-with: none — this diff removes no public type or member from src/; it touches two workflows, one CI harness and three docs.

🤖 Generated with Claude Code

https://claude.ai/code/session_018LzrcSMRzD2R5UcGDk5TKU

Two CI signals that could not mean what they claimed. Both are placement, not logic.

#3176 — the heal comment ended "Close this issue if nothing else is outstanding" and
closed nothing, while `gate` and `alert-on-failure` both append to whichever `ci-failure`
issue is OPEN. The first one ever filed therefore absorbed every attempt and every heal:
324 comments over four days and a dozen unrelated causes, under a title that stopped being
true on day one. That destroys the only two things an alert says — its EXISTENCE ("delivery
is broken right now", which an always-open issue cannot mean) and its AGE. `verify-images`
now closes it; the next failure files a fresh issue, so a title names its own cause and an
age is its own outage's. The comment stays fatal, the close is a warning: a lost comment
hides the delivery record, a failed close hides nothing (the next heal closes it), and a
fatal close would red a delivery leg — an alert about the alerting, on a run that shipped.

The step gained `id: heal` and takes every value through `env:`, so `test-cd-steps.py`
extracts and EXECUTES it against a stubbed `gh`. Its cases assert on the recorded `gh`
invocations, never stdout: a step that prints "closing" and calls no `gh issue close` is
the defect, and the two are identical in a log. The harness refuses to run at all if the
step stops calling it (negative control: removing the call fails the harness, exit 1).

#3508 — a gate added to `main` reaches NEITHER build of a pull request already green. Its
own `pull_request` run executes the workflow file as it stood when the branch was cut, and
its `merge_group` run skipped the whole `cross-repo-pair` job, whose `if:` is pull_request
only. Measured on the queue run for #3546 (34093021863): `Public surface (binary
compatibility)` = success, `Cross-repo pair (public surface)` = skipped.

Exit (1) is structural. Exit (2) is placement, so the parser-delta control moves into
`record-signatures`, which already runs on `pull_request` AND `merge_group` and carries no
fork/dependabot clause — closing those two exits as well. The strict/declaring split is
written on the EVENT, the only exemption AGENTS.md sanctions and the same shape the two
gates beside it use for `--pr`: on `pull_request` the body is supplied so a declared
decrease is honoured; on `merge_group` it is not, and any decrease is refused. Every build
gets exactly one of the two; there is no path on which neither runs.

Falsified by executing the step's own `run:` block, extracted from the workflow:
clean tree/merge_group = exit 0 on the byte-identical proof; a planted blindness in the
detector's git-tree reader = exit 1, publicTypesAtBase 1975 -> 1710 (-265) plus -1817 /
-11 / -46; the same plant on pull_request with an empty body = exit 1; with a body
declaring all four counters = exit 0, so the declaring arm is not lost.

Both fixes are written up (ContinuousDeliveryContract, CrossRepoPairGate) with a What's
New entry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018LzrcSMRzD2R5UcGDk5TKU
Copilot AI lite review requested due to automatic review settings September 7, 2026 07:36
@meshweaver-cloud
meshweaver-cloud Bot enabled auto-merge September 7, 2026 07:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The workflow behavior changes are internally consistent, avoid skip-trapdoors, and are backed by an extracted-step harness that would fail if the key side effects stopped happening.

Pull request overview

Updates MeshWeaver’s CI and its documentation so two previously misleading signals become mechanically truthful: CD “heal” alerts now actually resolve themselves, and the parser-delta gate is enforced in the merge queue (so it can’t miss in-flight PRs).

Changes:

  • Close ci-failure issues automatically on a verified successful CD heal (while keeping “close failed” non-fatal, per the alerting contract).
  • Move the parser-delta control into a job that runs on both pull_request and merge_group, enforcing the strict rule in the merge queue.
  • Extend the CD workflow-step harness to execute and assert the new “heal + close” behavior, and document both placement fixes.
File summaries
File Description
.github/workflows/main-cd.yml Adds a tested “heal” step that comments and then closes the open ci-failure ledger, with non-fatal close failure behavior.
.github/workflows/dotnet-test.yml Relocates the parser-delta control into record-signatures so it runs on merge_group and can’t be skipped in the merge queue.
.github/scripts/test-cd-steps.py Extends the extracted-step test harness to cover the new heal/close behavior by asserting recorded gh invocations.
src/MeshWeaver.Documentation/Data/Architecture/ContinuousDeliveryContract.md Documents the “ledger must close itself” contract and why close failures are warnings.
src/MeshWeaver.Documentation/Data/Architecture/CrossRepoPairGate.md Documents the “gate added to main misses already-green PRs” placement failure and the merge-queue enforcement strategy.
src/MeshWeaver.Documentation/Data/WhatsNew/2026-09-07-a-delivery-alert-closes-itself-and-a-new-gate-reaches-the-queue.md Adds a What’s New entry describing the two placement fixes as user-visible reliability improvements.
Review details
  • Files reviewed: 6/6 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 1)

492 tests  ±0   492 ✅ ±0   41s ⏱️ -4s
  1 suites ±0     0 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 5ad5ac5. ± Comparison against base commit 67bd3d0.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 3)

1 034 tests  ±0   1 034 ✅ ±0   1m 1s ⏱️ -15s
    2 suites ±0       0 💤 ±0 
    2 files   ±0       0 ❌ ±0 

Results for commit 5ad5ac5. ± Comparison against base commit 67bd3d0.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 0)

251 tests  ±0   251 ✅ ±0   1m 55s ⏱️ -8s
  1 suites ±0     0 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 5ad5ac5. ± Comparison against base commit 67bd3d0.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 4)

1 604 tests  ±0   1 410 ✅ ±0   1m 59s ⏱️ -10s
    4 suites ±0     194 💤 ±0 
    4 files   ±0       0 ❌ ±0 

Results for commit 5ad5ac5. ± Comparison against base commit 67bd3d0.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 5)

    5 files  ±0      5 suites  ±0   2m 18s ⏱️ -1s
1 473 tests ±0  1 473 ✅ ±0  0 💤 ±0  0 ❌ ±0 
1 474 runs  ±0  1 474 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 5ad5ac5. ± Comparison against base commit 67bd3d0.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 2)

1 113 tests  ±0   1 113 ✅ ±0   3m 28s ⏱️ -2s
    3 suites ±0       0 💤 ±0 
    3 files   ±0       0 ❌ ±0 

Results for commit 5ad5ac5. ± Comparison against base commit 67bd3d0.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Test Results

   16 files  ±0     16 suites  ±0   11m 24s ⏱️ -40s
5 967 tests ±0  5 773 ✅ ±0  194 💤 ±0  0 ❌ ±0 
5 968 runs  ±0  5 774 ✅ ±0  194 💤 ±0  0 ❌ ±0 

Results for commit 5ad5ac5. ± Comparison against base commit 67bd3d0.

@meshweaver-cloud
meshweaver-cloud Bot added this pull request to the merge queue Sep 7, 2026
Merged via the queue into main with commit 6f47c29 Sep 7, 2026
30 checks passed
@rbuergi
rbuergi deleted the fix/3176-3508-ci-gates branch September 13, 2026 10:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants