Repository navigation
Conversation
…d it asked for Systemorph/MeshWeaver.Plugins#781. `LastReleaseRequestHandledAt` is stamped at DISPATCH, which is correct as a re-fire guard and wrong as a delivery receipt — and the field was doing both jobs. Between the dispatch and a Release node existing there are three ways to end up with nothing, and all three were silent and indistinguishable from "no release was asked for": the create's bound expired through a bare `Timeout(_, Observable.Return(null))` with no log line at all; the create faulted or was refused (it runs under the REQUESTER's identity, so a partition the requester cannot write refuses it) and was swallowed at Warning; or the request was answered by the build already in hand (#1707 slice 3), which dispatches nothing and therefore creates nothing. In every case `ApplyCompileSuccess` stamps `releasePath ?? def.LatestReleasePath` — the PREVIOUS build's release — and the trigger is spent. Measured on `Publish/Deck`, 2026-08-27: status Ok, sources clean, lastCompiledVersion 575, latestAssemblyPath naming v575's bytes, and latestReleasePath naming a release cut the previous morning. Every single-field read is healthy; only holding the release up against the build shows it. Every consumer following LatestReleasePath — the build protocol, a release-pinned activation, the "→ release" link — therefore served the previous assembly under a green Ok. And the state was unrepairable from the outside: a second, ordinary request was absorbed by the same satisfied-branch, so asking again could not fix it. The invariant, from the issue: LatestReleasePath must never name a build older than LastCompiledVersion once RequestedReleaseAt has been consumed. It is answerable from the node alone — a release version is {yyyyMMddHHmmss}-{hash} where the hash is minted from the build's durable store coordinates, exactly what LatestAssembly{Collection,Path} carry. - `ReleaseVersionHash` — the mint and the check are now ONE function, so they cannot drift into disagreeing about what a release version means. - `ReleaseNamesBuild` — the cheap comparison, pure. Inconclusive answers TRUE in three deliberate cases: no release at all (absence is not staleness — an ADOPTED build has never had one, and calling that stale would put a compile back on every install), no durable coordinates (the Null-store path), and a version shape this mint did not produce. - `IsSatisfiedByCurrentBuild` = `BuildInHandAnswersRequest` AND `ReleaseNamesBuild`. - `TryCutReleaseForBuildInHand` — when the bytes are right but the release is not, the release is cut FROM THOSE BYTES, no compile, under the same System execution scope the compile's own release create uses. A cut that does not land leaves the clause false, so the request falls through to the compile path and gets its release there; it is never consumed for nothing. - The bound now DELIVERS its diagnostic (ERROR naming the type and the attempted release) — the same rule ReadBudget states for nested read bounds. Red-then-green: `ReleaseNamesTheBuildTest` fails on the unfixed tree with `ARequestIsNotSatisfiedWhileTheReleaseNamesOtherBytes ... found True` while its nine sibling cases pass, including the two that pin the #1707 non-regression. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Test Results (shard 0)216 tests 216 ✅ 2m 27s ⏱️ Results for commit 0d21a51. |
There was a problem hiding this comment.
🟡 Changes recommended
It introduces an Observable.Using(access.ImpersonateAsSystem, ...) identity scope in TryCutReleaseForBuildInHand, which is a known AsyncLocal-latch hazard and conflicts with established RunAsSystem sealing patterns.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR fixes a correctness gap in the NodeType release-request watcher: a release request could be marked “handled” even when no release existed for the current build, leaving LatestReleasePath pointing at an older build under a green CompilationStatus.Ok. It restores the invariant that once a release request is consumed, the node must have (or cut) a release that names the bytes the request asked to publish.
Changes:
- Adds a pure “release names these bytes” check (
ReleaseVersionHash+ReleaseNamesBuild) and uses it to tightenIsSatisfiedByCurrentBuild. - When the build bytes are already current but the release is stale, cuts the missing release from the build-in-hand (no recompile) before consuming the request.
- Adds a regression test suite for the #781 shape and documents the incident/fix in What’s New + the release redesign postmortem.
File summaries
| File | Description |
|---|---|
| test/MeshWeaver.Compiler.Pipeline.Test/ReleaseNamesTheBuildTest.cs | New regression tests that pin the “release must name the build” invariant and the “build-in-hand” satisfaction rules. |
| src/MeshWeaver.Compiler.Pipeline/NodeTypeCompilationHelpers.cs | Updates the release request watcher to pre-cut a release from the current build when needed and only satisfy/consume requests when the release names the build. |
| src/MeshWeaver.Compiler.Pipeline/NodeTypeBuildState.cs | Centralizes release hash minting/checking and adds TryCutReleaseForBuildInHand; improves timeout diagnostics for release creation. |
| src/MeshWeaver.Documentation/Data/WhatsNew/2026-09-01-a-release-now-names-the-build-you-asked-it-to-publish.md | User-facing What’s New entry describing the silent-stale-release defect and the behavioral fix. |
| src/MeshWeaver.Documentation/Data/Architecture/Postmortems/NodeTypeReleaseRedesign.md | Postmortem addendum documenting the measured failure mode, invariant, and design rule left behind. |
Review details
Suppressed comments (2)
src/MeshWeaver.Compiler.Pipeline/NodeTypeBuildState.cs:212
- This failure log message may now come from a non-compile release cut (TryCutReleaseForBuildInHand), so wording that asserts a compile succeeded / that the previous build is still referenced can be inaccurate. Prefer a message that describes only the failed release create and the possible consequence (LatestReleasePath may remain stale).
logger?.LogError(ex,
"CompileWatcher: failed to create Release node at {ReleasePath} — the "
+ "compile SUCCEEDED but no release was cut, so LatestReleasePath still "
+ "names the PREVIOUS build", releasePath);
src/MeshWeaver.Compiler.Pipeline/NodeTypeBuildState.cs:220
- TryCreateReleaseNode can now be used for a release cut without a compile, so this log message shouldn’t assume a successful compile or that a previous release necessarily exists. Prefer wording that states the release cut failed and the type’s LatestReleasePath may remain stale.
logger?.LogError(ex,
"CompileWatcher: TryCreateReleaseNode threw for {NodeTypePath} — no release was "
+ "cut, so LatestReleasePath still names the PREVIOUS build", nodeTypePath);
- Files reviewed: 5/5 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| var accessService = hub.ServiceProvider.GetService<AccessService>(); | ||
| return Observable.Using( | ||
| () => accessService?.ImpersonateAsSystem() | ||
| ?? (IDisposable)System.Reactive.Disposables.Disposable.Empty, | ||
| _ => TryCreateReleaseNode( | ||
| hub, | ||
| nodeTypePath, | ||
| new NodeCompilationResult( | ||
| AssemblyLocation: null, | ||
| NodeTypeConfigurations: [], | ||
| CompiledSources: def.CompiledSources as ImmutableDictionary<string, long> | ||
| ?? def.CompiledSources?.ToImmutableDictionary(), | ||
| Collection: def.LatestAssemblyCollection, | ||
| ContentPath: def.LatestAssemblyPath, | ||
| Version: def.LastCompiledVersion), | ||
| node, | ||
| activityPath: def.LastCompilationActivityPath, | ||
| logger)); |
| logger?.LogError( | ||
| "CompileWatcher: the Release node create for {ReleasePath} did not land " | ||
| + "within {Bound}s — the compile SUCCEEDED but no release was cut, so " | ||
| + "LatestReleasePath still names the PREVIOUS build", | ||
| releasePath, ReleaseCreateBound.TotalSeconds); |
Test Results (shard 2)531 tests 528 ✅ 1m 30s ⏱️ For more details on these failures, see this check. Results for commit 0d21a51. |
Test Results (shard 3)586 tests 586 ✅ 50s ⏱️ Results for commit 0d21a51. |
Test Results (shard 5) 3 files 3 suites 1m 17s ⏱️ Results for commit 0d21a51. |
Test Results (shard 1)419 tests 419 ✅ 1m 28s ⏱️ Results for commit 0d21a51. |
Test Results (shard 4)1 703 tests 1 703 ✅ 1m 50s ⏱️ Results for commit 0d21a51. |
Test Results 14 files 14 suites 9m 24s ⏱️ For more details on these failures, see this check. Results for commit 0d21a51. |
|
Closing as superseded by its own sibling branch, which landed on main while this sat DIRTY. Evidence first, because "superseded" is a claim that deserves proof rather than a judgement call. What is already on main
Why this branch should not be merged on topMerging would add a second, parallel mechanism for one invariant. Two remedies racing to cut a release for the same build is worse than either alone — that is the dual-writer shape that produces split-brain state, and it is the class of defect this very postmortem exists to document. It would also reintroduce a pattern main just removed. Copilot's review here asked this branch to replace The second review finding — the timeout log wording claiming "the compile SUCCEEDED" / "PREVIOUS build" on a path where there may have been neither — was also correct, and is moot once the code carrying it is not merged. What was genuinely different here, and why it is not lostThis branch's distinctive piece was the cheap pure comparison ( The postmortem is not lost either: HousekeepingThis PR was DIRTY, and a DIRTY PR runs no CI at all — so its last red ( Reopen if I have misread the overlap — the branch is intact. |
Pull request was closed
Fixes Systemorph/MeshWeaver.Plugins#781.
The mechanism
LastReleaseRequestHandledAtis stamped at dispatch, on the same commit that flips toPending. That is correct as a re-fire guard and wrong as a delivery receipt — and the field was doing both jobs. Between the dispatch and aReleasenode existing there are three ways to end up with nothing, and all three were silent and indistinguishable from "no release was asked for":TryCreateReleaseNode's bound expired — a bareTimeout(_, Observable.Return(null))with no log line at all.In every case
ApplyCompileSuccessstampsLatestReleasePath = releasePath ?? def.LatestReleasePath— the PREVIOUS build's release — and the trigger is spent.Measured on
Publish/Deck, 2026-08-27:compilationStatus: Ok,compiledSourcesidentical tocurrentSourceVersions,lastCompiledVersion 575,latestAssemblyPathnaming v575's bytes — andlatestReleasePathnaming a release cut the previous morning. Every single-field read is healthy; only holding the release up against the build shows it. So every consumer followingLatestReleasePath(the build protocol, a release-pinned activation, the Configuration pane's "→ release" link) served the previous assembly under a greenOk.And the state was unrepairable from the outside. A second, ordinary release request hit case 3 — the bytes ARE current — so it was consumed again without producing anything. Only
RequestedReleaseForceescaped, and only if someone knew to look.The invariant restored
Answerable from the node alone, no store probe: a release version is
{yyyyMMddHHmmss}-{hash}where the hash is minted from the build's DURABLE store coordinates — exactly whatLatestAssemblyCollection/LatestAssemblyPathcarry.ReleaseVersionHash— the mint and the check are now ONE function, so they cannot drift into disagreeing about what a release version means.ReleaseNamesBuild— the cheap comparison, pure. Inconclusive answers TRUE, three deliberate cases: no release at all (absence is not staleness — an ADOPTED build has never had one, and calling that stale would put a compile back on every install), no durable coordinates (the Null-store path mints from a process-local location), and a version shape this mint did not produce.IsSatisfiedByCurrentBuild=BuildInHandAnswersRequestandReleaseNamesBuild.TryCutReleaseForBuildInHand— when the bytes are right but the release is not, the release is cut from those bytes, no compile, under the same System execution scope the compile's own release create uses. A cut that does not land leaves the clause false, so the request falls through to the compile path and gets its release from the compile's terminal create; it is never consumed for nothing.ReadBudgetstates for nested read bounds: the inner bound is the only one that knows WHICH write starved.No window was widened and no timeout was raised.
ReleaseCreateBoundis the same 10s, now named and documented as a bound nested inside the hub's request timeout.Failing-then-passing evidence
ReleaseNamesTheBuildTeston the unfixed tree:The nine siblings pass on the unfixed tree — including
AnAdoptedBuildWithNoReleaseIsStillSatisfiedandNoReleaseAtAll_IsInconclusive_NotStale, which are what stop this fix from quietly putting the #1707 compile back on the install path. After the fix:Passed! - Failed: 0, Passed: 17(withRecompileStormAbsorptionTest+ForcedReleaseIsSpentTest), and593/593for the wholeMeshWeaver.Compiler.Pipeline.Testproject.The real-mesh release suites moved to
MeshWeaver.Plugins, so nothing in this repo's CI drives the watcher end to end. They were run locally against this branch via-p:MeshWeaverRoot=<this worktree>:NodeTypeReleaseGateTest,ReleaseRequestSatisfiedByCurrentBuildTest,ReleaseRequestWatcherHighWaterTest,ReleaseArtifactLinkTest,BuildCoordinationTest,CodeEditRecompileTest,CompileSourceSnapshotWedgeTest— 35/35 passed.Same family as #2992?
Partly, and worth saying precisely. #2992 was two nested bounds that were EQUAL, so the inner one could never fire first. Here the inner bound does fire first — and then throws its diagnostic away, degrading to a
nullthe caller cannot tell from "nothing was asked for". Same underlying rule fromReadBudget.cs(a nested bound is the only one that knows WHICH read starved), failing at the delivery step rather than the ordering step. Two instances of one rule in two weeks is worth a sweep forTimeout(_, Observable.Return(<empty>))across the pipeline.Not reproduced
The 2026-08-27 dump cannot say WHICH of the three swallow paths ran that night (
requestedReleaseForce: truewas still standing, which today'sApplyCompileSuccesswould have cleared, so the node predates several fields). All three are closed here, and the third one — the only one that was permanent — now has a regression test.