Skip to content

fix(graph): eliminate the compile-flake population (single compile driver + complete terminal write + search ordering) - #173

Merged
rbuergi merged 1 commit into
mainfrom
fix/content-faithful-base-full
Jul 2, 2026
Merged

rbuergi merged 1 commit into
mainfrom
fix/content-faithful-base-full

Conversation

@rbuergi

@rbuergi rbuergi commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Root-causes the 2-core CI flake family that's been reddening branch runs (MeshNodeCompilation, OrleansDynamicCompilation, NodeTypeReleaseGate, FrameworkStale, SearchQueryTests). NOT the remembered wall-clock/base-Full model (that mechanism no longer exists — memory corrected); reproduced and pinned the real owner-side races. 5/5 full-suite iterations green @2 cores; Orleans 122/122, Query 342/342, Data 273/273, solution Release -warnaserror clean.

🤖 Generated with Claude Code

… + complete terminal write + confirmed release path

Reproduced at 2 cores (not the remembered wall-clock base-Full model — that mechanism is
gone). Actual roots, all owner-side:
- NodeTypeContractHandler ran Roslyn INLINE for a fresh NodeType (status==null treated as
  settled) concurrently with the watcher's first-build → loser reads the winner's mid-emit
  DLL → BadImageFormatException → deletes it → terminal Error forever. Now a single
  status-guarded dispatch (EnsureCompileDispatched); the watcher is the only driver.
- Success write-back omitted CompiledFrameworkVersion → HasUsableBuild false forever →
  every fv-gated waiter times out. Now stamped on fresh compile.
- TryCreateReleaseNode was fire-and-forget yet its path was stamped into LatestReleasePath
  immediately → phantom-path NotFound. Now emits the path only after the create lands.
- SEPARATE root: storage query discarded its fuzzy score → free-text top-N rode readdir
  order (CI-only SearchQueryTests flake). Now ranked before the Take(limit) clip.

5/5 full Monolith-suite iterations green @2 cores; Orleans 122/122; Query 342/342.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR targets a set of CI flakes and occasional runtime wedges in the graph/node-type compilation pipeline by enforcing a single compile driver (status control plane → watcher-driven compile), ensuring terminal compile writes are complete, and making query results deterministic under free-text search.

Changes:

  • Added an integration test that ratchets the “single compile driver + complete terminal write” invariants for fresh NodeTypes, including concurrent requester consistency.
  • Updated GetCompilationPathRequest handling to dispatch compiles via the status control plane, wait for settling, and ensure CompiledFrameworkVersion is stamped on fresh-success write-back.
  • Stabilized in-memory free-text query results by ranking matches by relevance before applying the limit clip; and changed release-node creation to be observed/bounded (but see review comment on AccessContext).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
test/MeshWeaver.Hosting.Monolith.Test/CompileSingleDriverConsistencyTest.cs New integration test covering fresh compile terminal-state completeness and concurrent request single-flight behavior.
src/MeshWeaver.Hosting/Persistence/Query/StorageAdapterMeshQueryProvider.cs Adds relevance sorting for free-text search to eliminate limit-window nondeterminism.
src/MeshWeaver.Graph/MeshDataSource.cs Changes release creation helper to an observed/bounded IObservable<string?> surface rather than fire-and-forget.
src/MeshWeaver.Graph/Configuration/NodeTypeContractHandler.cs Enforces watcher-driven single-flight compilation and stamps framework version on fresh compile success write-back.
src/MeshWeaver.Graph/Configuration/NodeTypeCompileActivityHandler.cs Ensures parent terminal writes occur only after release-create observation completes (bounded), avoiding phantom release paths.
src/MeshWeaver.Graph/Configuration/NodeTypeCompilationHelpers.cs Mirrors bounded/observed release-create behavior inside the watcher compile pipeline.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +564 to +572
return Observable.Using(
() => !string.IsNullOrEmpty(requestedBy) && accessService is not null
? accessService.SwitchAccessContext(new AccessContext
{
ObjectId = requestedBy,
Name = requestedBy
})
: System.Reactive.Disposables.Disposable.Empty,
_ => meshService.CreateNode(node).Take(1))
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 2)

   14 files  ±  0     14 suites  ±0   6m 46s ⏱️ -31s
1 137 tests  - 164  1 134 ✅  - 164  3 💤 ±0  0 ❌ ±0 
1 138 runs   - 192  1 135 ✅  - 192  3 💤 ±0  0 ❌ ±0 

Results for commit 563afa3. ± Comparison against base commit 214ba1b.

This pull request removes 164 tests.
MeshWeaver.Persistence.Test.ApplicationPageResolutionTest ‑ ResolvePathAsync_FutuReSubPaths_ShouldResolve(path: "FutuRe/Analysis", expectedPrefix: "FutuRe/Analysis", expectedRemainder: null)
MeshWeaver.Persistence.Test.ApplicationPageResolutionTest ‑ ResolvePathAsync_FutuReSubPaths_ShouldResolve(path: "FutuRe/EuropeRe", expectedPrefix: "FutuRe/EuropeRe", expectedRemainder: null)
MeshWeaver.Persistence.Test.ApplicationPageResolutionTest ‑ ResolvePathAsync_FutuReSubPaths_ShouldResolve(path: "FutuRe/EuropeRe/LineOfBusiness", expectedPrefix: "FutuRe/EuropeRe/LineOfBusiness", expectedRemainder: null)
MeshWeaver.Persistence.Test.ApplicationPageResolutionTest ‑ ResolvePathAsync_FutuReSubPaths_ShouldResolve(path: "FutuRe/EuropeRe/Overview", expectedPrefix: "FutuRe/EuropeRe", expectedRemainder: "Overview")
MeshWeaver.Persistence.Test.ApplicationPageResolutionTest ‑ ResolvePathAsync_FutuRe_ShouldNotReturnNull
MeshWeaver.Persistence.Test.ApplicationPageResolutionTest ‑ ResolvePathAsync_UnknownDeepPath_ShouldReturnNull
MeshWeaver.Persistence.Test.ConcurrentRequestsTest ‑ ConcurrentRequests_MultipleNodeTypes_AllLoadWithoutHanging
MeshWeaver.Persistence.Test.DocumentationCodeBlockCompilationTest ‑ ExecutedCsharpBlocks_MustCompile(embeddedResourceName: "MeshWeaver.Documentation.Data.AI.AgenticAI.md")
MeshWeaver.Persistence.Test.DocumentationCodeBlockCompilationTest ‑ ExecutedCsharpBlocks_MustCompile(embeddedResourceName: "MeshWeaver.Documentation.Data.AI.ChatCommands.md")
MeshWeaver.Persistence.Test.DocumentationCodeBlockCompilationTest ‑ ExecutedCsharpBlocks_MustCompile(embeddedResourceName: "MeshWeaver.Documentation.Data.AI.ExecuteScript.md")
…

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 3)

781 tests  ±0   674 ✅ ±0   2m 13s ⏱️ -22s
 12 suites ±0   107 💤 ±0 
 12 files   ±0     0 ❌ ±0 

Results for commit 563afa3. ± Comparison against base commit 214ba1b.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 1)

1 160 tests   - 100   1 159 ✅  - 100   3m 57s ⏱️ - 1m 8s
   14 suites ±  0       1 💤 ±  0 
   14 files   ±  0       0 ❌ ±  0 

Results for commit 563afa3. ± Comparison against base commit 214ba1b.

This pull request removes 102 and adds 2 tests. Note that renamed tests count towards both.
MeshWeaver.Hosting.Monolith.Test.AgentSyncedQueryFromHostedHubTest ‑ HostedSubHub_GetQuery_AgentDropdownNamesAreNonEmpty
MeshWeaver.Hosting.Monolith.Test.AgentSyncedQueryFromHostedHubTest ‑ HostedSubHub_GetQuery_ReturnsAgentsAndModels
MeshWeaver.Hosting.Monolith.Test.BrokenNodeTypeAccessTest ‑ AccessingInstance_OfNonCompilingNodeType_AnswersTerminalError_NotSilence
MeshWeaver.Hosting.Monolith.Test.CessionLayoutAreaTest ‑ BusinessRulesDoc_RelativeReference_ResolvesToMotorXL
MeshWeaver.Hosting.Monolith.Test.CessionLayoutAreaTest ‑ Cession_Trace_HubConfiguration
MeshWeaver.Hosting.Monolith.Test.CessionLayoutAreaTest ‑ MotorXL_LayoutArea_ReturnsContent
MeshWeaver.Hosting.Monolith.Test.CessionLayoutAreaTest ‑ MotorXL_Overview_ShouldRender
MeshWeaver.Hosting.Monolith.Test.CessionLayoutAreaTest ‑ MotorXL_PathResolves
MeshWeaver.Hosting.Monolith.Test.CessionLayoutAreaTest ‑ NonExistentPath_Failure
MeshWeaver.Hosting.Monolith.Test.CodeEditRecompileTest ‑ CodeEdit_ExplicitRelease_IsUpToDate_RecompilesOnSourceChange
…
MeshWeaver.Hosting.Monolith.Test.CompileSingleDriverConsistencyTest ‑ ConcurrentRequests_OnFreshNodeType_BothSucceedConsistently
MeshWeaver.Hosting.Monolith.Test.CompileSingleDriverConsistencyTest ‑ FreshCompile_TerminalStateIsComplete

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 0)

1 587 tests  ±0   1 587 ✅ ±0   5m 22s ⏱️ -17s
   13 suites ±0       0 💤 ±0 
   13 files   ±0       0 ❌ ±0 

Results for commit 563afa3. ± Comparison against base commit 214ba1b.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Test Results

   53 files  ±  0     53 suites  ±0   18m 19s ⏱️ - 2m 18s
4 665 tests  - 264  4 554 ✅  - 264  111 💤 ±0  0 ❌ ±0 
4 666 runs   - 292  4 555 ✅  - 292  111 💤 ±0  0 ❌ ±0 

Results for commit 563afa3. ± Comparison against base commit 214ba1b.

This pull request removes 266 and adds 2 tests. Note that renamed tests count towards both.
MeshWeaver.Hosting.Monolith.Test.AgentSyncedQueryFromHostedHubTest ‑ HostedSubHub_GetQuery_AgentDropdownNamesAreNonEmpty
MeshWeaver.Hosting.Monolith.Test.AgentSyncedQueryFromHostedHubTest ‑ HostedSubHub_GetQuery_ReturnsAgentsAndModels
MeshWeaver.Hosting.Monolith.Test.BrokenNodeTypeAccessTest ‑ AccessingInstance_OfNonCompilingNodeType_AnswersTerminalError_NotSilence
MeshWeaver.Hosting.Monolith.Test.CessionLayoutAreaTest ‑ BusinessRulesDoc_RelativeReference_ResolvesToMotorXL
MeshWeaver.Hosting.Monolith.Test.CessionLayoutAreaTest ‑ Cession_Trace_HubConfiguration
MeshWeaver.Hosting.Monolith.Test.CessionLayoutAreaTest ‑ MotorXL_LayoutArea_ReturnsContent
MeshWeaver.Hosting.Monolith.Test.CessionLayoutAreaTest ‑ MotorXL_Overview_ShouldRender
MeshWeaver.Hosting.Monolith.Test.CessionLayoutAreaTest ‑ MotorXL_PathResolves
MeshWeaver.Hosting.Monolith.Test.CessionLayoutAreaTest ‑ NonExistentPath_Failure
MeshWeaver.Hosting.Monolith.Test.CodeEditRecompileTest ‑ CodeEdit_ExplicitRelease_IsUpToDate_RecompilesOnSourceChange
…
MeshWeaver.Hosting.Monolith.Test.CompileSingleDriverConsistencyTest ‑ ConcurrentRequests_OnFreshNodeType_BothSucceedConsistently
MeshWeaver.Hosting.Monolith.Test.CompileSingleDriverConsistencyTest ‑ FreshCompile_TerminalStateIsComplete

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants