Repository navigation
fix(compiler): bound compile CPU on a dedicated-thread IIoPool lane - #5375
Conversation
Follow-up to #5327, which moved the NodeType emit off the ThreadPool onto CompileThread: off the pool, but one new thread per DISTINCT NodeType compiling at once — hundreds on a roll. And the language service's diagnostics binds still ran as Compile-pool leaves on ThreadPool workers (the script workspace with Roslyn's default ConcurrentBuild on top). - IoPool gains dedicatedThreads: InvokeBlocking leaves drain on threads the pool's LimitedConcurrencyLevelTaskScheduler starts itself (mw-cpu-lane), same queue and cap, never a ThreadPool worker. - IoPoolNames.CompileCpu (cap IoPoolOptions.CompileCpu = processor count), built by the registry with dedicatedThreads. Separate from Compile, which also runs kernel scripts that may wait on a compile — sharing its cap is the nested-gate deadlock. A lane leaf only computes, so it cannot nest. - EmitCompiled and the failure-path DiagnoseInputs run on the lane. - Every language-service diagnostics bind (workspace, speculative, script) splits into an IO half on the Compile pool and a bind on the lane; SpeculativeCompilation exposes CreateCompilationAsync + Diagnose. The script workspace binds with ConcurrentBuild off. - Tests with negative controls that fail on the unfixed routing: CompileCpuLaneIsBoundedAndOffThePoolTest, LanguageServiceDiagnosticsRunOn TheCpuLaneTest, and ServiceCompileStaysOffTheThreadPoolTest (now asserts the lane thread). - Docs: CompileOffTheThreadPool (the lane), ControlledIoPooling (pool row). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The critical scheduler failure and unresolved cancellation, async-boundary, and exception-path issues block approval.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
Open (5)
What changed in this PR
This PR bounds Roslyn compilation and diagnostic CPU work on a dedicated, limited-concurrency CompileCpu lane.
Changes:
- Adds dedicated-thread
IoPoolscheduling andCompileCpuconfiguration. - Routes emits and diagnostic binding through the CPU lane.
- Adds regression tests and updates architecture documentation.
| File | Summary |
|---|---|
test/MeshWeaver.Hosting.Test/CompileCpuLaneIsBoundedAndOffThePoolTest.cs |
Tests lane concurrency limits and dedicated-thread execution. |
test/MeshWeaver.Compiler.Pipeline.Test/ServiceCompileStaysOffTheThreadPoolTest.cs |
Verifies emits run on the CPU lane. |
test/MeshWeaver.Compiler.Pipeline.Test/LanguageServiceDiagnosticsRunOnTheCpuLaneTest.cs |
Verifies diagnostic binding runs on lane threads. |
src/MeshWeaver.Mesh.Contract/Threading/LimitedConcurrencyLevelTaskScheduler.cs |
Adds dedicated-thread scheduling. Critical: startup failure can leave the scheduler wedged. |
src/MeshWeaver.Mesh.Contract/Threading/IoPoolRegistry.cs |
Configures the CompileCpu pool. |
src/MeshWeaver.Mesh.Contract/Threading/IoPoolOptions.cs |
Defines lane configuration. Nit: the type summary contradicts the dedicated-thread exception. |
src/MeshWeaver.Mesh.Contract/Threading/IoPool.cs |
Adds dedicated-thread pool construction. |
src/MeshWeaver.Documentation/Data/Architecture/ControlledIoPooling.md |
Documents the new pool. Nit: existing no-custom-threads wording is now contradictory. |
src/MeshWeaver.Documentation/Data/Architecture/CompileOffTheThreadPool.md |
Documents the CPU-lane design. |
src/MeshWeaver.Compiler/SpeculativeCompilation.cs |
Separates compilation creation from diagnosis. Moderate: the added async boundary may lose scheduler context. |
src/MeshWeaver.Compiler.Pipeline/MeshNodeLanguageService.cs |
Splits diagnostics IO and CPU binding. Moderate: Task-returning helpers introduce unsafe async boundaries. |
src/MeshWeaver.Compiler.Pipeline/MeshNodeCompilationService.cs |
Moves emits and failure diagnostics to the lane. Moderate: cancellation is discarded, an async bridge remains, and the exception path synchronously uses another pool. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| new Thread(_ => DrainQueue(), maxStackSize: 0) | ||
| { | ||
| IsBackground = true, | ||
| Name = DedicatedThreadName, | ||
| }.UnsafeStart(); |
There was a problem hiding this comment.
Agreed, fixed in 468c6ac. The reservation is now taken inside lock(_tasks) together with the queued node. If NotifyThreadPoolOfPendingWork() throws (thread creation or UnsafeStart failed), the catch decrements _delegatesQueuedOrRunning, removes that node from _tasks and rethrows. TaskFactory.StartNew in InvokeBlocking then faults the leaf, so the scheduler never sits at its cap with no drain loop behind it.
| ? Observable.Return<IReadOnlyList<Lsp.DiagnosticInfo>>(Array.Empty<Lsp.DiagnosticInfo>()) | ||
| : OnThreadPool(() => CompileDiagnostics.DiagnoseInputs(inputs))) | ||
| // Pure Roslyn compute -> the bounded CPU lane, off the ThreadPool (CompileOffTheThreadPool). | ||
| : _cpuLane.InvokeBlocking(_ => CompileDiagnostics.DiagnoseInputs(inputs))) |
There was a problem hiding this comment.
Agreed, fixed in 468c6ac. CompileDiagnostics.DiagnoseInputs now takes a CancellationToken and passes it to RunSourceGenerators and GetDiagnostics(ct). The service hands it the lane token: _cpuLane.InvokeBlocking(ct => CompileDiagnostics.DiagnoseInputs(inputs, ct)). An abandoned failure diagnose therefore releases its CompileCpu slot when it is cancelled. The emit leaf already linked the lane token with the compile's own bound.
| // queue. EmitCompiled does nothing but compute and write its own artifact — no pool call, no | ||
| // mesh read — so it can never wait on another lane leaf (the nested-gate deadlock the Compile | ||
| // pool hit). Doc/Architecture/CompileOffTheThreadPool. | ||
| return await _cpuLane |
There was a problem hiding this comment.
This await is not new. CompileAsyncCore was already an async Task compile leaf (NuGet restore, debug source write). #5327 awaited CompileThread.Run at this exact spot, and this PR swaps that for the lane.
- It never parks a turn scheduler. The leaf is entered only through
OnThreadPool(Func<Task<T>>)(Task.Run), never on a hub or grain thread. - Identity is preserved.
OnThreadPoolre-establishesImpersonateAsSysteminside the async lambda for its whole span, andEmitCompileddoes no mesh reads. - The wait uses the sanctioned bridge. It goes through
ObservableAwait.Await(ct)(RunContinuationsAsynchronously), notToTask/await observable.
Turning the whole compile leaf reactive means replacing the CompileAsync/_inflightCompiles Task shape. That refactor is unrelated to bounding CPU, so I've kept it out of this PR. No change here.
| }); | ||
|
|
||
| /// <summary>The IO half of a node workspace's diagnostics: the project's compilation (not yet bound).</summary> | ||
| private static async Task<Compilation?> WorkspaceCompilationAsync(CachedWorkspace cached, CancellationToken ct) |
There was a problem hiding this comment.
WorkspaceCompilationAsync / ScriptCompilationAsync are the former GetDiagnosticsAsync / GetScriptDiagnosticsAsync bodies with the bind moved out; they are not new async boundaries.
- Both are
privateand are reached only as_ioPool.Run(ct => …)leaves, so the IO pool already owns the async leaf. - Every other private leaf in this service (
GetHoverAsync,GetCompletionsAsync, …) has the same shape. - The async part is Roslyn's
Project.GetCompilationAsync, which has no synchronous form.
No change.
| string sourcePath, | ||
| string proposedCode, | ||
| CancellationToken ct) | ||
| => Diagnose(await CreateCompilationAsync(inputs, sourcePath, proposedCode, ct).ConfigureAwait(false), ct); |
There was a problem hiding this comment.
SpeculativeCompilation.GetDiagnosticsAsync was already a public async Task helper before this PR. Its callers use it as an IIoPool.Run leaf. This PR only splits it into CreateCompilationAsync (the IO half: NuGet resolve, parse) and a synchronous Diagnose, and keeps GetDiagnosticsAsync as their composition so existing callers are unchanged.
The language service now does what you suggest: _ioPool.Run(ct => CreateCompilationAsync(...)).SelectMany(c => _cpuLane.InvokeBlocking(ct => Diagnose(c, ct))). The async IO leaf sits behind IIoPool, and the result is composed into the lane bind rather than awaited. No change.
…art returns its slot Review of #5375: DiagnoseInputs ran generators and GetDiagnostics with CancellationToken.None, so an abandoned failure diagnose held a CompileCpu slot to the end; it now takes the lane token. And a dedicated-thread drain whose Thread.Start throws now rolls back its reservation and withdraws the task instead of leaving the scheduler at its cap with no drain loop. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
What
This follow-up to #5327 closes the two gaps that PR left open.
CompileThread. That is off the ThreadPool, but it starts one new OS thread per distinct NodeType compiling at once, and a roll produces hundreds of those._ioPool.Runleaves on theCompilepool. The script workspace also bound with Roslyn's defaultConcurrentBuild.Change: one bounded CPU lane over dedicated threads
IoPooldedicated-thread mode. A new constructor overload takesdedicatedThreads. With it on,InvokeBlockingleaves run on threads thatLimitedConcurrencyLevelTaskSchedulerstarts itself (namedmw-cpu-lane) instead of ThreadPool work items. The queue and the cap are the same as before. There is no new primitive and noSemaphoreSlim.IoPoolNames.CompileCpu. Its cap isIoPoolOptions.CompileCpu, which defaults to the processor count. The registry builds this pool with dedicated threads.Compile.Compilealso runs kernel scripts, which may wait on a compile. If an emit queued behind that same cap, a script holding a slot could wait forever on a compile that needs one: the nested-gate deadlock the compile leaf already hit once.EmitCompiled(generators, bind, emit, artifact write);CompileDiagnostics.DiagnoseInputs;GetCompilationAsync) that stays on theCompilepool, and a bind that runs on the lane.SpeculativeCompilationgainsCreateCompilationAsyncandDiagnosefor this;GetDiagnosticsAsyncstays as the composition of the two.ConcurrentBuildoff.Tests (each has a negative control that fails on the unfixed code)
CompileCpuLaneIsBoundedAndOffThePoolTest(Hosting.Test): six parked leaves on a lane capped at 2.Compilepool with the same cap runs all 6 on pool workers.LanguageServiceDiagnosticsRunOnTheCpuLaneTest: all three diagnostics arms bind onmw-cpu-lanethreads. It fails when the binds are routed back to theCompilepool.ServiceCompileStaysOffTheThreadPoolTest: now also asserts that the emit runs on the lane thread. It fails when the emit is routed through theCompilepool.Local runs:
-warnaserrorbuilds are clean for Mesh.Contract dependents (Hosting, Hosting.Orleans, Graph, Kernel.Hub, Compiler.Pipeline, Documentation).Still on the ThreadPool, by design (written up in the doc)
QuickInfoServiceandCompletionServiceresume withConfigureAwait(false), so they cannot be pinned to a thread.CompileResultFromAssembly. That is assembly load and user type initializers, not a CPU compile. It has its own timeout and stays onCompileThread.Docs
Doc/Architecture/CompileOffTheThreadPool: new section "The CPU lane".Doc/Architecture/ControlledIoPooling: newCompileCpurow in the pool table.Refs #5327, #5291, #5262, #5249, #5307, #5226
These issues need post-roll verification. None is closed here.
Pairs-with: none — nothing public removed. This PR only adds a public
IoPoolconstructor overload and anIoPoolNames/IoPoolOptionsmember.Implementers: none — no interface or abstract member is added.
Recycle after deploy: none. The lane is process-wide and takes effect at pod start.
🤖 Generated with Claude Code