perf: plugin gate install — children index for the in-memory store; SLOW_DISPATCH at a real 500 ms by rbuergi · Pull Request #4169 · Systemorph/MeshWeaver

rbuergi · 2026-09-13T09:25:33Z

Where the gate's time goes (measured 2026-09-13, Plugins run 34731952463, `Gate shard 1/1`, 26 min)

phase	time
pull + extract both images	1.4 min
bake (`compile /repo`, 1,100 nodes / 63 packages, no mesh)	1.3 min
install the 63 packages into the disposable mesh	18.3 min
compile + render + tests of 93 NodeTypes	3.6 min

A dotnet-trace (EventPipe sampling) of one package locally (Hosting, 198 nodes) puts 12.5 % of the samples in InMemoryStorageAdapter.ListChildPaths — a full scan of every key of the store per call, called once per directory level of every subtree query the installer issues per package: quadratic in the mesh, and the part that scaled 4× from a laptop to a 2-vCPU runner. Fix: a children index (directory → immediate child nodes + implied directories), maintained by every write path, keyed by the dictionary instance (shared stores share one index), rebuilt once if a test seeds the dictionary directly. Pinned by InMemoryStorageAdapterChildIndexTest; the 37 Hosting tests touching the adapter and subtree queries pass.

Second finding: SlowDispatchTicks = TimeSpan.TicksPerMillisecond * 500 (100-ns ticks) is compared against Stopwatch.GetTimestamp() deltas — on Linux that is 5 ms, so every dispatch over 5 ms logged MESSAGE_FLOW: SLOW_DISPATCH at Information: 366 lines per package install, 16,000 per shard (the gate raises MeshWeaver.Messaging.MessageHub to Information for QUIESCE attribution). Now Stopwatch.Frequency / 2 — the 500 ms the comment always described, on every platform. Production instances stop emitting that flood too.

Second commit — no ramp-up/down for foreign-typed roots. The installer did the placeholder → retype → root-recycle dance for EVERY non-static root; it is only needed when the root's NodeType is defined in the package itself. Every Store/Plugin satellite (nearly all of Plugins) now writes its root with its final type once and recycles nothing — which is where the ADVANCE_WITHOUT_HANDOFF bound=5000ms batches (13 s per package with NodeTypes) came from. Graph install/placeholder/retype tests (19) green.

Still next, measured: the sequential prebuilt-assembly adoption (1.5 s per type on CI). Note for readers of red gates meanwhile: on ubuntu-latest an UNSHARDED gate is cut at the 30-min cap by install time alone (Plugins#1764 run 34747207818); the 4-shard gate (#1771) and this PR are the fix, not the cap.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DqTDZe4Skkn81GuGkwMrXS

…emory store, and SLOW_DISPATCH at a real 500 ms Measured 2026-09-13 on Plugins run 34731952463 (Gate shard 1/1, 26 min): 1.3 min bake, 18.3 min INSTALLING 63 packages into the disposable mesh, 3.6 min compile+render+tests. A dotnet-trace of one package locally (Hosting, 198 nodes): 12.5 % of samples in InMemoryStorageAdapter.ListChildPaths — a full scan of every key per call, one call per directory level of every subtree query the installer issues per package, so quadratic in the mesh; and the log flood: SlowDispatchTicks was 100-ns ticks compared against Stopwatch ticks, i.e. 5 ms on Linux — 16,000 SLOW_DISPATCH lines per shard at Information (366 per package). The index maps a directory to its immediate children, keyed by the dictionary instance so adapters sharing one store share one index; pinned by InMemoryStorageAdapterChildIndexTest. The threshold is now Stopwatch.Frequency / 2. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DqTDZe4Skkn81GuGkwMrXS

…ped by ANOTHER package The dance exists for a root whose NodeType is defined in the package being installed; every non-static root took it. On the plugin gate that is every Store/Plugin satellite (Hosting, Signature, Essentials, …): one placeholder write, one retype, one hub recycle per package, and behind the recycle a batch of [UpdateQueue] ADVANCE_WITHOUT_HANDOFF bound=5000ms waits on the package's compile-state writes — 13 s per package with NodeTypes (Plugins run 34731952463). Maintainer, 2026-09-13: "must be done on running mesh not ramp up and down". Graph tests (19 install/placeholder/retype cases) green. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DqTDZe4Skkn81GuGkwMrXS

github-actions · 2026-09-13T09:37:01Z