Batch bake is opt-in: its source discovery read 50 of thousands of Code nodes in production by rbuergi · Pull Request #1217 · Systemorph/MeshWeaver

rbuergi · 2026-08-11T12:07:55Z

What happened

First production run of #1207/#1211 (memex-cloud, image ci.2947, gated pod):

DynamicTypePreWarmer: warm-up complete in 00:00:19.36 —
  compiled=4 compileErrors=59 timedOut=0 skipped=5 contentBroken=169
REFUSING READINESS — 64 NodeType(s) regressed on this image

Loki carried the cause in the sweep's own diagnostic:

BatchBake: source discovery resolved 50 Code node(s) into source sets for 237 pending type(s) in one pass

50 is the default query limit. ResolveSources issues one global nodeType:Code fetch through MeshQueryRequest.FromQuery, which leaves Limit null, so on a mesh with thousands of Code nodes it returns the first 50. 169 types resolved an empty source set (NoSources), 59 a partial one (CompileError) — all of them types that compile fine on the activation path. The tests pass because a test mesh has fewer than 50 Code nodes, so the cap never binds.

Why this PR flips the default rather than only fixing the limit

The mode defaulted ON for any readiness-gated pod, so every gated portal that self-updates stalls its own rollout the same way. The live PreWarm__BatchBake=false patches on memex-cloud / memex / client-i are a stopgap that a helm upgrade would revert.
On an ungated pod it is worse than a stall: a wrongly-empty source set classifies as PreWarmStatus.NoSources, a content verdict that deliberately does not gate — so the same bug would bake a fleet of empty assemblies with nothing refusing readiness.

Correctness before speed: batch bake now has to be asked for explicitly, and the code says why. The measured win that did hold — #1206's zero pacing on gated bakes — is untouched.

Follow-up (#1216)

Unbounded/paginated discovery, verified through the Postgres provider rather than assumed; a scale test with more Code nodes than the default limit; and — the deeper fix — discovery asserting its own invariant, so a type with a non-empty Sources list that resolves to nothing fails the BATCH (falling back to the activation path) instead of being reported as content-broken, which currently makes a discovery bug indistinguishable from genuinely deleted content.

What's New entry included (Category: Fix).

🤖 Generated with Claude Code

…s in prod First production run (memex-cloud, ci.2947) finished in 19s with compiled=4 compileErrors=59 contentBroken=169 of 237. Loki has the cause in the sweep's own line: BatchBake: source discovery resolved 50 Code node(s) into source sets for 237 pending type(s) in one pass 50 is the default query limit. ResolveSources issues ONE global nodeType:Code fetch via MeshQueryRequest.FromQuery, which leaves Limit null, so a mesh with thousands of Code nodes yields the first 50: 169 types resolved an EMPTY source set and 59 a PARTIAL one. Tests never saw it — a test mesh has fewer than 50 Code nodes, so the cap never binds. The gate caught it (readiness refused, previous image kept serving), but the mode defaulted ON for every readiness-gated pod, so each gated portal that self-updated would stall its own rollout; live PreWarm__BatchBake=false patches on memex-cloud/memex/atioz are a stopgap a helm upgrade would revert. On an UNGATED pod it is worse: an empty source set classifies as NoSources, which by design does not gate, so the same bug would bake empty assemblies silently. Default is now false everywhere; the key must be set explicitly. The pacing differentiation (#1206) is untouched and keeps its measured win. Discovery fix + scale test tracked in #1216. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot

Pull request overview

This PR makes the new “batch bake” (linked batch build) warm-up mode explicitly opt-in after a production-scale source discovery truncation caused many dynamic NodeTypes to be classified as content-broken. It updates the hosted service’s configuration behavior/documentation and adds a “What’s New” entry describing the change.

Changes:

Change PreWarm:BatchBake behavior to default OFF everywhere (only enabled when explicitly configured).
Update in-code documentation to clearly call out the production-scale risk and reference the follow-up work.
Add a “What’s New” entry explaining why updates now use the activation-driven preparation path by default.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
src/MeshWeaver.Hosting/DynamicTypePreWarmerHostedService.cs	Makes batch bake opt-in by default and documents the production-scale failure mode and rationale.
src/MeshWeaver.Documentation/Data/WhatsNew/2026-08-11-batch-bake-opt-in.md	Adds release-note entry explaining the opt-in flip and impact on platform updates.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

github-actions · 2026-08-11T12:23:00Z