Skip to content

feat: mw-plugin-test build — compile + test per package as a dependency cascade, no mesh import - #2784

Merged
rbuergi merged 7 commits into
mainfrom
feat/cascade-build
Aug 30, 2026
Merged

rbuergi merged 7 commits into
mainfrom
feat/cascade-build

Conversation

@rbuergi

@rbuergi rbuergi commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

The new build process for node repos

Maintainer direction (2026-08-30): build always means compile AND run tests; a build runner and test runner of our own where every package observes the streams of its dependencies and starts itself — a natural cascade; on red we break, on green we continue; the input is one or many plugins or all (full rebuild); dependencies resolved across the monorepo; Roslyn builds against the references the dependency packages produce; no mesh import — sources come from the git checkout on disk ("grains cannot handle compile workload"); and timings.

mw-plugin-test build <repo-root> [<package>... | all] [--module <dll>]... [--out <dir>] [--report <file>] [--max-parallel <n>] [--case-timeout <s>] [--no-tests]

What it does

  • Graph from disk: index.json → requires (PackageDependencyGraph.DependencyId), selection = named packages + transitive in-repo requirements; all = every package.
  • Cascade (Cascade.cs, generic, pure): each package's result stream is PublishLast-shared; a package CombineLatests its dependencies' streams, starts when the last is green, is Blocked(by: X) at once when one is not. Bounded parallelism; ready/started/finished per node; critical path.
  • Compile exactly as the portal does (NodeSetCompiler, the same skeleton/options, /app refs via CompileReferences.ComposeWithModules) plus the dependency packages' emitted assemblies as references.
  • Tests without a mesh: the Test/*.cs convention's static cases run from the emitted assembly in a collectible ALC, timed and capped. Host-taking cases are counted and named needs-mesh (the gate, seeded from --out, still runs them). Nothing is silently skipped.
  • Parity flag: a type whose emitted assembly binds a dependency package's assembly is marked binds-dependency-assembly — the portal reaches other packages by shared= source inclusion, so that green is on grounds the portal does not have. Visible, not hidden.
  • Report: a table (verdict, ready, queued, work, compile, tests, types, passed/failed/mesh) + critical path + parallel speed-up; --report writes JSON.

Tests

CascadeTest — 9 cases, executed: dependency observation and ordering, red-breaks with the block named, fault isolation, exactly-once execution across 20 dependents, bounded parallelism with real queue timings, cycle refusal, external dependencies, critical path, empty selection.

Not in this PR (named on purpose)

  • The Plugins lane YAML that pulls the pinned image and runs build /repo all (Plugins PR, after this image ships).
  • PG via Testcontainers wired into the runner's mesh for the substrate suites.
  • Running hosted cases as Activities.

🤖 Generated with Claude Code

…cy cascade, no mesh import

The new build process for node repos (maintainer, 2026-08-30). Build always means compile AND
run tests, per package, over the package dependency network: every package OBSERVES the result
streams of the packages it requires and starts itself the moment the last one completes green;
on red we break (every dependent is reported 'blocked by X' and never starts), on green we
continue. Independent packages build in parallel up to --max-parallel. Nothing is imported into
a mesh: sources are read from the checkout on disk (TreeNodeLoader), composed exactly as the
portal composes a NodeType compile (NodeSetCompiler: same skeleton, same options, the image's
/app reference set), and each package compiles against the assemblies its dependency packages
just emitted. Grains cannot carry a Roslyn workload; a build process can.

  mw-plugin-test build <repo-root> [<package>... | all] [--module <dll>]... [--out <dir>]
      [--report <file>] [--max-parallel <n>] [--case-timeout <s>] [--no-tests]

Selection = the named packages plus their transitive in-repo requirements; 'all' (default) is
the full rebuild the platform-rebuild case needs. Timings are part of the result: every package
carries ready/started/finished, the compile and test splits and per-type compile times; the
report prints them, the critical path and the parallel speed-up, and --report writes it as JSON.

Tests: the Test/*.cs convention (static classes, parameterless static methods that throw) runs
directly from the emitted assembly in a collectible load context, each case timed and capped.
A case that takes a host is COUNTED and NAMED as needs-mesh, never dropped — the gate, seeded
from --out, still owns those. A type whose emitted assembly binds a dependency package's
assembly is flagged binds-dependency-assembly: the portal reaches other packages by shared=
source inclusion, so that green rests on grounds the portal does not have.

Cascade is generic and unit-tested without a compiler (9 cases): dependency observation,
red-breaks/green-continues, faults, one execution per node however many dependents, bounded
parallelism with real queue timings, cycle refusal, external dependencies, critical path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HWc8YSndL2gpW2i9wRUQWt
Copilot AI lite review requested due to automatic review settings August 30, 2026 14:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new mw-plugin-test build workflow to compile and execute in-mesh “static” test conventions per package as a dependency-driven reactive cascade, producing timing/report outputs without importing sources into a mesh.

Changes:

  • Introduces a generic Rx-based dependency cascade scheduler (Cascade) with timing and critical-path reporting.
  • Adds CascadeBuild to compile NodeTypes from disk, reference dependency-emitted assemblies, and run static Test/*.cs cases in a collectible ALC.
  • Wires the new build verb into the existing CLI and reuses TreeBake module/bundle helpers.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tools/MeshWeaver.PluginTester/TreeBake.cs Exposes module/bundle helpers for reuse by the new build runner.
tools/MeshWeaver.PluginTester/StaticTestRunner.cs Adds no-mesh static test discovery/execution from emitted assemblies.
tools/MeshWeaver.PluginTester/Program.cs Adds mw-plugin-test build CLI verb and usage text.
tools/MeshWeaver.PluginTester/CascadeBuild.cs Implements package selection, compilation, static test execution, and reporting/bundle output.
tools/MeshWeaver.PluginTester/Cascade.cs Adds the reactive dependency-cascade engine with bounded parallelism and timings.
test/MeshWeaver.PluginTester.Test/CascadeTest.cs Adds unit tests covering cascade invariants and failure behaviors.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +177 to +183
var all = idSet.OrderBy(i => i, StringComparer.Ordinal).Select(NodeOf).ToArray();
return all.Length == 0
? Observable.Return(ImmutableArray<NodeResult<T>>.Empty)
: all.CombineLatest()
.Take(1)
.Select(results => results.OrderBy(r => r.Id, StringComparer.Ordinal).ToImmutableArray())
.Finally(slots.Dispose);
Comment on lines +277 to +284
try
{
if (options.OutputDirectory is null && Directory.Exists(workDirectory))
Directory.Delete(workDirectory, recursive: true);
}
catch (IOException)
{
}
Comment on lines +181 to +187
if (!done.Wait(timeout))
{
return new Case(name, Outcome.Failed, clock.Elapsed,
$"did not return within {timeout.TotalSeconds:F0}s — a hung case; the thread is "
+ "abandoned and the build continues");
}
clock.Stop();
Comment on lines +17 to +24
private static readonly IReadOnlyDictionary<string, string[]> Diamond =
new Dictionary<string, string[]>(StringComparer.Ordinal)
{
["Store"] = [],
["AI"] = ["Store"],
["Observability"] = ["Store"],
["Edu"] = ["Store", "AI"],
};
rbuergi and others added 3 commits August 30, 2026 16:09
…gate now calls it stale

The allow file's own rule: delete the line once the change has shipped, because a stale entry
fails the gate. #2771 merged at 14:00; from then on every core PR's binary-compatibility gate
(which runs on the PR's merge ref) read STALE ALLOW ENTRY for
MeshWeaver.Fixture.ObservableAwait — this PR's run 33315891997 did.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HWc8YSndL2gpW2i9wRUQWt
…ss in WritingTests

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HWc8YSndL2gpW2i9wRUQWt
rbuergi and others added 2 commits August 30, 2026 16:20
…, conserved as a page

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HWc8YSndL2gpW2i9wRUQWt
@github-actions

github-actions Bot commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

Test Results (shard 3)

730 tests   538 ✅  4m 46s ⏱️
  5 suites  192 💤
  5 files      0 ❌

Results for commit 1717121.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

Test Results (shard 5)

1 556 tests   1 556 ✅  5m 25s ⏱️
    8 suites      0 💤
    8 files        0 ❌

Results for commit 1717121.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

Test Results (shard 0)

951 tests   951 ✅  5m 16s ⏱️
  6 suites    0 💤
  6 files      0 ❌

Results for commit 1717121.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

Test Results (shard 4)

   10 files     10 suites   6m 8s ⏱️
3 482 tests 3 482 ✅ 0 💤 0 ❌
4 146 runs  4 146 ✅ 0 💤 0 ❌

Results for commit 1717121.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

Test Results (shard 1)

971 tests   967 ✅  5m 57s ⏱️
  6 suites    4 💤
  6 files      0 ❌

Results for commit 1717121.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

Test Results (shard 2)

803 tests   803 ✅  6m 28s ⏱️
  6 suites    0 💤
  6 files      0 ❌

Results for commit 1717121.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

Test Results

   41 files     41 suites   34m 2s ⏱️
8 493 tests 8 297 ✅ 196 💤 0 ❌
9 157 runs  8 961 ✅ 196 💤 0 ❌

Results for commit 1717121.

♻️ This comment has been updated with latest results.

…, the report awaited through the sanctioned bridge

The production blocking-bridge inventory (shard 3, run 33316772312) named four new sites in the
runner: SemaphoreSlim.Wait() in the cascade, Observable.Wait() twice and a GetAwaiter().GetResult()
in the build verb, and nine .Wait() in its tests. It may shrink, never grow, and it is right:
a parked thread under a turn-based scheduler is the deadlock shape this whole day was about.

Now: the cascade bounds parallelism by handing work items to one queue consumed through
Merge(maxParallel) — Rx subscribes at most that many at a time and takes the next as one
completes; no semaphore, and 'Started' is the moment the gate subscribes the item, so the queue
time stays exact. CascadeBuild.Run returns IObservable<Report> with package discovery inside the
chain; Program awaits it through ObservableAwait.Await, the ONE sanctioned bridge. The compiler's
synchronous NuGet hook — the one place the bake lane genuinely must block — becomes a shared
TreeBake helper so the inventory keeps counting exactly one site, where it already was. The
tests await the stream the same way.

Every progress line now leads with the UTC time and the managed thread id (packages build in
parallel, so a reader must be able to tell whose line it is — maintainer, 2026-08-30).

Verified: cascade 9/9; ObservableToTaskBridgeGuard, BlockingBridgeInTestRatchetGuard and
HubDisposalJoinRatchetGuard 14/14 locally.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HWc8YSndL2gpW2i9wRUQWt
@rbuergi
rbuergi merged commit 2201ff4 into main Aug 30, 2026
25 checks passed
rbuergi added a commit that referenced this pull request Sep 1, 2026
The binary-compat gate went red on this branch demanding
`[assembly: TypeForwardedTo(typeof(…DocumentPaths))]` in
src/MeshWeaver.ContentCollections.Indexing — where it already was. TWO
independent false negatives, either of which alone hides a correct forwarder:

1. FORWARD_RE required the BARE attribute name, so a fully-qualified
   `System.Runtime.CompilerServices.TypeForwardedTo` did not count. The
   qualifier is now optional.
2. parse_forwards matched LINE BY LINE, which silently defeated the regex's
   own `\s*`, so a forwarder wrapped across two lines did not count either —
   and wrapping is how a long type name is naturally written. It now strips
   line comments first and matches over the whole text.

The second was not visible from the regex alone: tested in isolation the
pattern spans newlines, and only the CALLER breaks it. Worth stating because
the obvious check passes.

Both matter beyond this branch. This file's own header warns that "a gate that
only understood one of the two spellings would demand a forwarder that is
already there", and a gate that rejects correct work is how people learn to
allow-list past it — which here would be worse than the disease: allow entries
are one-shot and evaluated on the merge ref, so one landing on main reds every
other open core PR's diff as STALE ALLOW ENTRY (measured 08-30: #2771 merged
14:00, #2784 red 14:05).

The declaration on this branch is now the plain bare one-liner. The gate change
is what stops the next person hitting this, and a self-test pins the qualified
+ wrapped shape so a future narrowing cannot quietly bring it back. The #2398
negative control still holds: a commented-out forwarder, on one line or two,
still contributes nothing.

Verified: 29/29 self-tests (28 + the new one), and the gate passes on this branch.

Reported by meshweaver-plugins-cf, who read the regex rather than the symptom.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XM3bKbVAFCCDH9ZyYCxKQy
@rbuergi
rbuergi deleted the feat/cascade-build branch October 10, 2026 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants