Skip to content

fix(compile): the adopted-source fingerprint now covers @@ includes - #3000

Merged
github-actions[bot] merged 1 commit into
mainfrom
fix/2948-fingerprint-covers-includes
Sep 1, 2026
Merged

github-actions[bot] merged 1 commit into
mainfrom
fix/2948-fingerprint-covers-includes

Conversation

@rbuergi

@rbuergi rbuergi commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Closes #2948

The gap

NodeTypeSourceFingerprint.Compute hashed the compile input as
NodeCompileShaping.CollectCompileSources yields it. An @@ include pulls a Code node that no
source query matches
: it reaches the emitted assembly (NodeSetCompiler.ResolveInputs substitutes
it after the fold) and was absent from the hash. So editing an included-only snippet moved neither
AdoptedSourceFingerprint nor CurrentSourceFingerprint, and a prebuilt assembly baked before
that edit still adopted as AdoptedVerified.

That is the worst of the three rows to be wrong on. AdoptedUnverified says "nobody established
where these bytes came from" and an operator reads it as the warning it is. AdoptedVerified is an
assertion that the shipped bytes match the source this mesh holds — standing over source that
was never hashed, it is a false verification: the #2813 incident one layer in.

The fix — both sides resolve includes before hashing

  • Producer. NodeCompileShaping.ResolveCodeIncludes takes an optional collector and records
    resolved path → code text at the point it consumes each include, so the closure is a by-product
    of the one walk that already substitutes them — there is no second traversal to disagree with
    the first. CompileInputs.ResolvedIncludes carries it; TreeBake / CascadeBuild fold it
    straight in. No second walk, no second read, and no new blocking bridge at a synchronous build
    step (the ProductionBlockingBridgeSites ratchet does not move).
  • Consumer. The sources watcher resolves it through the mesh. Reads cannot happen inside a pure
    Update lambda, so the fingerprint moved into an observable step ahead of it, composed with
    Switch() so a newer source set supersedes an in-flight resolution rather than racing it.
    CollectIncludeClosure scans each source's text for @@ and issues no read at all when there
    is none — almost every type — so only a type that actually has includes pays, and it pays the same
    reads its own compile would.

Deterministic and cycle-safe

An order-dependent fingerprint manufactures phantom staleness (endless recompiles, or a false
refusal), which is worse than the gap being closed. Three properties:

  1. Order cannot reach the result. The closure is keyed by resolved path and returned sorted
    (ordinal); PartitionSourceFingerprint sorts again. Roots are walked in CollectCompileSources
    order — itself ordinal by node path — and serially (one SelectMany chain, never a Merge),
    so there is no read interleaving to observe either.
  2. Suppression is result-preserving. The per-root visited set skips a path that root already
    walked; skipping a re-read of the same anchored path can only re-derive an entry that is
    already present, so which parent reached a shared snippet first cannot change the closure.
  3. A cycle terminates. A → B → A adds each path once; the second visit takes the already-added
    branch, reads nothing, recurses no further. A self-include is the same case with one hop.

An unreadable include is INCONCLUSIVE, never absent

The producer's lookup is in-memory and never stalls. A consumer that quietly treated a stalled read
as "the include is gone" would hash a shorter closure — indistinguishable from a stale bundle — so
a perfectly good adoption is refused, and on a Modules:RequirePrebuilt mesh that is terminal.
SourceFingerprintIncludeReader reads via GetMeshNodeOutcome and keeps the three states apart:
Present contributes, Absent contributes nothing (it contributes nothing to the bytes either —
the directive stays verbatim, so both sides agree), and Unavailable / DeleteInProgress / a
timeout raise SourceIncludeUnavailableException. The watcher catches exactly that and leaves the
previously published fingerprint standing
, so the judgement takes the "nothing has been compared"
branch. Same #890 rule: a probe must not answer its scariest branch on its own inability to run.

AdoptedUnverified is still never a refusal. Pinned by a test in this PR.

What was NOT changed, deliberately

The include closure is not added to CompiledSources / CurrentSourceVersions. That is the
recompile-trigger half — a change feed, not a hash — and sourceVersions is a producer/consumer
contract mirroring the RAW query match on both bakes. #2948 closes the verified claim, which
is decided by the two fingerprints and nothing else. Stated in the doc rather than assumed.

BakeEquivalenceTest — reconciled, not loosened

It pinned Widget/Snippets/Greeting absent from sourceVersions. That assertion stays and
is now explained: sourceVersions mirrors CompiledSources (the raw query match), which is a
different field answering a different question. A new assertion pins the include present in
the bake's resolved include set — because without it, assertion 3b's cross-producer hash equality
would be satisfiable by both producers ignoring the include, which is exactly the state #2948
found them in. MeshDrivenAndCompilerDrivenBakes_ProduceTheSameArtifacts passing is now the proof
that the mesh bake (real workspace, mesh reads) and the tree bake (NodeSet) resolve the same
closure.

Binary compatibility

CompileInputs.ResolvedIncludes is a required init member, not a positional parameter:
positional would have changed the primary constructor's arity, which
scripts/check-record-signatures.py correctly rejects as a binary break between a module and the
platform it loads into. required because forgetting it is silent — an empty closure for a type
with includes is a refusal, not "the fingerprint minus the includes". For the same reason the
include-less Compute overload now refuses a set that has an include rather than under-covering
it: a convenience overload that can under-cover is exactly how #2813 sat inert (the 7-arg Seed
hard-coded sourceFingerprint: null and both production callers took it).

Types with no includes fold to the identical value as before, so there is no fleet-wide
fingerprint churn; and this file is inside FrameworkBuildIdentity.FullMvidAssemblies, so a producer
and a consumer that can adopt across each other necessarily run the same implementation.

Verification

Every new test was proven falsifiable — inverted, seen red, restored (output in the PR thread).

  • MeshWeaver.Compiler.Pipeline.Test — 583 passed (whole project, no --filter)
  • MeshWeaver.PluginTester.Test — 312 passed (whole project; includes BakeEquivalenceTest)
  • MeshWeaver.Documentation.Test — 166 passed (whole project), plus a broken-link probe proving
    DocumentationLinkIntegrityTest names both edited pages
  • dotnet build -c Release -warnaserror — 61 projects, 0 Warning(s) 0 Error(s)
  • scripts/check-record-signatures.py — clean

Docs: Doc/Architecture/NodeTypeCompilation gains the mechanism (determinism argument, cycle brake,
the inconclusive rule) and its "deliberately does NOT do" list is rewritten to state what is still
open. What's New entry, Category: Fix.

…2948)

`NodeTypeSourceFingerprint.Compute` hashed the compile input as
`NodeCompileShaping.CollectCompileSources` yields it. An `@@` include pulls a
Code node that NO source query matches: it reaches the emitted assembly
(`NodeSetCompiler.ResolveInputs` substitutes it AFTER the fold) and was absent
from the hash. So editing an included-only snippet moved neither
`AdoptedSourceFingerprint` nor `CurrentSourceFingerprint`, and a prebuilt
assembly baked before that edit still adopted as `AdoptedVerified`.

That is the worst of the three rows to be wrong on. `AdoptedUnverified` says
"nobody established where these bytes came from" and reads as the warning it
is; `AdoptedVerified` is an ASSERTION that the shipped bytes match the source
this mesh holds. Standing over source that was never hashed it is a FALSE
VERIFICATION — the #2813 incident one layer in.

Both halves now resolve includes before hashing:

- PRODUCER: `ResolveCodeIncludes` takes an optional collector and records
  `resolved path -> code text` at the point it consumes each include, so the
  closure is a by-product of the ONE walk that already substitutes them.
  `CompileInputs.ResolvedIncludes` carries it; TreeBake/CascadeBuild fold it
  straight in — no second walk, no second read, no new blocking bridge.
- CONSUMER: the sources watcher resolves it through the mesh. Reads cannot
  happen inside a pure `Update` lambda, so the fingerprint moved into an
  observable step ahead of it, composed with `Switch()` so a newer source set
  supersedes an in-flight resolution.

Deterministic and cycle-safe: the closure is keyed by resolved path and
returned SORTED (ordinal), roots are walked in `CollectCompileSources` order
and SERIALLY (one SelectMany chain, never a Merge), and the per-root visited
set both brakes cycles (A->B->A, and self-includes) and is result-preserving —
skipping a re-read of the same anchored path can only re-derive an entry that
is already there. An order-dependent fingerprint would manufacture phantom
staleness, which is worse than the gap being closed.

And an unreadable include is INCONCLUSIVE, never absent.
`SourceFingerprintIncludeReader` reads via `GetMeshNodeOutcome` and keeps the
three states apart: Present contributes, Absent contributes nothing (it
contributes nothing to the bytes either), Unavailable/DeleteInProgress/timeout
raise `SourceIncludeUnavailableException`. The watcher catches exactly that and
leaves the previous fingerprint standing, so the judgement degrades to
`AdoptedUnverified`. Degrading a stalled read to "absent" would shorten the
closure, hash differently from every honest producer and REFUSE a good
adoption — terminal on a `Modules:RequirePrebuilt` mesh. Same #890 rule: a
probe must not answer its scariest branch on its own inability to run.

`AdoptedUnverified` is still never a refusal, and the include closure is
deliberately NOT added to `CompiledSources`/`CurrentSourceVersions`: that is
the recompile-TRIGGER half (a change feed, not a hash) and a producer/consumer
contract `BakeEquivalenceTest` pins. That test is reconciled, not loosened —
the include stays asserted ABSENT from `sourceVersions`, and a new assertion
pins it PRESENT in the bake's resolved include set, so assertion 3b's
cross-producer hash equality can no longer be satisfied by both producers
ignoring it.

The include-less `Compute` overload REFUSES a set that has an include rather
than silently under-covering — a convenience overload that can under-cover is
exactly how #2813 sat inert (the 7-arg `Seed` hard-coded
`sourceFingerprint: null` and both production callers took it).

Closes #2948

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 1, 2026 18:21
@github-actions
github-actions Bot enabled auto-merge September 1, 2026 18:21
@rbuergi

rbuergi commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

Falsification output — every new test proven able to fail

A test that passes after a change proves nothing unless it would have failed before it. Four
mutations, each applied to the working tree, built, run, then reverted. All 14 tests in
SourceFingerprintIncludeClosureTest pass unmutated.

Probe 1 — CollectIncludeClosure always returns the empty closure (i.e. #2948 never fixed)

Failed ...EditingAnIncludedOnlySnippet_MovesTheFingerprint
  Did not expect value to be "b59c18eb06dc5846" because the snippet is INSIDE the emitted bytes,
  so a change to it must move the hash — otherwise an assembly compiled before the edit still
  claims AdoptedVerified against source it was never built from.

Failed ...ABundleBakedBeforeAnIncludedOnlyEdit_IsRefused_NotVerified
  Expected value to be AdoptionRefused because the only difference between the two builds is an
  included-only snippet, and that is the whole of #2948 — before the fix these fingerprints were
  EQUAL and the verdict was AdoptedVerified, but found AdoptedVerified.

Failed ...ACyclicIncludeTerminates_AndIsStable
  Did not expect value to be "7d43d876b1693e92" because B is in the closure — the cycle brake
  stops the walk, not the coverage.

Failed ...ANestedIncludeIsCovered_Transitively
Failed ...AnUnreadableInclude_IsInconclusive_NeverAbsent
Failed ...ThePreResolvedOverload_AgreesWithTheReaderOverload

Failed!  - Failed: 6, Passed: 8, Total: 14

That second one is the issue's acceptance criterion, reproduced exactly: on main the verdict for
an included-only edit is AdoptedVerified.

Probe 2 — an UNSTABLE token (SHA256(code + Guid.NewGuid())): the hash moves when nothing changed

Failed ...TheFingerprintIsIndependentOfDeliveryOrder
Failed ...ACyclicIncludeTerminates_AndIsStable
Failed ...AnEditToANodeTheCompileNeverSees_LeavesTheFingerprintAlone
Failed ...AnIncludeThatResolvesToNothing_ContributesNothing
Failed ...ASetWithoutIncludes_FoldsIdenticallyThroughEveryOverload
Failed ...ThePreResolvedOverload_AgreesWithTheReaderOverload

Failed!  - Failed: 6, Passed: 8, Total: 14

This is the phantom-staleness control arm: the no-move / order-stability / cross-overload-agreement
family all bite.

Probe 3 — the include-less Compute overload stops refusing an include-bearing set

Failed ...TheIncludeLessOverload_RefusesASetThatHasAnInclude
  Expected a InvalidOperationException, but no exception was thrown.

Failed!  - Failed: 1, Passed: 13, Total: 14

Probe 4 — ApplyAdoptedSourceStamp refuses EVERY row

Failed ...ALegacyBundleWithNoFingerprint_StillAdopts
Failed ...ABundleWhoseIncludeStillMatches_AdoptsAsVerified

Failed!  - Failed: 2, Passed: 12, Total: 14

Note what stays green under probe 4: the refusal test itself. That is the point of carrying the
two positive controls — "widen the refusal" would satisfy the headline test while breaking the
anti-outage property that AdoptedUnverified is never refused.

Doc probe — DocumentationLinkIntegrityTest actually covered the edited pages

A green trx that names no pages proves nothing, so a deliberately broken link was inserted into each
page this PR touches:

Failed MeshWeaver.Documentation.Test.DocumentationLinkIntegrityTest.AllInternalDocLinks_ResolveToExistingNodes
  Doc/WhatsNew/2026-09-02-a-shared-snippet-no-longer-hides-behind-a-verified-build:
      (/Doc/Architecture/AlsoDoesNotExist2948) — resolves to /Doc/Architecture/AlsoDoesNotExist2948,
      which does not exist
  Doc/Architecture/NodeTypeCompilation:
      (/Doc/Architecture/ThisPageDoesNotExist2948) — resolves to
      /Doc/Architecture/ThisPageDoesNotExist2948, which does not exist, but found 2 item(s).

Both pages are named, so the suite is genuinely reading them. Restored, and the whole project is
green (166 passed).

Unmutated

MeshWeaver.Compiler.Pipeline.Test   Passed! - Failed: 0, Passed: 583, Total: 583
MeshWeaver.PluginTester.Test        Passed! - Failed: 0, Passed: 312, Total: 312
MeshWeaver.Documentation.Test       Passed! - Failed: 0, Passed: 166, Total: 166
dotnet build -c Release -warnaserror   61 projects, 0 Warning(s), 0 Error(s)
scripts/check-record-signatures.py     ✓ 10 changed C# file(s): no binary-breaking changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a correctness gap in build provenance verification: NodeTypeSourceFingerprint now incorporates @@-included Code nodes (which are compiled into the output but were previously invisible to source-query-based hashing). This prevents stale prebuilt assemblies from being incorrectly marked AdoptedVerified when only included snippets changed.

Changes:

  • Producer side: capture the resolved @@ include closure during include substitution (CompileInputs.ResolvedIncludes) and fold it into the source fingerprint during baking.
  • Consumer side: resolve @@ include closure through the mesh when computing CurrentSourceFingerprint, with deterministic, cycle-safe traversal and an “inconclusive” failure mode for unreadable includes.
  • Add/adjust tests and documentation to pin the contract (include closure affects fingerprint but not sourceVersions) and validate refusal/adoption outcomes.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tools/MeshWeaver.PluginTester/TreeBake.cs Folds resolved include closure into baked fingerprint; exposes resolved include paths for reporting/tests.
tools/MeshWeaver.PluginTester/CascadeBuild.cs Folds resolved include closure into baked fingerprint for cascade builds.
tools/MeshWeaver.PluginTester/BakeOutput.cs Computes fingerprint via include-reader path so mesh-driven bake matches compiler-driven bake.
test/MeshWeaver.PluginTester.Test/BakeEquivalenceTest.cs Pins include targets absent from sourceVersions but present in resolved include set (and thus fingerprint input).
test/MeshWeaver.Compiler.Pipeline.Test/SourceFingerprintIncludeClosureTest.cs New unit tests covering include closure hashing, determinism, cycles, inconclusive reads, and adoption verdicts.
src/MeshWeaver.Documentation/Data/WhatsNew/2026-09-02-a-shared-snippet-no-longer-hides-behind-a-verified-build.md Adds user-facing “What’s New” entry describing the verification fix.
src/MeshWeaver.Documentation/Data/Architecture/NodeTypeCompilation.md Documents the new fingerprint behavior, determinism arguments, and what remains deliberately unchanged.
src/MeshWeaver.Compiler/NodeTypeSourceFingerprint.cs Extends fingerprinting to include the resolved @@ closure; adds overloads and refusal behavior for include-bearing sets without closure.
src/MeshWeaver.Compiler/NodeSetCompiler.cs Records resolved includes during ResolveInputs and surfaces them via CompileInputs.ResolvedIncludes.
src/MeshWeaver.Compiler/NodeCompileShaping.cs Adds CollectIncludeClosure and optional closure collection to ResolveCodeIncludes.
src/MeshWeaver.Compiler.Pipeline/SourceFingerprintIncludeReader.cs New mesh reader for include closure resolution with “inconclusive != absent” semantics.
src/MeshWeaver.Compiler.Pipeline/NodeTypeCompilationHelpers.cs Moves fingerprint computation upstream of the Update lambda and uses Switch() to supersede in-flight resolutions.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 0)

216 tests   216 ✅  2m 25s ⏱️
  1 suites    0 💤
  1 files      0 ❌

Results for commit 81e2e25.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 3)

586 tests   586 ✅  50s ⏱️
  2 suites    0 💤
  2 files      0 ❌

Results for commit 81e2e25.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 1)

419 tests   419 ✅  1m 13s ⏱️
  1 suites    0 💤
  1 files      0 ❌

Results for commit 81e2e25.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 5)

    3 files      3 suites   1m 11s ⏱️
1 218 tests 1 026 ✅ 192 💤 0 ❌
1 219 runs  1 027 ✅ 192 💤 0 ❌

Results for commit 81e2e25.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 2)

529 tests   527 ✅  1m 31s ⏱️
  4 suites    2 💤
  4 files      0 ❌

Results for commit 81e2e25.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 4)

1 685 tests   1 685 ✅  1m 44s ⏱️
    3 suites      0 💤
    3 files        0 ❌

Results for commit 81e2e25.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Test Results

   14 files     14 suites   8m 57s ⏱️
4 653 tests 4 459 ✅ 194 💤 0 ❌
4 654 runs  4 460 ✅ 194 💤 0 ❌

Results for commit 81e2e25.

@github-actions
github-actions Bot merged commit dc6c5a4 into main Sep 1, 2026
27 checks passed
@rbuergi
rbuergi deleted the fix/2948-fingerprint-covers-includes branch September 13, 2026 10:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The adopted-source fingerprint does not cover @@ includes — a change to an included-only snippet is invisible to staleness detection

2 participants