Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/MeshWeaver.Documentation/Data/Architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ The catalog below lists every page; these are the load-bearing reads per theme.
| **UI** | [User Interface](UserInterface) | [Blazor Data Binding](BlazorDataBinding) · [Per-Tab Session State](PerTabSessionState) — a node is shared by every tab of one account, so "which page is this viewer on" and "navigate ME there" can never live on one · [Blazor Async](BlazorAsync) · [Available Controls](UserInterface/AvailableControls) · [Menus as Data](MenuAsData) — re-word a menu without a build · [The Menu Contribution Boundary](MenuContributionBoundary) — what may be data, what stays compiled · [Markdown Fence Extensions](MarkdownFenceExtensions) — the platform emits a marker, the clients hydrate it; a new fence is always a two-repo change · [Localization](Localization) — the viewer's language, resolved explicitly, never from ambient culture · [Chrome and Content Language](ChromeAndContentLanguage) — ownership decides the language, and in-flow chrome minimises words · [The Supplied Navigation Rail](SuppliedNavigationRail) — a module supplies its own left-hand index, and core renders it in the order the module gave |
| **Node types** | [Adding a New Node Type](AddingANewNodeType) | [Retiring a NodeType](RetiringANodeType) — the prune keeps the definition and deletes its sources · [Dangling NodeTypes](DanglingNodeTypes) — a node whose type resolves to nothing, and the two write paths that allowed it · [Node Type Compilation](NodeTypeCompilation) · [Execute-Time Interlock](ExecuteTimeInterlock) — a build proven stale is never armed · [Graph / Compiler Layering](GraphCompilerLayering) — the four assemblies, the cycle, and the full-MVID size rule · [Toolchain Re-evaluation Lane](ToolchainReevaluationLane) — why a toolchain change stopped rebaking the world · [Rebake Waves](RebakeWaves) — why a roll rebakes the world anyway, and what one rebake writes · [Import Write Ordering](ImportWriteOrdering) — type before instance, and what a foreign type does · [Language Services](LanguageServices) · [Extensible Defaults](ExtensibleDefaults) |
| **Plugins & content delivery** | [Plugins](Plugins) — node repos from git, no NuGet | [Plugin Manual](PluginAuthoring) — author · publish · install · own registry · [Plugin Registry](PluginRegistry) — memex re-serves plugins over REST · [Webhook Inbox](WebhookInbox) — external services deliver into {target}/_Inbox · [Plugin Packaging](PluginPackaging) — bundles, the framework identity, and the `Release` node that links a release to its assemblies per architecture · [Install Readability](InstallReadability) — the two doors an install can open, and the cover-grant deadlock detector · [Static Repo Import](StaticRepoImport) · [Node Type Compilation](NodeTypeCompilation) · [The Platform Image's Closure](PlatformImageClosure) — the image IS the reference set every satellite's modules compile against; the two invariants, and why every consumer used to discover them by failing to compile |
| **Reliability & wedges** | [Error Propagation & Wedges](ErrorPropagationAndWedges) — drive wedges to 0 | [Action-Block Wedge Prevention](ActionBlockWedgePrevention) · [Riding Out a ShuttingDown Address](RidingOutAShuttingDownAddress) — the one transient NACK, and the two axes a ride-out must bound separately · [Hub Initialization Failure](HubInitializationFailure) · [Orleans Stream Pub-Sub Durability](OrleansStreamPubSubDurability) — a publish with no subscriber succeeds, so a cross-silo reply can vanish with nothing logged · [Durable Streams Are Mesh Nodes](DurableStreamsViaMeshNodes) — the design that retires the memory stream without a provider · [The Pod-Hub Claim Must Be Re-Asserted](PodHubClaimReassertion) — a claim asserted once into a directory that is re-partitioned on every membership change is lost silently, and forever · [Oversized Delivery Refusal](OversizedDeliveryRefusal) — a message too large for its transport destroys the connection carrying it; refuse at the producer, never raise the limit · [Content Sync Visibility](ContentSyncVisibility) — a Space whose assets the transport refuses says so, on the Space itself, naming the file, its size and the limit · [Out-of-Band Content Transfer](OutOfBandContentTransfer) — a content file too large for one delivery travels through the content store behind a content-addressed handle, never on the message · [An Unreachable Store Is Not a Refusal](StoreUnreachableIsNotARefusal) — one classification, three consumers; reporting an availability failure as a verdict is how a retried create becomes a duplicate · [Reading a Silo Eviction](ReadingASiloEviction) — a heartbeat newer than the suspect votes is not proof the silo was healthy; the control arm that tells a correct eviction from a false positive · [Dead-Circuit Fan-Out Storm](DeadCircuitFanOutStorm) — a closed tab's owner pushed to the corpse for 46 minutes because the only verdict the eviction acts on could not be said; the release tombstone that says it · [Bake Seal — NodeOps Saturation](BakeSealNodeOpsSaturation) — the mesh's ONE node-CRUD hub stops draining under a bulk burst, and every consumer then reports its own bound · [The /api/content 503](ContentRoute503) — two causes with opposite fixes, and the one field in the line already printed that tells them apart · [Refused Replies During Teardown](RefusedRepliesDuringTeardown) — every failure route answers the SENDER, which for a reply is the responder; the answer the caller is parked on is dropped with nobody told |
| **Reliability & wedges** | [Error Propagation & Wedges](ErrorPropagationAndWedges) — drive wedges to 0 | [Action-Block Wedge Prevention](ActionBlockWedgePrevention) · [Riding Out a ShuttingDown Address](RidingOutAShuttingDownAddress) — the one transient NACK, and the two axes a ride-out must bound separately · [Hub Initialization Failure](HubInitializationFailure) · [Orleans Stream Pub-Sub Durability](OrleansStreamPubSubDurability) — a publish with no subscriber succeeds, so a cross-silo reply can vanish with nothing logged · [Durable Streams Are Mesh Nodes](DurableStreamsViaMeshNodes) — the design that retires the memory stream without a provider · [The Pod-Hub Claim Must Be Re-Asserted](PodHubClaimReassertion) — a claim asserted once into a directory that is re-partitioned on every membership change is lost silently, and forever · [Oversized Delivery Refusal](OversizedDeliveryRefusal) — a message too large for its transport destroys the connection carrying it; refuse at the producer, never raise the limit · [Content Sync Visibility](ContentSyncVisibility) — a Space whose assets the transport refuses says so, on the Space itself, naming the file, its size and the limit · [Out-of-Band Content Transfer](OutOfBandContentTransfer) — a content file too large for one delivery travels through the content store behind a content-addressed handle, never on the message · [An Unreachable Store Is Not a Refusal](StoreUnreachableIsNotARefusal) — one classification, three consumers; reporting an availability failure as a verdict is how a retried create becomes a duplicate · [Undetermined Is Not No](UndeterminedIsNotNo) — a read that did not answer is a THIRD state; the second door that shared the first door's failure domain, and the rule for what a gate does with "I could not determine" · [Reading a Silo Eviction](ReadingASiloEviction) — a heartbeat newer than the suspect votes is not proof the silo was healthy; the control arm that tells a correct eviction from a false positive · [Dead-Circuit Fan-Out Storm](DeadCircuitFanOutStorm) — a closed tab's owner pushed to the corpse for 46 minutes because the only verdict the eviction acts on could not be said; the release tombstone that says it · [Bake Seal — NodeOps Saturation](BakeSealNodeOpsSaturation) — the mesh's ONE node-CRUD hub stops draining under a bulk burst, and every consumer then reports its own bound · [The /api/content 503](ContentRoute503) — two causes with opposite fixes, and the one field in the line already printed that tells them apart · [Refused Replies During Teardown](RefusedRepliesDuringTeardown) — every failure route answers the SENDER, which for a reply is the responder; the answer the caller is parked on is dropped with nobody told |
| **Testing & debugging** | [Writing Tests](WritingTests) | [Negative Controls](NegativeControls) — a pin is only a pin if it fails against the defect · [Reactive Test Assertions](ReactiveTestAssertions) · [Test State Isolation](TestStateIsolation) · [Disposable-mesh e2e](DisposableMeshE2E) · [Debugging Message Flow](DebuggingMessageFlow) · [Debugging Disposal & Leaks](DebuggingDisposalAndLeaks) · [Ambient Test-Host Hangs](AmbientTestHostHangs) — what decides whether a killed test host can be diagnosed at all, and the readings of it already falsified · [In-Mesh Tests and the Seal](InMeshTestsAndTheSeal) — a Tests area no required context executes is a latent trunk red the seal detonates fleet-wide; how to measure a gate before requiring it |
| **Deployment & ops** | [Deployment](Deployment) (the router) | [AKS](DeploymentAKS) · [Database Migration Procedure](DatabaseMigrationProcedure) — the schema moves before the image, every roll; the 2026-09-03 wedge behind a 200, the recovery, and why a migration deadlocks under load · [Container Apps](DeploymentContainerApps) · [Local Dev Workflow](LocalDevWorkflow) · [Onboarding a New Environment](OnboardingNewEnvironment) · [Release & Self-Update Strategy](ReleaseStrategy) · [The Continuous Delivery Contract](ContinuousDeliveryContract) — all-or-nothing publication; verify the image, never the tick · [The Self-Update Schema Wall](SelfUpdateSchemaWall) — every schema-bumping release is un-takeable by self-update, the stall is invisible, and a promoted tag is not a deployable tag · [Bake Identity Mismatch](BakeIdentityMismatch) — why a green CD can publish a bake no portal adopts, and the one rule that keeps two images of one commit on one address · [Release Availability Gates](ReleaseGates) — one predicate; never roll or build into a release a package cannot survive · [Combo Gate Wiring](ComboGateWiring) — the roll consults the combo verdict; Red refuses, and "could not find out" is neither · [Release Process](ReleaseProcess) · [Repository Dependency Direction](RepositoryDependencyDirection) — the platform never depends on a plugin repo; the inventory of every edge that still does · [The Cross-Repo Pair Gate](CrossRepoPairGate) — a removal here reds a plugin repo's trunk hours later; the deleting half lands LAST · [Transitional Allow Entries](TransitionalAllowEntries) — an allow entry is written for ONE merge and expires with it by mechanism; the instruction that was ignored once cost every C#-touching PR in the fleet ~40 minutes of red · [Image Pair Skew](ImagePairSkew) — a promoted image pairs a core commit with a Plugins head resolved hours later; each half green, the pair never run (the 2026-09-03 sign-in outage) · [The Merge Queue](MergeQueue) — one entry built at a time so nothing churns, and a steward that re-queues an ejected PR on evidence and never re-runs · [Carving Projects Out Of Core](CarvingProjectsOutOfCore) — what a SOURCE move costs and what it does not · [Red-Log Watching & Ticketing](LogWatchTriage) — every `fail:`/`crit:` becomes exactly one triaged issue · [Measuring a Live Portal Read-Only](MeasuringALivePortalReadOnly) — the four read-only instruments, and why an absence needs a coverage fact before it counts as evidence · [Chart Ownership and the Runner Pool](ChartOwnershipAndRunnerPool) — why the chart's gate is here, what a relocation must carry, and why path-filtering it is unsafe · [Sharding the Node-Repo Gate](NodeRepoGateSharding) — a cap cut reports as `cancelled`, so the fan-out that removes it, and the fold that keeps ONE required context and ONE gate log · [Probe Semantics](ProbeSemantics) — readiness, liveness and startup ask three different questions with three different remedies; why they get three paths and three tags · [Why a GC-Bound Pod Stays in Rotation](WhyAGcBoundPodStaysInRotation) — the GC's hard limit sits below the container limit, so a portal short of memory is defended rather than restarted |
| **Contributing docs** | [Authoring Documentation](AuthoringDocumentation) | [Docs Follow The Functionality](DocsFollowTheFunctionality) — which repo a page belongs in, and what pins the rest here · [Specifying Software](SpecifyingSoftware) · [Glossary](/Doc/Glossary) |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -364,11 +364,12 @@ Four properties are load-bearing, and each is pinned by a case in
fingerprint up by exact key; a GO for any other image reads as no GO and the door stays shut.
Granting on a foreign GO would certify a build the process is not running — strictly worse than
the refusal it replaces.
- **Fail-closed is unchanged.** Neither door answering — including the two cases `ReadBuildGo`
deliberately folds into `null`, no durable store and a failed read — re-throws the ORIGINAL
`BuildCoordinationUnreachableException`, so the sweep still faults, readiness is still refused,
and `DescribesUnreachableCoordination` still separates "no verdict" from "a bad verdict" in the
health payload.
- **Fail-closed is unchanged where the witness ANSWERS.** A witness that says "no GO for this
fingerprint" re-throws the ORIGINAL `BuildCoordinationUnreachableException`, so the sweep still
faults, readiness is still refused, and `DescribesUnreachableCoordination` still separates "no
verdict" from "a bad verdict" in the health payload — and it refuses even when this pod's own
share is fully baked, because a pod does not certify from its own volume a build the coordination
node never approved (`NoDurableGo_StillRefuses_EvenWhenTheShareIsFullyBaked`).
- **The transport fault stays fully visible.** Nothing here widens a timeout, retries, polls or
swallows. `RetryUnreachableCoordination` logs the unreachability at its own severity with its own
diagnostic detail either way, and the grant logs a second `Warning` naming what could not be
Expand All @@ -383,6 +384,32 @@ Four properties are load-bearing, and each is pinned by a case in
`WithdrawBuildClaim` would post a second write into the same unreachable hub. The two paths share
the post-GO share probe; they deliberately do not share the stand-down.

### 🚨 …and the second door was in the FIRST door's failure domain (#3404, second half)

The first cut of that door had **two** branches — a GO, or no GO — while `ReadBuildGo` folds
**three** outcomes into its `null`: the row carries no GO, there is no durable store, and *the read
failed*. So a read that never completed was acted on **and narrated** as a definitive negative
(*"the durable witness carries no GO for framework X"*).

That is not a corner case here, because **the second door is not in a different failure domain from
the first**. In the fleet's portal wiring `AddPartitionStorageHubs` replaces `IStorageAdapter` with
`RoutingProxyAdapter`, whose `Read` is
`hub.Observe<ReadNodeResponse>(new ReadNodeRequest(path, options), o => o.WithTarget(addr))` — the
same hub transport under the same 60 s budget as the `SubscribeRequest`. Of the five candidates
below, the first, fourth and fifth take **both** doors down together, and the durable read then
fails with the same `TimeoutException`. On the very fault this door exists for, the expected
reading is *undetermined*, not *no GO*.

So the reading is now a three-state `BuildGoReading` (`Go` / `NoGo` / `Undetermined`, always with a
`Detail` and, when there was a fault, its `Error`), read through `ReadBuildGoReading`; `ReadBuildGo`
keeps its fold for the callers whose negative branch is *bake*, and its contract now says a caller
that REFUSES on the negative must use the three-state read instead. On `Undetermined` the pod does
not guess in either direction: it measures on the one witness the broken transport cannot touch —
its own `IAssemblyStore` — and grants only when `NodeTypeBakeReport.GateRelevant` is empty, which is
**stricter** than the GO branch (a still-pending type after a GO is non-gating, so a half-baked
share passes *with* a GO and is refused *without* one). The full argument, and the general rule, is
[Undetermined Is Not No](../UndeterminedIsNotNo).

🚨 **The durable door is a FAIL-SAFE, not a cure — and the cause is still open.** The pod's own
diagnostic names three candidates for the silence: the request never reached the target (routing),
the target received it and is wedged (a per-node hub that stops answering — the #2896 class), or the
Expand Down
Loading
Loading