Skip to content

A proven-commit import applies modifications but not additions — the partition is left referenced-but-incomplete, and a force update imports 0 nodes #4456

Closed

Description

@rbuergi

What is failing

A GitSync import at a proven commit applies modifications to a partition but does not create files the same commit adds. The partition is then referenced-but-incomplete, its NodeTypes cannot compile, and the bake gate holds readiness — so the instance cannot roll. A forced re-import does not heal it: it imports 0 nodes.

Measured on memex.systemorph.com, 2026-09-15, rolling onto sealed set 3.0.0-ci.8692 (core 3cd1935709, plugins sealed at 85515f62).

The import

Hosting/_Activity/10f16763 — "Update Hosting to the built commit 85515f62", 19:00:52 → 19:01:48:

Recompiling 8 NodeType(s): Hosting/Admin, Hosting/Deployment, Hosting/Backup, Hosting/FleetConsole,
                           Hosting/InstanceAction, Hosting/InstanceRequest, Hosting/Issue, Hosting/PlatformBuildInbox
Adopted 8 prebuilt assembly(ies) — the rest compile.
Pruned 118 node(s) absent from the repo: Hosting/Triage/ci-failure/…, Hosting/Triage/ci-green/…, Hosting/Triage/Status
Re-imported ImportedWithErrors (41 node(s)) at 85515f62.

Hosting/_GitSync afterwards: lastAttemptedCommitSha: 85515f62, lastSyncOutcome: ImportedWithErrors, lastSyncCommitSha still c7f03952.

What landed and what did not

In Hosting/Deployment/Source/, that import updated two existing nodes — PlatformBuildInboxWatcher (v14) and HelmValues (v8), both lastModified 2026-09-15T19:01:12Z. In the same commit range it did not create Hosting/Deployment/Source/SelfUpdateRouting. The partition holds 13 Code nodes in that folder and it is not among them.

SelfUpdateRouting.cs is present at 85515f62 (git ls-tree), and at that same commit it is referenced by Hosting/Deployment/Source/PlatformBuildInboxWatcher.cs — the very file the import did apply — and by Hosting/PlatformBuildInbox/Test/PlatformBuildInboxTests.cs. So the import wrote the reference and skipped the referent.

🚨 Hosting/ is byte-identical between 85515f62 and today's main (git diff --stat 85515f62 origin/main -- Hosting is empty). This is not a "newer content" or seal-ordering problem: the importer had one consistent tree and applied it partially.

The consequence

On the 3.0.0-ci.8692 replica (memex-portal-deployment-6546c779d4-4ngqv):

CS0103 Error (line 6362): The name 'SelfUpdateRouting' does not exist in the current context
CS0246 Error (line 7308): The type or namespace name 'SelfUpdateRouting' could not be found
DynamicTypePreWarmer: REFUSING READINESS — 5 NodeType(s) regressed on this image:
  Essentials/Mailbox, Hosting/InstanceAction, Hosting/Issue, Hosting/ModuleReportInbox, Hosting/PlatformBuildInbox
BuildProtocol: NOT publishing GO for s4b2836629f0c58ad4ed0aa683190981a — 5 gating regression(s)

The gate behaved correctly: the previous replicas kept serving and there was no outage. But the rollout cannot complete, and the helm-release deploy failed at "Observe the rollout until it completes" after 28 minutes.

The repair path does not repair

git_hub_sync update force: true on Hosting (branch HEAD = main, whose Hosting tree is identical to the sealed commit) — Hosting/_Activity/89ae5941:

Pruned 6 node(s) absent from the repo: Hosting/Triage/…
Imported ImportedWithContentErrors (0 node(s)).

Zero nodes imported, and the missing file is still missing. A force update is documented as "overwriting local changes"; it cannot create the absent node either. That is what makes this a hard stop rather than an inconvenience: there is no supported operation that restores the partition.

Same shape, second partition

Essentials/Mailbox fails identically on MailboxLiveTests (CS0103/CS0246, 7 sites): the referencing test-area file is in the mesh, Essentials/Mailbox/Test/MailboxLiveTests.cs is not. Unlike Hosting, Essentials/ on main is 10 files ahead of 85515f62, so that partition additionally carries content newer than the seal — the two-writer mix of #4355 — but the immediate failure is the same missing-addition shape.

Why this is worth fixing as its own defect

#4355 is about two writers disagreeing over one partition. This is narrower and, I think, more serious: one importer, one consistent tree, and additions silently dropped — with ImportedWithErrors / ImportedWithContentErrors as the only signal, and no per-node error in the activity log naming what failed. The outcome string is on the sync config where nothing reads it; the activity's maxSeverity is Information and its status is Succeeded.

Suggested, in order of value:

  1. An import that could not write a node must say which node and why, at Warning or Error, in the activity log — ImportedWithErrors (41 node(s)) names neither the failures nor the count of them.
  2. A force update must be able to create a missing node. Today it imported 0 while a referenced file was absent from the partition.
  3. A partition left referenced-but-incomplete should not be handed to the compiler, or the compile failure should name the import that produced it — the operator's visible symptom is CS0103 on a symbol whose file is in git.

Evidence

  • Hosting/_Activity/10f16763, Hosting/_Activity/89ae5941, Hosting/_GitSync, Essentials/_GitSync on memex.systemorph.com
  • pod memex-portal-deployment-6546c779d4-4ngqv (3.0.0-ci.8692), 18:56Z onwards
  • Memex deploy run 35010439754 (failed at Observe the rollout), Ops/Actions/helm-memex-20260915-deploy-8692
  • git ls-tree -r 85515f62 -- Hosting/Deployment/Source/SelfUpdateRouting.cs; git diff --stat 85515f62 origin/main -- Hosting (empty)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions