Skip to content

fix(sync): sources sync on PUSH (a red build holds nothing) + prune only what the repo put there - #6122

Merged
rbuergi merged 1 commit into
mainfrom
fix/gitsync-on-push
Oct 5, 2026
Merged

rbuergi merged 1 commit into
mainfrom
fix/gitsync-on-push

Conversation

@rbuergi

@rbuergi rbuergi commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

What

Two policies. Both are registered in PolicyNotProse, and the manual is Doc/Architecture/SourcesSyncOnPush.

sources-sync-on-push. A push to a sync source's configured branch imports every source of that repository and branch at the pushed commit (after), through the new UpdateToPushedCommitFromGitHub. Nothing waits for a CI verdict. Inside the import, each module is still judged by ModuleSyncDecision.Decide:

  • unchanged: nothing is written;
  • changed: the module syncs;
  • declared floor above the running platform: that module alone is declined, with both versions named.

The other parts of the change:

  • Green builds. A green workflow_run now only records BuildCompletion, which the plugin catalog consumes. It no longer imports, because a green build of an older commit that finishes after a newer push would move the Space backwards.
  • Lost deliveries. GitSyncBranchReconcileService runs every GitSync:BranchReconcileMinutes (default 10). It calls GitHubWebhookProcessor.ReconcileBranches(), which does one GetHeadSha lookup per repository branch and then runs the same per-source selection at that sha. A lost webhook therefore repairs itself, and a source that is already on the head costs only the lookup.

prune-requires-provenance. StaticRepoImporter.ComputePrunableNodes now prunes a node only when that node is in the prior import manifest, in every pruning mode. FullReplace used to prune every extra node. Nodes with unknown provenance are never pruned.

Why (measured on memex.systemorph.com)

  • The freeze. Plugins main was red from 2026-10-04 14:39Z. AI/_GitSync and Hosting/_GitSync were last attempted at 14:56Z, then nothing happened for 15 h. A manual update at 05:29Z imported immediately: 18 NodeTypes recompiled, 0 compile errors. The green-build trigger had re-introduced the whole-repository hold that module-sync-per-manifest-hash removed.
  • The prune. That same manual import pruned 16 runtime nodes, including Hosting/Babysitter and Hosting/Queues/*.

Overlap with open PRs

None of the five touches GitHubWebhookProcessor or ComputePrunableNodes.

Local results (Release, -warnaserror, one project per build)

Builds: 0 warnings, 0 errors for each of GitSync, Graph.Test, Hosting.Test, Documentation.Test, Memex.Portal.Shared, Memex.Portal.Shared.Test, Messaging.Hub.Test and PluginTester.

Suite Scope Result
Graph.Test full 2510/2510
Documentation.Test full 670/670
Hosting.Test webhook, sync and prune classes 43/43, then 97/97 on a wider filter
Memex.Portal.Shared.Test import and webhook classes 60/60
Messaging.Hub.Test Localization 66/66

Negative controls (actually run):

  • Restoring the old FullReplace rule fails 3 tests: ARuntimeNodeSurvivesAnImportTest and two in StaticRepoImporterSyncModeTest.
  • Disabling the push import fails 2 tests in BuildTriggeredSyncPinsTheBuiltCommitTest: the push-while-red test, and the test where a green build does not import but a push does.

Not established

  • Production behaviour after the roll. It is not checked whether the hooks deliver push events.
  • No end-to-end decline of an incompatible module. The local -ci.0 platform makes floors advisory, so the decline is pinned by a pure decision test on a pushed tree.
  • Replicas. Each replica runs the reconcile, so a duplicate import can happen. It is cheap, not prevented.
  • Out-of-order push deliveries can still step a Space back until the next push or reconcile.
  • Lost provenance. Partitions with a missing or partial manifest keep stale extras.
  • The 16 already-pruned nodes are not restored.

Recycle after deploy

Hosting and AI NodeTypes pick up new sources through the import. No manual recycle is needed for this change.

Pairs-with: none — no public type or member is removed; additions only (UpdateToPushedCommitFromGitHub, GitHubSyncService.GetBranchHead, GitHubWebhookProcessor.ReconcileBranches).
Implementers: none — no interface member added.
Mirror-sync: none — the two new keys activity.gitsync.updateToPushedCommit.{title,fetching} are server-rendered activity lines; the React mirror sync (npm run sync:i18n -- --ref <merged core sha>) can ride the next routine sync.

🤖 Generated with Claude Code

…import prunes only what it put there

Policy sources-sync-on-push: the GitHub push webhook imports every sync source of
that repository + branch AT the pushed commit (`after`), with each module judged
alone by ModuleSyncDecision (unchanged => nothing, changed => synced, floor above the
running platform => that module alone declined). A green workflow_run now only records
BuildCompletion; it no longer imports (a green build of an older commit finishing after
a newer push could move a Space backwards). GitSyncBranchReconcileService resolves each
configured branch head every GitSync:BranchReconcileMinutes (default 10) and imports at
that sha, so a lost delivery self-repairs.

Measured on the control instance 2026-10-04/05: Plugins main was red from 14:39Z, no
green build arrived, AI/_GitSync and Hosting/_GitSync froze 15 h; a manual update
imported at once with 0 compile errors.

Policy prune-requires-provenance: ComputePrunableNodes prunes a node only when it is in
the prior import manifest, in every pruning mode. FullReplace used to prune every extra,
and the same manual import pruned 16 runtime nodes (Hosting/Babysitter, Hosting/Queues/*).
Unknown provenance is not prunable.

Doc: Doc/Architecture/SourcesSyncOnPush; both policies registered in PolicyNotProse.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@rbuergi
rbuergi enabled auto-merge October 5, 2026 05:57
@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Test Results

    18 files      18 suites   42m 17s ⏱️
10 330 tests 10 330 ✅ 0 💤 0 ❌
10 343 runs  10 343 ✅ 0 💤 0 ❌

Results for commit 4d3bf45.

♻️ This comment has been updated with latest results.

@rbuergi
rbuergi disabled auto-merge October 5, 2026 06:16
@rbuergi
rbuergi enabled auto-merge October 5, 2026 06:18
@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 0)

  4 files    4 suites   3m 50s ⏱️
405 tests 405 ✅ 0 💤 0 ❌
409 runs  409 ✅ 0 💤 0 ❌

Results for commit 4d3bf45.

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 2)

1 862 tests   1 862 ✅  5m 17s ⏱️
    3 suites      0 💤
    3 files        0 ❌

Results for commit 4d3bf45.

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 1)

    3 files      3 suites   6m 32s ⏱️
2 923 tests 2 923 ✅ 0 💤 0 ❌
2 927 runs  2 927 ✅ 0 💤 0 ❌

Results for commit 4d3bf45.

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 3)

1 629 tests   1 629 ✅  6m 33s ⏱️
    2 suites      0 💤
    2 files        0 ❌

Results for commit 4d3bf45.

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 5)

    2 files      2 suites   8m 43s ⏱️
1 100 tests 1 100 ✅ 0 💤 0 ❌
1 101 runs  1 101 ✅ 0 💤 0 ❌

Results for commit 4d3bf45.

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 4)

    4 files      4 suites   11m 20s ⏱️
2 411 tests 2 411 ✅ 0 💤 0 ❌
2 415 runs  2 415 ✅ 0 💤 0 ❌

Results for commit 4d3bf45.

@rbuergi
rbuergi merged commit c618b5c into main Oct 5, 2026
meshweaver-cloud Bot added a commit that referenced this pull request Oct 5, 2026
test(ladder): rows F1–F4 — push imports, red build holds nothing, one incompatible module declined alone, runtime nodes survive (stacked on #6122)
@rbuergi
rbuergi deleted the fix/gitsync-on-push branch October 10, 2026 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant