GitSync: a source that never converges re-clones its WHOLE repository on every green build — lastSyncCommitSha is both the cheapness gate and the convergence guard #3945

Summary

GitHubWebhookProcessor.SkipReason makes a green-build delivery free for exactly one reason —
lastSyncCommitSha == headSha — and GitHubSyncService.MayAdvanceBaseline deliberately holds
that field whenever an import did not fully converge (Preserved > 0, Failed > 0, or outcome
Failed). Both behaviours are individually correct and each has an incident behind it (#675/#677
for the horizon, #2229 item C for the sha). Together they mean:

A sync source that cannot converge pays a full git fetch --depth 1 of its entire repository,
plus a full parse, on EVERY publish-signal delivery — and the rate is the source repository's CI
cadence, not anything about the source.

The allow-list that admits schedule and repository_dispatch states the opposite as its
justification, in PublishSignalTriggers:

Widening this cannot cause churn. A sync source already sitting on the built sha is skipped by
SkipReason ("already at this commit"), so a scheduled or dispatched re-verification of an
unchanged default branch triggers no import at all.

That invariant holds for every source that converges and for no source that does not — and the
sources that do not converge are precisely the ones whose imports are most expensive.

Measured

Trigger rate — Systemorph/MeshWeaver main, 24 h ending 2026-09-10T17:35Z: 254 green
publish-signal runs (gh api repos/Systemorph/MeshWeaver/actions/runs?branch=main&status=success),
of which 87 Prod synthetic probe (schedule, cron: "*/15 * * * *" — an HTTP probe that builds
nothing), 47 Chart Gate, 47 Hosting Operator, 46 MeshWeaver Build and Test (all push), 19
Continuous Delivery (main), 8 other. Independently confirmed by the node the webhook writes:
Admin/_Build/Systemorph.MeshWeaver on memex.meshweaver.cloud read version 4778 at
2026-09-10T17:21Z, first written 2026-08-06T07:38:37Z — 4,775 deliveries in 35 days, 256 in the
last 24 h (10.7/h). So one merge is ~3 deliveries at the same sha, plus 96/day from a cron whose
head_sha does not move between merges.

Two live frozen sources (all 67 _GitSync nodes on memex.meshweaver.cloud read 2026-09-10
17:30–17:39Z; search returned 67, truncated:false; 67 read, 0 unreadable):

Essentials is on lastSyncCommitSha: 709e7307…, ahead_by: 436, behind_by: 0 (still an
ancestor — nothing force-pushed), with ≥11 unimported files under Essentials/. It is the only one of the 34 sources on that
repository off the shared baseline: its 33 siblings all carry f4570459… with lastSyncedAt
equal to lastSyncAttemptAt. Same repo, same webhook, same schedule — the difference is convergence.

The independent node at Deployments/_GitSync on memex.systemorph.com reads
lastSyncedAt: 2026-08-19T20:06:37Z against lastSyncAttemptAt: 2026-09-10T15:27:00Z — 22 days
of attempts that never reconciled. It is the only source in either set carrying twoWay: true, i.e.
the Preserved > 0 arm.

None of the three freeze conditions clears itself: a preserved node stays preserved until someone
commits it back or discards it; a node refused on a content rule is refused identically on the same
bytes for ever; an import that failed outright fails again until its cause is fixed.

The cost of one non-skipped delivery

GitHubSyncService.FetchAndImport fetches first and diffs second — it has to, because
GetChangedPaths needs snapshot.CommitSha:

repoClient.Fetch(...)          →  git init; git fetch --depth 1; git checkout --detach; read worktree
ParseSnapshot(...)             →  every file materialised as a MeshNode
repoClient.GetChangedPaths(...)→  the diff — only now
StaticRepoImporter.ImportSource(..., changedNodePaths)   ← the diff scopes THIS

So the git-diff scope is a write scope, never a fetch scope, and there is no sparse-checkout or
partial-clone filter: a configured subdirectory is applied while READING the worktree
(GitProtocolRepoClient.Fetch), so a source watching a four-file subdirectory still clones the whole
repository. Essentials has been doing that against MeshWeaver.Plugins on every green Plugins build
since 2026-08-10.

What #3146 already closes, and what it does not

The content-addressed marker turns a second pass at an unchanged fingerprint into Skipped, which
carries Failed == 0, so the baseline finally advances and the webhook gate re-arms — "every
delivery" becomes "twice per new commit". Two limits:

• It engages after the fetch. The marker id is the fingerprint of the parsed source, so the
  clone and the parse are already paid. It saves the write fan-out and the recompiles, not the
  transfer.
• It needs the pass to be ALL-deterministic. IsContentVerdict is a deliberately short
  allow-list (InvalidPath, InvalidNodeType, ValidationFailed, Unauthorized); everything else
  stays retryable because getting that backwards is A Space whose content sync is refused stops syncing SILENTLY — AgenticEngineering's assets are over the frame limit outright #3101. One Unknown among four hundred content
  verdicts keeps the outcome ImportedWithErrors, keeps the marker Warning, and keeps the
  partition re-importing on every delivery.
• It does nothing at all for the Preserved > 0 arm, which is the Essentials and Deployments
  case.

Proposed fix (needs a decision — deliberately not implemented here)

The engine has no way to say "we have already looked at exactly these bytes for this source"
independently of "the mesh is known to hold this commit". One field answers both questions and the
two answers must diverge.

Sketch: record the commit the last attempt targeted — a new LastAttemptedCommitSha, stamped on
every conclusion beside the existing LastSyncAttemptAt / LastSyncOutcome, which nothing branches
on today — and add a fifth SkipReason: the last attempt for this source already reached a verdict
at this exact commit. LastSyncCommitSha and LastSyncedAt keep their present, conservative
semantics, so #675 / #677 / #2229 item C are untouched: the next NEW commit still triggers a full
unscoped import and the preserved/failed nodes get another chance then.

The regression this must not cause, and why it needs a decision rather than a patch: today a
genuinely transient failure (Unknown — "Persistence read failed…", "Inner CreateNode faulted…") is
retried by the next delivery, which on core is minutes away. Under the sketch it would wait for the
next commit. That accidental retry loop is load-bearing for exactly the failures the classifier
refuses to call final, so the change wants either a bounded re-attempt of its own or a carve-out for
non-deterministic outcomes. Both are design choices about the sync engine of a live fleet, which is
why this is filed rather than shipped.

Documentation

The mechanism, the measurements above and the read-only check for a frozen source are written up in
Doc/Architecture/GitSyncTriggerCost — What a Green Build Costs a Synced Space.

Related

Systemorph/Memex#237 (the live worst case: a MeshWeaver/_GitSync on memex.meshweaver.cloud
pointed at https://github.com/Systemorph/MeshWeaver with no subdirectory, 389 permanently-refused
nodes per pass) · #675 · #677 · #2229 · #3101 · #3146 · #3581 · #3589