fix(self-update): establishing the policy is not the same as re-establishing it

[rbuergi]

The incident. memex-cloud served memex.meshweaver.cloud from memex-portal-ai:3.0.0-rc8.ci.6829 while ACR had reached rc9.ci.7231 — roughly 400 builds behind. Its pinned module set had advanced past the image, so modules reported "CONTRIBUTING NOTHING", new pods failed required_modules at the startup probe and crash-looped (140 restarts), memory sat at 89%, and other repos' CI gates failed against its bundle endpoints. Systemorph/Memex#154. The environment is rolled and healthy again; this is the reason it got there.

The defect. Every self-update trigger — startup, build completion, policy change, and the safety net that exists to bound a dead event channel — is gated on the first emission of the live Admin/UpdatePolicy read. That gate is correct and deliberate (#2731/#2797 removed the synthetic default emission because deciding under a guessed policy rolled pinned installs on every pod start).

The cadence behind it was wrong. A faulted first read was paced by RetryInterval — six hours, documented for re-establishing a watch that has already produced. Before the first read nothing is retained and no check can run, so that value does not pace anything; it removes the feature, and the safety net cannot bound it because the safety net sits behind the same gate. On a heavy instance whose boot-time SubscribeRequest exceeds its minute, that is six hours of an install that looks healthy and has silently stopped asking.

Measured, not inferred. Two pods of the same build, same instance:

pod	policy read	checks run
…-q5zcm	policy stream faulted; re-establishing in 06:00:00	0
…-7sqbk	clean	2

The fix. Two states, two cadences, chosen in one place (SelfUpdateOptions.PolicyRetryDelay): PolicyEstablishRetryInterval (30 s) while no policy has ever been read; RetryInterval once one has — where the value is retained, checks continue, and a fault costs freshness rather than the feature. The warning now names which state it is in.

🚨 Why the existing test could not catch this: it set RetryInterval itself to 500 ms, so it proved recovery happens while saying nothing about recovery happening usefully — a guard whose knob hides the outcome it is meant to observe. It now sets the establishing interval and leaves RetryInterval at its production default; a second test pins the same fault shape with the real six-hour value, so only the fix can make it pass.

Mutation-proven: PolicyRetryDelay reverted to RetryInterval behind a runtime-false condition fails 3 tests (both new, plus the corrected pre-existing one); restored, 31/31. Release -warnaserror clean on MeshWeaver.Hosting and MeshWeaver.Hosting.Monolith.Test.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR fixes a self-update resilience gap where a faulted first Admin/UpdatePolicy read could effectively disable all self-update checks for the full RetryInterval (6 hours by default), by introducing a distinct, faster retry cadence specifically for the “policy establishing” state.

Changes:

• Added SelfUpdateOptions.PolicyEstablishRetryInterval plus a single decision point (PolicyRetryDelay) to select the correct retry delay based on whether a policy has ever been successfully read.
• Updated SelfUpdateHostedService to track when the first policy value has been read and to log/pace retries differently before vs. after establishment.
• Strengthened tests to cover production cadence behavior (6h retry) and to pin the retry-delay decision logic, and added a “What’s New” entry for the user-visible impact.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
test/MeshWeaver.Hosting.Monolith.Test/SelfUpdatePollerResilienceTest.cs	Updates the first-read resilience test to avoid masking the defect and adds a production-cadence regression test plus a pure unit test for the delay decision.
src/MeshWeaver.Hosting/SelfUpdate/SelfUpdateOptions.cs	Introduces PolicyEstablishRetryInterval and PolicyRetryDelay(...) to centralize the two-cadence behavior.
src/MeshWeaver.Documentation/Data/WhatsNew/2026-09-01-an-install-that-cannot-read-its-update-policy-no-longer-stops-updating.md	Documents the fix as a user-noticeable reliability improvement, including the new configuration knob.
memex/Memex.Portal.Shared/SelfUpdate/SelfUpdateHostedService.cs	Tracks “policy established” state and uses it to choose retry delay and improve warning clarity during faults.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

Test Results (shard 0)

793 tests   789 ✅  3m 20s ⏱️
  7 suites    4 💤
  7 files      0 ❌

Results for commit 792acf0.

[github-actions]

Test Results (shard 1)

1 138 tests   1 138 ✅  3m 53s ⏱️
    7 suites      0 💤
    7 files        0 ❌

Results for commit 792acf0.

[github-actions]

Test Results (shard 5)

    8 files      8 suites   4m 13s ⏱️
1 493 tests 1 492 ✅ 1 💤 0 ❌
2 196 runs  2 195 ✅ 1 💤 0 ❌

Results for commit 792acf0.

[github-actions]

Test Results (shard 4)

1 165 tests   1 165 ✅  4m 35s ⏱️
    7 suites      0 💤
    7 files        0 ❌

Results for commit 792acf0.

[github-actions]

Test Results (shard 3)

1 152 tests   1 152 ✅  7m 59s ⏱️
    7 suites      0 💤
    7 files        0 ❌

Results for commit 792acf0.

[github-actions]

Test Results (shard 2)

3 059 tests   2 867 ✅  6m 31s ⏱️
    8 suites    192 💤
    8 files        0 ❌

Results for commit 792acf0.

[github-actions]

Test Results

   44 files     44 suites   30m 33s ⏱️
8 800 tests 8 603 ✅ 197 💤 0 ❌
9 503 runs  9 306 ✅ 197 💤 0 ❌

Results for commit 792acf0.