Skip to content

cr.meshweaver.cloud: the rule is declared (nothing-deletes, cleanup unauthorized) — read the storage account's lifecycle policy and decide whether unbounded growth is accepted #6499

Open

Description

@rbuergi

(re-filed 2026-10-11; replaces a deleted issue, formerly number 4230; originally by @rbuergi on 2026-09-13)

Split out of #3438 at the coordinator's request. #3438 and everything under it — the lock lane,
the tag lock, the index closure, the 30-day window, #3859's interlock — are about one registry:
meshweaver.azurecr.io. This is the same question asked about the second one.

What is true today

cr.meshweaver.cloud is the fleet's own container registry, hosted by MeshWeaver as part of the
memex-cloud release (#3697 — registry.host: cr.meshweaver.cloud, serving from
ghcr.io/distribution/distribution). It is the default for newly provisioned instances (the
new-deployment path pulls from it, not from ACR).

acr purge cannot reach it, and no retention rule we have is about it. #3842's policy (at least
30 days by age, no build-count quota) is implemented in three places — PrebuiltBundleRetention,
AssemblyCacheRetention, and the ACR task record — and all three are about the portal's own stores
or about the ACR. Nothing states what keeps, or deletes, an image in cr.meshweaver.cloud.

Measured 2026-09-13 (read-only)

Every deployment overlay in Systemorph/Memex (twelve files), run through this repository's own
shipped extractors on the committed bytes:

installation pins in meshweaver.azurecr.io pins in another registry
build (build.meshweaver.cloud, LIVE, answers /api/version with core c84c6c0) 0 2 → cr.meshweaver.cloud (memex-portal-ai:3.0.0-ci.8411, memex-migration:3.0.0-ci.8411)
client-b (declared not-installed, no DNS record) 0 2 → cr.meshweaver.cloud (…:3.0.0-ci.8080)
memex 6 0
memex-cloud 3 1 → ghcr.io (distribution/distribution — the registry service's own image, third-party)

So exactly one live installation runs entirely from cr.meshweaver.cloud today, and it is the
build server. client-b will be the second the day it is provisioned, and every instance provisioned
after it.

This surfaced because the nightly protection lane refused on it: AXIS 3 — installation 'build' runs core c84c6c0 and its overlay pins no image at all (run
34730206406). That sentence was
false — the overlay pins two images — and #4221 fixes the reading. What #4221 does not do, and
says so in as many words, is claim that anything protects them.

What is unknown

  1. Does anything delete from cr.meshweaver.cloud? A timer, a registry-side GC, a
    distribution garbage-collect job, a volume that simply fills — nobody has looked. Both
    readings are dangerous in different ways: unbounded growth, or a deletion nobody can see.
  2. If something does, what is protected from it? There is no lock concept, no pin scan and no
    closure walk on that registry. The manifest lock this fleet relies on
    (changeableAttributes.deleteEnabled) is an ACR feature.
  3. What is the storage bound? ACR's is measured (2,936 manifests on 2026-09-13). This one is
    not.

What would close it

Any of these, stated rather than assumed:

The current state is recorded, not papered over: .github/acr-retention/instances.json (#4221)
declares cr.meshweaver.cloud as fleet-unlockable with a reason that says what retains it is
established by nothing
, and the protection run prints its out-of-scope installations on their own
line reading "NOT locked here, and whether anything retains that registry is UNVERIFIED". That
makes the gap visible; it does not close it.

Not this issue

Design context: Doc/Architecture/ArtifactRetentionInterlock → A second registry.

Measured 2026-09-13 (date -u), read-only throughout: the twelve committed overlays, the run
log over REST, and az role definition list. No purge, no lock, no dry run, no registry mutation of
any kind, on either registry.

Activity

  1. added
    choreCI, build, dependencies, cleanup - no user-visible behaviour change
    area:hostingCore subsystem: hosting - deployments, rolls, the control plane
    on Oct 11, 2026
  2. rbuergi commented on Oct 11, 2026

    @rbuergi
    ContributorAuthor

    (re-posted 2026-10-11; originally by @rbuergi on 2026-09-14)

    Answered, and the answer is the SECOND of the two readings this issue names — #4315

    "Both readings are dangerous in different ways: unbounded growth, or a deletion nobody can see."

    It is unbounded growth. Nothing deletes a manifest, a tag or a referenced blob from
    cr.meshweaver.cloud, and nothing collects its garbage. That is a cost rather than an incident, and
    it is the safe direction to be wrong in — the hazard is the fix, which is what the rest of this
    is about.

    Design, committed: Doc/Architecture/FleetRegistryRetention.

    1. The enumeration — measured 2026-09-14 (date -u), core origin/main 192bb073b8

    Read-only throughout: no registry call of any kind, no purge, no dry run, no deletion. 409 files
    under deploy/ and .github/ swept, and the sweep pattern was proven to fire on a synthetic
    control first
    — a grep that matches nothing is not evidence until it has been shown able to match
    something.

    mechanism present? verdict
    maintenance.uploadpurging YES — enabled (age: 168h, interval: 24h, dryrun: false) the one automatic deletion here, and it reaches incomplete upload sessions only — never a manifest, a tag or a referenced blob
    registry garbage-collect (blob GC) no no Job, no CronJob (8 objects rendered under the registry chart, none a Job) and no invocation anywhere. Blobs are never collected
    an explicit registry DELETE no zero callers. CD only ever pushes here
    a non-publisher account deleting no by the ACL, not by a grep: docker_auth grants delete to exactly one account, the publisher. An installation holding an instance key cannot delete whatever it asks
    acr purge no cannot reach this host — the fact that makes this issue separate from #3438
    an Azure blob lifecycle policy on the storage container UNVERIFIED 🚨 the one row that is not a measurement — see §4

    This answers the issue's questions 1 and 3 (question 3, the storage bound, is measured by
    nobody and stays open — §4).

    2. 🚨 The finding, and it is why "port the ACR lane one registry along" is wrong

    distribution has NO LOCK. changeableAttributes.deleteEnabled is an ACR feature, so there is
    no object a protected set can be written into ahead of a deleter. On the ACR the protected set is
    persistent registry state, written by a different process on a different clock — an incomplete
    nightly run writes fewer locks and last night's still hold. Here the derivation is the entire
    safety margin, with nothing behind it: an incomplete run deletes what it could not see, in the same
    act.

    So the rule (R0–R6 on the page) is deliberately stricter than #3438's, not equivalent. The two
    clauses that matter: an incomplete derivation deletes nothing (not less), and the protected set
    is derived — age only ever narrows the complement, never defines it, which is #3438's own root
    cause stated as a rule instead of a window.

    3. What landed

    This closes the issue's third bullet — "a retention policy for cr.meshweaver.cloud expressed the
    way #3842's is, plus the protection that makes it safe"
    — with the protection expressed as the one
    form a lock-less registry admits.

    • The declaration. Every registries entry in .github/acr-retention/instances.json now
      answers the second question as well as the first, so a registry joining the fleet cannot enter
      undeclared. nothing-deletes requires a deleters enumeration with at least one
      present: true
      — an enumeration in which nothing is present inspected nothing, and reads
      exactly like a clean one.
    • The gate. lock-pinned-digests.py --check-registry-retention, credential-free, on every pull
      request (in dotnet-test.yml's workflow-shell job, a needs: of Consolidate test results).
      🚨 The arm that is not about JSON: it re-reads the committed chart — a Job/CronJob beside
      the registry, a moved maintenance: window, a new key in that stanza, an ACL rule granting
      delete to anyone but the publisher, or an executable deletion anywhere, each goes RED while the
      record still reads "nothing deletes"
      . A reason field alone would read identically on that day.
    • The dry run, and its output is the deliverable. The nightly protection run already extracted
      every committed reference to this registry and then discarded them. It now prints them —
      repository, tag, and the file that pins each — plus its own incompleteness on its own line:
      committed pins are a floor, the plugin-bundle family is not derivable at all today, and no
      cleanup may run against it.

    Nothing was deleted and no cleanup that runs was written. R0 is that nothing deletes and
    nothing may
    , until a protected set can be derived over what the registry holds.

    4. What stays open, and it is three reads rather than a design

    1. The blob-lifecycle question — the UNVERIFIED row. The storage account is
      registry.storage.accountName, provisioned outside this chart, so no committed file can answer
      it. 🚨 A lifecycle rule over a content-addressed store is not old data expiring — it deletes a
      layer a live manifest still names, and the image reads as present until something pulls it.
      In the record this is present: "unverified" with a verifiedBy, it prints on every run, and it
      keeps cleanupAuthorized false by mechanism.
    2. Whether unbounded growth is accepted, with a storage bound measured the way the ACR's is
      (2,936 manifests on 2026-09-13); this one has never been measured — the issue's question 3.
    3. Whether to build the last-pulled receiver. The registry is already configured to POST every
      push/pull/delete to registry.notifications.url; it defaults to "" and no receiver exists.
      That is the difference between a cleanup that can ever be safe and one that cannot. Filed against
      An image's closure is still not queryable mesh data — /app is asserted by a shell script (the unwired mirror, item 1, was deleted in #4573) #4066, where the rest of "an image's closure as mesh data" lives.

    Leaving this open for those three. The question it was filed on — what is established about
    retention there
    — is answered and committed.

    Measured 2026-09-14 (date -u), read-only: the committed chart, the committed extractors and
    the task record. No purge, no lock, no dry run, no registry mutation of any kind, on either
    registry.

    🤖 Generated with Claude Code

  3. rbuergi commented on Oct 11, 2026

    @rbuergi
    ContributorAuthor

    (re-posted 2026-10-11; originally by @rbuergi on 2026-09-14)

    🚨 The premise of this issue is REVERSED, and the title and body now describe a defect that does not exist

    Landed in #4315 (e73047d3233cc4816fd15735528a4766feb8d8d5). Before anything else, the thing a future reader must not be misled by:

    This issue asks which of two dangers is real — "unbounded growth, or a deletion nobody can
    see"
    . It is the FIRST. Nothing deletes from cr.meshweaver.cloud.

    So the title — "Retention for cr.meshweaver.cloud is established by nothing" — was right about the
    gap in the record and wrong about the shape of the hazard it implied. The body's framing
    (images at risk, an acr purge equivalent needed, protection to be built before something eats a
    pin) describes a defect that is not there. What is there is a registry that keeps every image and
    every bundle ever pushed to it, plus every blob, for ever, and has never collected any of it.

    That reversal is not a technicality: it changes what is urgent. Nothing is being lost. The hazard
    is the FIX
    — the day somebody adds a cleanup, there is nothing to stop it deleting what a live
    portal still runs, because (see below) this registry has no lock to protect anything with.

    What was measured

    2026-09-14 (date -u), core origin/main 192bb073b8, read-only throughout: no registry call of
    any kind
    , no purge, no dry run, no deletion.

    mechanism present? verdict
    maintenance.uploadpurging YES — enabled (168 h / 24 h, dryrun: false) the one automatic deletion, and it reaches incomplete upload sessions only — never a manifest, a tag or a referenced blob
    registry garbage-collect no no Job, no CronJob (8 objects rendered under the registry chart, none a Job), no invocation anywhere. Blobs are never collected
    an explicit registry DELETE no zero callers. CD only ever pushes here
    a non-publisher account deleting no by the ACL, not by a grep — and measured through the gate's own reader: 6 ACL rules, exactly 1 grants delete, and its match names the publisher
    acr purge no cannot reach this host — the fact that made this a separate issue from #3438
    an Azure blob lifecycle policy 🚨 "unverified" — a state of its own in the record, not a false the storage account is provisioned outside this chart, so no committed file can answer it

    Two denominators, stated as two: the hand sweep that produced this table read all 409 files
    under deploy/ and .github/; the gate sweeps the 246 that can actually run a command and names
    the 3 it skips. Both ran with the deleter pattern proven to fire on a synthetic control first —
    a grep that matches nothing is not evidence until it has been shown able to match something.

    One row's provenance is different and is flagged as such: what uploadpurging deletes is
    distribution's documented behaviour, not something measured here. The chart pins the image by
    digest, so the version that statement is true of is pinned beside it.

    The finding: this registry has NO LOCK, so the rule is stricter than #3438's

    changeableAttributes.deleteEnabled is an ACR feature. There is no object here a protected set
    can be written into ahead of a deleter — so lock-pinned-digests.py does not port one registry
    along, and the difference is structural rather than an implementation gap:

    ACR fleet registry
    what a protected set IS persistent registry state, written ahead of the deleter, on a different clock a list in the memory of the process about to delete
    an INCOMPLETE derivation writes fewer locks; everything locked earlier stays locked deletes what it could not see, in the same act
    a lane outage degrades gracefully — the lane was red two days in September and nothing already locked was at risk there is nothing to stand

    Hence R0–R6 on the page, and the two clauses the gate enforces rather than describes: an incomplete
    derivation deletes NOTHING
    (not less), and the protected set is derived — age only ever narrows
    the complement, never defines it, which is #3438's own root cause stated as a rule instead of a
    window.

    What CLOSES with this

    • This issue's third acceptance bullet — "a retention policy for cr.meshweaver.cloud
      expressed the way No platform pins in module repos — build against the released platform, and be TOLERANT of drift (same major serves) #3842's is, plus the protection that makes it safe"
      — in the only form a
      lock-less registry admits: a declared rule plus the derivation of what it protects, not a
      cleanup with a longer window. 🚨 Nothing was deleted and no cleanup that runs was written. R0
      is that nothing deletes and nothing may, until a protected set can be derived over what the
      registry holds.
    • Question 1 ("does anything delete?") — answered, enumerated, and now held by a gate that
      re-reads the chart rather than the claim
      : a Job/CronJob beside the registry, a moved
      maintenance: window, a new key in that stanza, an ACL rule granting delete to anyone but the
      declared principal
      , or an executable deletion anywhere each goes RED while the record still
      reads "nothing deletes"
      . Credential-free, on every pull request. Seven sabotages, each firing
      exactly its own arms.
    • Question 2 ("if something does, what is protected?") — the protected set is now derived and
      printed
      instead of discarded: the nightly run lists every committed reference to this registry
      (repository, tag, and the file that pins it) with its own incompleteness on its own line.
    • Retention: the locks hold nightly — apply the decided 30-day window to the paused purge-old-images, read production back, and inventory bundles/active builds as consumers #3438's open item 3 ("NEW: what retains cr.meshweaver.cloud") — closed there and moved
      here; that thread keeps its ACR items unchanged.
    • The instances.json entry no longer says "established by nothing" — but the lane's
      out-of-scope report line still says UNVERIFIED, and must: this lane still protects nothing
      there.

    What stays OPEN, and why this issue is not closed

    Question 3 — the storage bound — is still measured by nobody. The ACR's is (2,936 manifests on
    2026-09-13); this one has never been. Since the answer above is unbounded growth, that number is
    now the whole of the remaining risk, and it is not one I can take: it needs a read on the storage
    account.

    Design, committed: Doc/Architecture/FleetRegistryRetention.
    Related: #3438 (the ACR side, unchanged), #4066 (what is missing to derive the protected set from
    mesh data — stated precisely there, including that ContainerImageRecord cannot supply the registry
    inventory whichever way #4066's item 1 is decided).


    🧑 Needs the maintainer — the only part of this that needs a human

    1. Read the registry's storage account for a blob lifecycle / management policy. The one
      "unverified" row. 🚨 A lifecycle rule over a content-addressed store is not old data
      expiring — it deletes a layer a live manifest still names, and the image reads as present until
      something pulls it. The account is registry.storage.accountName; nothing committed can answer
      it. Until it is answered the record keeps cleanupAuthorized: false, by mechanism.
    2. Decide whether unbounded growth is accepted, and measure the storage bound plus a threshold
      that would change the answer — this issue's own question 3, and now its only live risk.
    3. Decide whether to build the last-pulled receiver. The registry is already configured to
      POST every push / pull / delete to registry.notifications.url; it defaults to "" and no
      receiver exists. It is the difference between a cleanup that can ever be safe and one that
      cannot.
    4. An image's closure is still not queryable mesh data — /app is asserted by a shell script (the unwired mirror, item 1, was deleted in #4573) #4066 item 1 — map MapContainerImages in a host, or delete src/MeshWeaver.ContainerImages.
      Nothing here depends on the answer; the finding on that thread removes a reason to wire it, not a
      reason to delete it.
    5. Retention: the locks hold nightly — apply the decided 30-day window to the paused purge-old-images, read production back, and inventory bundles/active builds as consumers #3438's remaining ACR items — Retention: interlock cleanup with protection updates and active publication #3859's coordination mechanism (an RBAC grant on the shared
      registry and the disabling of a destructive schedule) and applying the decided window to the
      disabled task, which is the same act as lifting the pause.

    Measured 2026-09-14 (date -u), read-only: the committed chart, the committed extractors and
    the task record. No purge, no lock, no dry run, no registry mutation of any kind, on either
    registry. No cluster access and no token value read, printed or stored.

    🤖 Generated with Claude Code

  4. rbuergi commented on Oct 11, 2026

    @rbuergi
    ContributorAuthor

    (re-posted 2026-10-11; originally by @rbuergi on 2026-09-15)

    The one "unverified" row is now a single command — the storage account is named in committed sources

    Read-only, origin/main b8da52e054, 2026-09-15. Nothing here changes the verdict: the rule is declared and gated (#4315 e73047d323 → .github/acr-retention/instances.json registries["cr.meshweaver.cloud"].retention = rule: "nothing-deletes", cleanupAuthorized: false, five deleters rows, enforced credential-free by lock-pinned-digests.py --check-registry-retention at dotnet-test.yml:2287), and cleanupAuthorized: false is held by mechanism while any "unverified" row stands.

    What this adds: the record and the design page both decline to name the storage account, so maintainer item 1 reads as a hunt. It is not — the value is committed in two places. deploy/helm/values.registry.example.yaml:17-19:

      storage:
        accountName: "meshweaverregistry"
        container: "registry"

    and the same pair in Systemorph/Memex mesh/Deployments/memex-cloud.json (content.registry.storageAccountName / storageContainer). So the blob-lifecycle row — the one deleters entry sitting at present: "unverified" — is answered by:

    az storage account management-policy show --account-name meshweaverregistry -g <rg>
    

    plus the container's own lifecycle rules. That is the instrument, and it is the only thing standing between the record and a fully verified nothing-deletes claim. The same account answers the issue's question 3 (the storage bound), which has never been measured: blob-service metrics or az storage blob list on meshweaverregistry/registry, or a /v2/_catalog + per-repo tag walk as the publisher. Both are outside every repo, which is why this stays open.

    Two commits landed after the last comment here and touch the same table: 76dbb637da fix(#4323) and d081c3f619 fix(#4324 review) (2026-09-14T14:11–14:17Z). They added the third question (publishes) to the registries record and corrected a false ghcr.io declaration that had read "Nothing this fleet produces is stored on ghcr.io" while main-cd.yml pushes twelve tags there per promoting run — worth knowing, since it is the same file this issue's record lives in and the same class of error the "unverified" row is guarding against.

    Question 2 (whether to build the last-pulled receiver) is unchanged and filed against #4066: registry.notifications.url defaults "" (deploy/helm/values.yaml:840) and _registry.tpl:49-52 requires a vault secret object the moment a URL is set.

  5. rbuergi commented on Oct 11, 2026

    @rbuergi
    ContributorAuthor

    (re-posted 2026-10-11; originally by @rbuergi on 2026-09-17)

    2026-09-17 re-measurement — nothing has drifted, and one maintainer item closed elsewhere

    Read-only on core main 0ec37cf2fc:

    • The record stands exactly as feat(#4230): the fleet's own registry says what DELETES from it — a declared rule, a derived protected set, and a gate that re-reads the chart #4315 (e73047d3233cc4816fd15735528a4766feb8d8d5) left it:
      .github/acr-retention/instances.json:200-202 → rule: "nothing-deletes",
      cleanupAuthorized: false, statedAt: "2026-09-14"; :239 → the blob-lifecycle row is still
      "present": "unverified". Last touch to that directory was 193e54ed4f (09-15), so no drift.
    • The gate is still wired and green. dotnet-test.yml:2367 runs
      lock-pinned-digests.py --check-registry-retention, with the record arm at :2349 and nightly at
      lock-pinned-digests.yml:291. Run here credential-free, EXIT=0: "3 registry(ies) declared …
      262 executable file(s) swept for a deleter … cr.meshweaver.cloud: 1 mechanism(s) UNVERIFIED — an
      Azure blob lifecycle / management policy on the registry's storage container; no cleanup may be
      authorized here while that stands"
      .
    • Maintainer item 4 is DONE: edeefa808a83d9839eb50cd891ec9847e8e7d751 (2026-09-17 09:50) deleted
      src/MeshWeaver.ContainerImages and its chart block. §7 of Doc/Architecture/FleetRegistryRetention
      records it struck through; a fifth item has appeared at §8.5 (the chart's default
      ghcr.io/systemorph/…:latest images).
    • Question 3 is still unmeasurable from outside. https://cr.meshweaver.cloud/v2/ and
      /v2/_catalog both answer 401 today — publisher-credential-only. The bound needs
      az storage account management-policy show on the registry's storage account, or a /v2/_catalog
      walk as the publisher.

    🚨 The title still states the reversed premise. This issue asked which of two dangers was real —
    "unbounded growth, or a deletion nobody can see" — and the answer, settled on 2026-09-14, is the
    first: nothing deletes there, the hazard is the fix, and cleanupAuthorized: false holds it by
    mechanism rather than by memory. Retitling to what is actually left, which is two decisions and one
    az read, not a defect
    . The durable record is
    Doc/Architecture/FleetRegistryRetention
    (584 lines, R0–R6 at :92-182, the maintainer's items at :344), not this thread.

  6. rbuergi commented on Oct 11, 2026

    @rbuergi
    ContributorAuthor

    (re-posted 2026-10-11; originally by @rbuergi on 2026-09-18)

    Triage 2026-09-18 — still open, and the body's "exactly one live installation" is now TWO

    Read-only: core origin/main b76f57a79d, Systemorph/Memex@main over REST, and two unauthenticated /api/version GETs. No registry call of any kind, no purge, no dry run, no mutation.

    The open item has not moved

    .github/acr-retention/instances.json on main today is exactly as #4315 left it:

    • registries["cr.meshweaver.cloud"].retention → "rule": "nothing-deletes", "cleanupAuthorized": false, "statedAt": "2026-09-14"
    • the blob-lifecycle deleter row → "present": "unverified", verifiedBy naming the one az storage account management-policy show read on meshweaverregistry

    So the single remaining instrument is still the maintainer's one Azure read, and the storage bound (question 3) is still unmeasured. Nothing to adjudicate into a close.

    🚨 What changed: client-b is live, so the second fleet-registry-only installation exists now

    The body's table says "exactly one live installation runs entirely from cr.meshweaver.cloud today, and it is the build server. client-b will be the second the day it is provisioned", and lists client-b as "declared not-installed, no DNS record".

    That day was 2026-09-16. Measured 2026-09-18 06:14Z:

    curl -s --max-time 15 https://client-b.meshweaver.cloud/api/version
      → {"commit":"67cbbe0ee4467ac30f086587c97f62c7b0d13689"}     http 200
    curl -s --max-time 15 https://build.meshweaver.cloud/api/version
      → {"commit":"c84c6c05503228860df03c4a8b596e497e6d218c"}     http 200
    

    Systemorph/Memex@main mesh/Deployments/client-b.json: status: "Live" (f86d9cc78, 2026-09-16 — "provisioned end to end through the Hosting plugin"), updatePolicy: "Continuous" (bdf675ddf, 2026-09-17), pinnedImageTag: "3.0.0-ci.8080", host: "client-b.meshweaver.cloud" — and its overlay's two pins are in cr.meshweaver.cloud, per the body's own table.

    Why it bears on this issue rather than being bookkeeping. The registry where nothing deletes, nothing may, and nothing can be locked now backs two serving portals instead of one, one of them on Continuous. That does not change the verdict (unbounded growth, the hazard is the fix) — it raises the cost of getting the eventual cleanup wrong, and it moves client-b from "a pin to protect" to "a running portal whose bytes must not be collected".

    It also leaves a stale exemption upstream: .github/acr-retention/instances.json:306-310 still declares client-b "state": "not-installed" on the reason "client-b.meshweaver.cloud has no DNS record (measured 2026-09-12)", whose own text says "Delete this entry the day it is provisioned." While it stands, the nightly AXIS 3 accounting skips a live installation. Recorded in more detail on #3438, whose lane reads that roster.

    Unchanged and still owed here

    One az storage account management-policy show --account-name meshweaverregistry, plus the storage bound (/v2/_catalog answers 401 to anyone but the publisher). Both outside every repo, which is why this stays open.

    Read-only triage 2026-09-18 by Claude Opus 5 (1M context).

  7. rbuergi commented on Oct 11, 2026

    @rbuergi
    ContributorAuthor

    (re-posted 2026-10-11; originally by @rbuergi on 2026-09-21)

    Triage 2026-09-21: left open, chore, no severity

    Blocked on: reading the storage account's lifecycle policy — which needs Azure access this side of the portal API — and then a decision about whether unbounded growth on cr.meshweaver.cloud is accepted. Both halves are outside what an agent should settle: the first because operations go through the control instance's Hosting API and a storage-account lifecycle policy has no Hosting surface, the second because "accept unbounded growth" is a cost decision.

    The rule as declared — nothing deletes, cleanup unauthorized — is the safe default and is the reason the question is open rather than urgent: an over-retaining registry costs money, an under-retaining one destroys a pinned digest, and this backlog has a live instance of the second (#3438: mw-plugin-test passed Memex#122's "would anything recreate this tag?" test and was purged anyway, wedging MeshWeaver.Education entirely, because --keep 10 is measured in newer builds).

    Two neighbouring findings that bear on it and are deliberately not folded in:

    🚨 And the locking half cannot be automated as things stand: --delete-enabled false needs repositories/metadata/write (Container Registry Repository Writer), held by none of the registry's three service principals — recorded on #3438 as "the last production action" and still outstanding. So a decision to retain by locking has a provisioning prerequisite that is nobody's code change.

  8. rbuergi commented on Oct 11, 2026

    @rbuergi
    ContributorAuthor

    (re-posted 2026-10-11; originally by @rbuergi on 2026-09-22)

    Disposition: HANDED BACK — two acts, neither an agent's, and both named exactly

    Re-measured read-only on core origin/main 94689968ef (2026-09-22). The record is unchanged since #4315 and its last touch (cceb637676) only tightened the check that reads it:

    .github/acr-retention/instances.json → registries["cr.meshweaver.cloud"].retention
      rule: nothing-deletes · cleanupAuthorized: false
      deleters: uploadpurging present=true · blob GC present=false · explicit DELETE present=false
                non-publisher delete present=false · acr purge present=false
                Azure blob lifecycle / management policy on the storage container: present="unverified"
    

    So the verdict recorded on this thread stands — nothing deletes, the hazard is the fix — and cleanupAuthorized: false is held by mechanism (lock-pinned-digests.py --check-registry-retention, wired on dotnet-test.yml and the nightly) while that one row reads unverified. Nothing here is code work, and this thread has been correct about that for eight days. What is left is exactly two acts:

    # the act owner what it waits on
    1 Read the storage account's lifecycle policy once — az storage account management-policy show --account-name meshweaverregistry -g <rg> plus the registry container's own rules — and flip that row to present: true (with the policy quoted) or present: false. Account and container names are committed (deploy/helm/values.registry.example.yaml:17-19; Systemorph/Memex mesh/Deployments/memex-cloud.json content.registry.storageAccountName). whoever holds Azure RBAC on that storage account — it has no Hosting-API surface, so it is not reachable through the control instance and an agent must not reach around it nothing; it is a five-minute read
    2 The cost decision: accept unbounded growth (then the record gains the measured bound and the threshold that would change the answer — question 3, still measured by nobody: /v2/_catalog answers 401 unauthenticated, so it is a publisher-credential walk or a blob-service metric), or authorize a cleanup — which distribution cannot make safe today, because it has no lock and the "last-pulled receiver" that would stand in for one is unbuilt (#4066). the maintainer act 1 — the check refuses to authorize cleanup while any row is unverified, by design

    Two things an agent must not do here, stated so the next triage does not re-derive them: not port the ACR purge lane one registry along (no lock exists to protect a pin from it), and not close this on the strength of the record — the record says the question is open, which is the correct state.

    Left chore, no severity. The durable text is Doc/Architecture/FleetRegistryRetention; this comment only names the two acts and their owners.

  9. rbuergi commented on Oct 11, 2026

    @rbuergi
    ContributorAuthor

    (re-posted 2026-10-11; originally by @rbuergi on 2026-10-09)

    Left open. Needs: (1) a read of the storage account's lifecycle policy, which needs Azure access and has no Hosting surface, and (2) a cost decision: is unbounded growth on cr.meshweaver.cloud accepted? Both are owner acts. No code is owed until (2) is answered.

  10. rbuergi commented on Oct 11, 2026

    @rbuergi
    ContributorAuthor

    (re-posted 2026-10-11; originally by @rbuergi on 2026-10-09)

    Implementing the maintainer's decision (2026-10-09): "cr.meshweaver.cloud: NOT unbounded — a lifecycle rule that never deletes anything a lock/active build references (inventory consumers first, per #3438), applied via the governed infra path (InstanceAction/InfraDeploy what-if first)."

    Recorded as policy fleet-registry-bounded (core #6379; Doc/Architecture/FleetRegistryRetention → The decided rule, §7 item 2 struck): a lifecycle rule whose protected set is DERIVED every run from the consumer inventory and which deletes nothing when any part of it cannot be read; through an InstanceAction, what-if first; never a blob lifecycle policy (it cannot see references). The record keeps rule: nothing-deletes, cleanupAuthorized: false until that cleanup exists — the gate refuses true while the lifecycle row is unverified.

    First governed step, built: the RegistryRetention action's read-only mode (MeshWeaver.Plugins#3251, Memex#712) reads the management policy of every registry storage account the committed records name — the one unverified row — as hosting-operator, through the subscription Reader includeAuditReader already declares. Once those PRs are merged and rolled, RegistryRetention dryRun: true on control answers it (::hosting:: registry_lifecycle=<account> present=true|false); then the row flips in instances.json.

    Not built yet (stated, not hidden): the deleting cleanup itself. Its protected set needs the image-closure inventory as mesh data (#4066, decided separately the same day); building a deleter on a registry with no lock before that inventory exists would make the run's own derivation the whole safety margin. Left open for it.

  11. rbuergi commented on Oct 11, 2026

    @rbuergi
    ContributorAuthor

    (re-posted 2026-10-11; originally by @rbuergi on 2026-10-10)

    Status after the #3438 merges. Maintainer decision (2026-10-09): "#4230 cr.meshweaver.cloud: NOT unbounded — a lifecycle rule that never deletes anything a lock/active build references (inventory consumers first, per #3438), applied via the governed infra path (InstanceAction/InfraDeploy what-if first)."

    Next acts: the maintainer runs the RegistryRetention dry run on control (step 4 on #3438) to read the current lifecycle state. Once #4066's inventory lands, the rule is added to the infra template and applied via InfraDeploy what-if, then deploy.

    Leaving this open.

  12. rbuergi commented on Oct 11, 2026

    @rbuergi
    ContributorAuthor

    The one unverified row is answered: the registry's storage account carries no lifecycle policy. Measured 2026-10-11 through the governed read, not a hand az call.

    • Request Ops/Actions/registry-retention-verify-20261011-3438 on the control instance (RegistryRetention, dryRun: true, unattended), operations-lane run 38141799080.
    • It printed ::hosting:: registry_lifecycle=meshweaverregistry present=false.
    • The lane prints present=false only on Azure's own ManagementPolicyNotFound answer. A read it cannot make is a red run naming the account.

    #6502 flips the record's row from "unverified" to present: false, naming this measurement, and updates Doc/Architecture/FleetRegistryRetention (§1, §6, §7 item 1). cleanupAuthorized stays false. With the row answered, the gate no longer refuses true over an unverified mechanism on this registry; what holds it is policy fleet-registry-bounded.

    What is left on this issue is the cleanup itself, and it is blocked on its inventory:

    Leaving this open for the cleanup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:hostingCore subsystem: hosting - deployments, rolls, the control planechoreCI, build, dependencies, cleanup - no user-visible behaviour changefeature:registry-retentionFeature: registry retention

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions