Chart Drift has never produced a verdict — 39 runs, 39 failures since 2026-08-15; it renders with one of the deploy’s three value sources #4640

Chart Drift (.github/workflows/chart-drift.yml) has run 39 times and failed 39 times — 2026-08-15T06:22Z through 2026-09-17T06:39Z, zero successes, ever. It has never once produced the verdict it exists for.

gh api "repos/Systemorph/MeshWeaver/actions/workflows/334176566/runs?per_page=100" \
  --jq '[.workflow_runs[]|.conclusion]|group_by(.)|map({c:.[0],n:length})'
[{"c":"failure","n":39}]

This is NOT #1709, and #1709 being closed is half the defect

#1709 tracked the first cause — MEMEX_DEPLOY_REPO_TOKEN authenticating as nothing — and was closed 2026-08-26. That cause was fixed: in today's run Assert the inputs exist and Read the environment list both pass. The next failure mode then took over silently, and with the ticket closed nothing was left pointing at it. Twelve more days of red went unread.

The workflow is not at fault: it fails RED exactly as designed. What failed is that a permanently-red check carries no information, and the one artefact that made the state visible was retired.

The current cause: it cannot render, because it only has two of the three value sources

Both environment jobs fail identically, at helm template:

##[error]helm template FAILED — cannot determine what the chart describes:
Error: execution error at (memex/templates/memex-portal/secrets.yaml:101:33):
memex.orleansConnectionString: 'memex_portal' runs AdoNet clustering on an EXTERNAL database
(postgres.enabled is false) but names no database for cluster membership: neither
secrets.memex_portal.ConnectionStrings__orleans nor secrets.memex_portal.ConnectionStrings__memex
is set. … Refusing to render.

That refusal is #3780's guard working correctly. The reason it fires is structural:

• check-chart-drift.sh's own header: "Render with the SAME -f list the deploy uses, or you are diffing against a chart nobody deployed."
• The deploy uses three sources — -f <Key Vault values half> -f <overlay> plus chart defaults (Memex#295).
• Chart Drift is given one: the committed values.<release>.public.yaml, which is deliberately secret-free (chart-drift.yml's own item 3 — publishing it would leak pgbackrest.azure.accountKey).

So the check is rendering with a strict subset of the deploy's -f list, which its own header forbids. Before #3780 it would have rendered something — a chart nobody deployed — and diffed against it. Either way it has never answered its question.

Why this matters right now — measured

Ops/Status/memex, 2026-09-17T18:03:59Z (fleet watch, read-only):

Nothing that is committed is running, and two different uncommitted images are — for ~7 hours at the time of reading. No guard in the fleet reports this:

• Memex/.github/workflows/deploy-drift.yml compares git against the deploy record; its header says plainly it cannot see the cluster, and delegates that to this workflow.
• check-chart-drift.sh does not compare the container image at all — its compared set is the ConfigMap, the pod template's inline env / envFrom / preStop / grace / probes, and the availability shape (PDB, ScaledObject). grep -n image finds only prose.
• The fleet watch defines IssueKinds.ImageDrift and explicitly excludes it from EvaluatedKinds ("needs inputs this sweep does not read").

A render IS possible with no secret — but not yet truthfully

Measured locally, helm v4.2.4 (the version PR Memex#352 measured against), against deploy/helm + the committed values.memex.public.yaml:

A stub containing only fields already committed on the record — vaultValuesKeys for the key names, and databaseServer/database/databaseUsername/host for the connection-string shape — renders all 18 objects, exit 0. The placeholder value reaches exactly two objects, memex-portal-secrets and memex-migration-secrets, and neither is compared by the check:

PodDisruptionBudget  memex-portal-pdb          placeholder=False
ConfigMap            memex-portal-config       placeholder=False
Deployment           memex-portal-deployment   placeholder=False
Secret               memex-portal-secrets      placeholder=True   ← not compared
Secret               memex-migration-secrets   placeholder=True   ← not compared

Do not ship that yet. While Memex#295 is unfixed the vault half also carries STRUCTURE — for memex: extraVolumes, extraVolumeMounts, extraEnvFrom, ingress, resources, replicas, persistence, portal, migration, config — and the ConfigMap this check compares is rendered from config.*. A stub render would report every vault-only config leaf as CLUSTER-ONLY, i.e. confident nonsense in precisely the category that is supposed to mean hand-applied drift.

The order, therefore

1. Memex#352 (draft, awaiting a maintainer decision) makes the vault half carry secrets only. Until it lands, this workflow cannot give a truthful verdict even if it renders.
2. Then give Chart Drift the record-derived secret-free stub above as its third -f. The workflow already checks the Memex repo out (deploy-repo), so the record is in hand — no new credential.
3. Separately: decide whether the container image belongs in the compared set. Today it is in nobody's.

And regardless of 1–3

A check that can be red for 34 days without anyone noticing needs the treatment deploy-drift.yml already gives the same class (Memex#95/#116): a tracking issue, updated in place, closed when it goes green. A red on the Actions tab is an event; "drift detection is not running" is a standing state, and standing states need an artefact that survives someone closing a ticket.

_{Measured with read-only calls: the Actions API, Ops/Status/memex via the control instance, and a local helm template. No InstanceAction was filed and nothing on any cluster was changed.}