Skip to content

hosting-kv-copy: materialise a fleet-shared vault object under an instance's prefix (Plugins#1723) - #4103

Merged
meshweaver-cloud[bot] merged 3 commits into
mainfrom
hosting/1723-kv-copy
Sep 12, 2026
Merged

meshweaver-cloud[bot] merged 3 commits into
mainfrom
hosting/1723-kv-copy

Conversation

@rbuergi

@rbuergi rbuergi commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

Operator half of MeshWeaver.Plugins#1723 (the plan half — keyVaultSecrets.secrets[].copyFrom → a "Copy fleet-shared secrets" step — is the paired Plugins PR; Pairs-with: none — this PR removes no public surface).

The hand step it replaces

Memex docs/new-deployment.md has no step for it — it was done ad hoc for build on 2026-09-12 (build.json notes):

build-GitHub-App-PrivateKey — a byte-identical COPY (sha256-verified) of memexsystemorph-GitHub-App-PrivateKey, made 2026-09-12 by piping az keyvault secret show into az keyvault secret set --file /dev/stdin

The Memex PR at the end of this series adds it to the runbook as "done by Provision since Plugins#1723".

What changes

New command hosting-kv-copy --vault <v> --copy <target>=<source> […]:

  • absent target → source read into a variable, written through a mode-600 --file (never argv), read back by id; ::hosting:: kv_copy_created=n;
  • existing target → KEPT, never rewritten (the master-key rule); compared with the source by sha256 and a difference is reported as kv_copy_drift=n — a fact for the operator, not a failure;
  • no value is ever printed; every pair is gated before the vault is touched; every refusal names the exact hand command; dry run reads no value.

Why a copy at all: hosting-kv-purge deletes by PREFIX on teardown, so a cross-prefix mapping would let the first teardown take the fleet's shared PEM with it. The copy makes the object the instance's own, purged with it.

Rights the operator identity needs

Key Vault secret get on the source object and set on the target — both under the existing Key Vault Secrets Officer grant on Systemorph. Nothing new.

Gates run

$ bash deploy/aks/operator/test/run-tests.sh
319 passed, 0 failed          (291 on main; +28 new hosting-kv-copy cases)
$ shellcheck -S warning deploy/aks/operator/bin/hosting-* deploy/aks/operator/test/stubs/kv-copy/az
(clean)

hosting-operator.yml: hosting-kv-copy added to the plan-command list, tracked-file floor raised to 26 (measured on the tree merged with main 8aaf35f: 23 commands + run.sh + _common.sh + _audit.jq; main is at 25), stub added to the tracked-stub list.

Ships how

Hop (C): published as hosting-operator:<sha> on the merge to main; reaches a Provision when operator.image on Deployments/memex moves. Merging is not shipping.

🤖 Generated with Claude Code

…tance's prefix (MeshWeaver.Plugins#1723)

The fleet GitHub App's private key existed only under the control instance's prefix
(memexsystemorph-GitHub-App-PrivateKey); the build instance maps it as
build-GitHub-App-PrivateKey, and the copy was a hand step (az show | az set --file
/dev/stdin). The copy is needed because hosting-kv-purge deletes by PREFIX on
teardown — mapping across prefixes would let the first teardown take the shared PEM.

hosting-kv-copy --vault V --copy <target>=<source> [...]: an absent target is copied
from its source through a mode-600 --file (never argv), read back by id; an existing
target is KEPT — never rewritten — and compared with its source by sha256, a
difference reported as ::hosting:: kv_copy_drift (a fact, not a failure). No value
is ever printed. Every pair is validated before the vault is touched; every refusal
names the exact hand command.

The plan half (keyVaultSecrets.secrets[].copyFrom → one step after the vault step) is
the paired MeshWeaver.Plugins PR. hosting-operator.yml: plan list + tracked-file
floor (24) + stub list.

Tests: 28 behaviour cases against a recorded-argv az stub (test/stubs/kv-copy/az).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 12, 2026 14:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical and moderate correctness and safety issues remain in the copy command.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds hosting-kv-copy to materialize fleet-shared Key Vault secrets under instance-specific prefixes while preserving existing targets.

Changes:

  • Adds copy, keep/drift, hashing, and dry-run behavior.
  • Adds tests and an Azure CLI stub.
  • Updates lifecycle documentation and CI tracking.
File summaries
File Summary
deploy/aks/operator/test/stubs/kv-copy/az Azure CLI test stub; moderate concern that whitespace splitting prevents byte-preserving and trailing-newline coverage.
deploy/aks/operator/test/run-tests.sh Adds command behavior and safety tests.
deploy/aks/operator/bin/hosting-kv-copy Implements copying; findings include critical source-read, byte-preservation, and write-error handling issues; moderate argument and target-read handling issues; and a nit in the refusal message.
deploy/aks/INSTANCE-LIFECYCLE.md Documents the lifecycle behavior.
.github/workflows/hosting-operator.yml Adds CI integration; moderate issue that the tracked-file floor and diagnostic should be 25 rather than 24.
Review details

Suppressed comments (1)

deploy/aks/operator/test/stubs/kv-copy/az:16

  • Because this expands HOSTING_KVC_VALUES unquoted, the stub splits on whitespace and cannot represent the multiline/trailing-newline PEM this command is meant to copy. The new “byte-for-byte” test therefore only exercises a one-line token and will not catch the -o tsv truncation; use a representation that preserves arbitrary secret bytes and add a trailing-newline case.
  for pair in ${HOSTING_KVC_VALUES:-}; do
    if [ "${pair%%=*}" = "$1" ]; then printf '%s' "${pair#*=}"; return 0; fi
  • Files reviewed: 4/5 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/hosting-operator.yml Outdated
@github-actions

github-actions Bot commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

Test Results (shard 0)

261 tests  ±0   261 ✅ ±0   2m 12s ⏱️ +10s
  1 suites ±0     0 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 9729f69. ± Comparison against base commit 8aaf35f.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

Test Results (shard 1)

508 tests  ±0   508 ✅ ±0   43s ⏱️ -6s
  1 suites ±0     0 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 9729f69. ± Comparison against base commit 8aaf35f.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

Test Results (shard 3)

390 tests  ±0   390 ✅ ±0   56s ⏱️ +7s
  2 suites ±0     0 💤 ±0 
  2 files   ±0     0 ❌ ±0 

Results for commit 9729f69. ± Comparison against base commit 8aaf35f.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

Test Results (shard 2)

1 331 tests  ±0   1 329 ✅ ±0   3m 58s ⏱️ -4s
    3 suites ±0       2 💤 ±0 
    3 files   ±0       0 ❌ ±0 

Results for commit 9729f69. ± Comparison against base commit 8aaf35f.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

Test Results (shard 4)

2 024 tests  ±0   2 024 ✅ ±0   4m 29s ⏱️ -58s
    4 suites ±0       0 💤 ±0 
    4 files   ±0       0 ❌ ±0 

Results for commit 9729f69. ± Comparison against base commit 8aaf35f.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

Test Results (shard 5)

    6 files  ±0      6 suites  ±0   8m 18s ⏱️ -6s
2 870 tests ±0  2 678 ✅ ±0  192 💤 ±0  0 ❌ ±0 
2 874 runs  ±0  2 682 ✅ ±0  192 💤 ±0  0 ❌ ±0 

Results for commit 9729f69. ± Comparison against base commit 8aaf35f.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

Test Results

   17 files  ±0     17 suites  ±0   20m 40s ⏱️ -55s
7 384 tests ±0  7 190 ✅ ±0  194 💤 ±0  0 ❌ ±0 
7 388 runs  ±0  7 194 ✅ ±0  194 💤 ±0  0 ❌ ±0 

Results for commit 9729f69. ± Comparison against base commit 8aaf35f.

♻️ This comment has been updated with latest results.

@rbuergi

rbuergi commented Sep 12, 2026

Copy link
Copy Markdown
Contributor Author

Corrected the tracked-file floor in hosting-operator.yml: main already tracks 24 files under deploy/aks/operator/bin/ (21 commands + run.sh + _common.sh + _audit.jq) while the guard declared 23 and its comment said "20 commands" — stale, so one lost file would have passed. This PR adds one command, so the floor is now 25 with the comment re-counted. 🚨 The four sibling PRs (#4103, #4105, #4109, #4110) each do the same on the same line: whichever lands after the first must merge main and set the floor to the real count at that point (26, 27, 28), not keep 25.

🤖 Generated with Claude Code

@rbuergi

rbuergi commented Sep 12, 2026

Copy link
Copy Markdown
Contributor Author

Correction to the comment above: this PR keeps the floor at 24, not 25 — the force-push was declined because the PR had already entered the merge queue (GH006 … Branches that are queued for merging cannot be updated), and dequeuing a green, queued PR to change one number would burn a queue slot and a full CI cycle for no behavioural gain.

The consequence is one file of slack in the guard (real count after this merges: 25, floor: 24) — exactly the slack main already carried before this series (24 real, 23 declared). The siblings #4105, #4109 and #4110 each set 25, so each will land with the same single-file slack. A final one-line PR after all of them land will set the floor to the true count (28) and re-count the comment — that is the clean place to fix it, and it is on my list.

🤖 Generated with Claude Code

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Sep 12, 2026
#4100 (hosting-kv-ensure --db-connection) landed first and touched the same three
places. All three conflicts were "both sides added": the INSTANCE-LIFECYCLE guarantee
list keeps main's extended kv-ensure bullet AND the kv-copy one; the behaviour suite
keeps both blocks (346 passed, 0 failed = 319 + main's 27); the workflow's tracked-stub
list is the union of both stubs.

The tracked-file floor is also CORRECTED rather than carried: main declared 23 while
already tracking 24 (the comment said "20 commands" — stale, so one lost script would
have passed). With hosting-kv-copy the real count is 25, and the floor and its comment
now say 25 (22 commands + run.sh + _common.sh + _audit.jq).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@rbuergi

rbuergi commented Sep 12, 2026

Copy link
Copy Markdown
Contributor Author

Merge resolved by content; floor settled at 25 — measured, not assumed.

origin/main merged in (no rebase), one push. All three conflicts were both sides added, and both sides are kept:

File Resolution
deploy/aks/INSTANCE-LIFECYCLE.md main's extended hosting-kv-ensure bullet (#4100) and this PR's guarantee bullet
deploy/aks/operator/test/run-tests.sh main's hosting-kv-ensure block and this PR's block
.github/workflows/hosting-operator.yml tracked-stub list = the union of both lists

The floor, and how it was counted. git ls-tree -r --name-only origin/main deploy/aks/operator/bin/ | wc -l = 24 on main (tip 69e587c5c): 21 hosting-* commands + run.sh + _common.sh + _audit.jq. main declares 23 with a comment saying "20 commands" — stale, so one lost script would have passed. This PR adds exactly one command, so the true count for this tree is 25 (22 commands + 3), and the guard and its comment now say 25. Verified on the pushed head: tracked=25, commands=22, floor=25.

Guards run on the pushed head: deploy/aks/operator/test/run-tests.sh green (both test blocks coexist), shellcheck -S warning clean over _common.sh, run.sh and every hosting-*. No run was re-run wholesale.

The follow-up is still needed and is now precisely scoped. Each of #4103/#4105/#4109/#4110 sets the floor true for its own merge, so as the siblings land the real count rises 25 → 26 → 27 → 28 while the floor stays 25 — up to 3 files of slack. After all four have merged, a one-line PR sets -lt 28 and expected 28 (25 commands + run.sh + _common.sh + _audit.jq). It cannot be pre-set here: 28 would fail the guard for whichever lands first.

🤖 Generated with Claude Code

@meshweaver-cloud
meshweaver-cloud Bot added this pull request to the merge queue Sep 12, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Sep 12, 2026
…d at 26

Conflicts resolved by content, keeping both sides:
- .github/workflows/hosting-operator.yml: tracked-stub list carries both
  kv-copy/az (this PR) and signin-app/az (main, Plugins#1719).
- deploy/aks/INSTANCE-LIFECYCLE.md: both the hosting-kv-copy and the
  hosting-signin-app bullets.
- deploy/aks/operator/test/run-tests.sh: both the kv-copy and the
  signin-app behaviour blocks (380 passed, 0 failed on the merged tree).

The tracked-file floor in hosting-operator.yml is the TRUE count on the
merged tree, not either side's: `git ls-files deploy/aks/operator/bin | wc -l`
on this tree = 26 (23 commands + run.sh + _common.sh + _audit.jq). main is
at 25 (22 commands); this PR adds hosting-kv-copy, so 26.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRLdhAiX5DjdZC2WUfPg9E
rbuergi added a commit that referenced this pull request Sep 12, 2026
…or measured at 26

Conflicts resolved by content, keeping both sides:
- .github/workflows/hosting-operator.yml: tracked-stub list carries both
  registry-register/az (this PR) and signin-app/az (main, Plugins#1719).
- deploy/aks/INSTANCE-LIFECYCLE.md: both the hosting-registry-register and
  the hosting-signin-app bullets.
- deploy/aks/operator/test/run-tests.sh: both the registry-register and the
  signin-app behaviour blocks (390 passed, 0 failed on the merged tree).

The tracked-file floor in hosting-operator.yml is the TRUE count on the
merged tree, not either side's: `git ls-files deploy/aks/operator/bin | wc -l`
on this tree = 26 (23 commands + run.sh + _common.sh + _audit.jq). main is
at 25 (22 commands); this PR adds hosting-registry-register, so 26. Once
#4103 (hosting-kv-copy) also lands, main tracks 27 and the floor stays a
valid lower bound; the next command added re-measures it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRLdhAiX5DjdZC2WUfPg9E
@meshweaver-cloud
meshweaver-cloud Bot added this pull request to the merge queue Sep 12, 2026
Merged via the queue into main with commit c7262a5 Sep 12, 2026
36 checks passed
rbuergi added a commit that referenced this pull request Sep 12, 2026
…register — floor measured at 27

#4103 (hosting-kv-copy) merged first and touched the same three files, so
the same conflicts came back; resolved by content, keeping every side:
- .github/workflows/hosting-operator.yml: tracked-stub list carries
  kv-copy/az (main), registry-register/az (this PR) and signin-app/az.
- deploy/aks/INSTANCE-LIFECYCLE.md: the hosting-registry-register,
  hosting-kv-copy and hosting-signin-app bullets.
- deploy/aks/operator/test/run-tests.sh: the registry-register, kv-copy and
  signin-app behaviour blocks (418 passed, 0 failed on the merged tree).

The tracked-file floor is re-measured on THIS merged tree:
`git ls-files deploy/aks/operator/bin | wc -l` = 27 (24 commands + run.sh +
_common.sh + _audit.jq). main is at 26 since #4103; this PR adds
hosting-registry-register, so 27.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRLdhAiX5DjdZC2WUfPg9E
@rbuergi
rbuergi deleted the hosting/1723-kv-copy branch October 10, 2026 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants