Skip to content

perf(release-audit-report, release-archive-sweep, release-keys-sync): decide and render as code - #1625

Merged
potiuk merged 3 commits into
perf/release-mgmt-5-preparefrom
perf/release-mgmt-6-audit-archive-keys
Oct 16, 2026
Merged

potiuk merged 3 commits into
perf/release-mgmt-5-preparefrom
perf/release-mgmt-6-audit-archive-keys

Conversation

@potiuk

@potiuk potiuk commented Oct 13, 2026

Copy link
Copy Markdown
Member

Summary

Last PR of the release-management stack; it builds on the prepare PR.

  • Scripts move. The last three skills' scripts move unchanged into tools/release-management: render_record → audit_report.record, retention → archive_sweep, check_key → keys_sync.
  • audit-report gets verdict, gather (the mail-archive search plan; the search itself stays with the agent), voters (roster handles through vote-tally's binding resolution, never citing an unresolved voter), check, record and pr. The PR step moves, byte-identical with its generated adversarial block, to audit-log-pr.md at the skill root.
  • archive-sweep gets verdict, listing, trains, retention and render. The render covers svn mv, gh release delete (with the permanent-deletion warning), aws s3 mv, self-hosted, and atr (no command).
  • keys-sync no longer finds the key by text scan:
    • A new read-only release-verify list-keys lists fingerprints and UIDs in the existing throwaway GNUPGHOME. It uses no agent, no keyring and no network, and refuses private-key material.
    • keys-sync verdict classifies that output as proceed / noop / blocked.
    • render draws the KEYS block, the svn / git / generic sequence and the upload reminder.
  • Size: audit-report goes from 6,657 to 5,136 tokens, archive-sweep from 4,573 to 4,165 and keys-sync from 4,871 to 4,584.

Security notes

  • Every rendered command quotes its values with the shared helpers.
  • release-management still runs nothing. The only gpg call is release-verify list-keys, with the same isolation guarantees as its other checks.

Behaviour changes worth a look

  • keys-sync matches parsed fingerprints, primary or subkey. A fingerprint that appears only in a KEYS comment no longer counts as present.
  • keys-sync fails closed when it cannot compare (no keyserver key or UID). noop means the keyserver's primary UID is among the KEYS entry's UIDs.
  • The svn blocks in all three skills lose their \ # comment continuation lines, which broke a pasted command.
  • archive-sweep: gh release list adds --exclude-drafts. The self-hosted backend needs --command-template, since no config key holds that command.
  • audit-report refuses a planning-issue URL that isn't canonical, and escapes markdown in a hostile product name.

Type of change

  • Skill change — eval fixtures updated below
  • Python package (tools/release-management, tools/release-verify)

Test plan

  • prek run --all-files passes on the stack tip.
  • tools/release-management and tools/release-verify: pytest / ruff / mypy, with three real-load integration tests. The gpg-based list-keys tests ran.
  • Evals: the scripted cases are now pytest cases named after each former case, with byte-identical expected records. The kept judgement and injection cases pass: audit-report 4/4, archive-sweep 2/2, keys-sync 2/2. The baselines were 9/9, 10/10 and 9/9.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5). I reviewed the diff and the test results; commits carry Generated-by:.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GTPsZ5aEE5bUVYqp47Dpvv

@potiuk potiuk added family:tools tools/* capability:resolve Close-out: invalidate, dedupe, CVE allocate, announcement family:release-management release-* skills substrate:release Tool substrate: release-artefact helpers (reproducible archive build, lint, comparison) labels Oct 13, 2026
@potiuk
potiuk added this pull request to stack #1620 October 13, 2026 15:24
@potiuk
potiuk force-pushed the perf/release-mgmt-6-audit-archive-keys branch from 784a463 to 2f20bb6 Compare October 14, 2026 08:09
@potiuk
potiuk force-pushed the perf/release-mgmt-6-audit-archive-keys branch from 2f20bb6 to a914089 Compare October 15, 2026 23:06
@github-actions github-actions Bot added family:docs Docs, MISSION.md, READMEs capability:triage Sweep + classify + propose disposition capability:stats Read-only dashboards, metrics, governance evidence substrate:framework-dev Tool substrate: build / validate / eval the framework itself labels Oct 15, 2026
@potiuk
potiuk force-pushed the perf/release-mgmt-6-audit-archive-keys branch from a914089 to 40fbaad Compare October 15, 2026 23:37
@potiuk
potiuk force-pushed the perf/release-mgmt-6-audit-archive-keys branch 2 times, most recently from a68e5ea to b2250d9 Compare October 16, 2026 00:20
@potiuk
potiuk force-pushed the perf/release-mgmt-6-audit-archive-keys branch from b2250d9 to 835ad69 Compare October 16, 2026 00:25
@potiuk
potiuk force-pushed the perf/release-mgmt-6-audit-archive-keys branch from 835ad69 to 8180f2e Compare October 16, 2026 03:18
@github-actions github-actions Bot added the family:setup setup-* skills label Oct 16, 2026
@potiuk
potiuk force-pushed the perf/release-mgmt-6-audit-archive-keys branch from 8180f2e to 897023c Compare October 16, 2026 04:54
@potiuk
potiuk force-pushed the perf/release-mgmt-6-audit-archive-keys branch from 897023c to 02fcdf7 Compare October 16, 2026 05:42
@potiuk
potiuk force-pushed the perf/release-mgmt-6-audit-archive-keys branch from 02fcdf7 to 0b6fa2a Compare October 16, 2026 06:28
@potiuk
potiuk force-pushed the perf/release-mgmt-6-audit-archive-keys branch from 0b6fa2a to 754a0be Compare October 16, 2026 06:56
@potiuk
potiuk force-pushed the perf/release-mgmt-6-audit-archive-keys branch from 754a0be to f33ed9f Compare October 16, 2026 07:15
… decide and render as code

The last three release skills each had one rule in a skill-local script
and the rest in prose. The scripts move unchanged into
tools/release-management (render_record → audit_report.record,
retention → archive_sweep, check_key → keys_sync), and new commands do
what the agent did by hand, printing JSON and running nothing:

- audit-report: `verdict` (planning issue found, `org/repo#n`
  normalised to the canonical URL), `gather` (PonyMail subjects and
  queries, the `svn info` revision command, the required fields; the
  search itself stays with the agent), `voters` (the [RESULT] voters
  mapped to roster handles with vote-tally's binding resolution,
  unresolved ones listed, never cited), `check` (Step 1 missing /
  redacted fields, schema violations, the thread-permalink shape),
  `record`, and `pr`.
- archive-sweep: `verdict`, `listing`, `trains` (strict `N.x` lines
  become patterns; a free-text retention rule leaves `keep` to the
  agent), `retention`, and `render` for svn mv, gh release delete (with
  the permanent-deletion warning), aws s3 mv, self-hosted, and atr
  (no command).
- keys-sync: whether the fingerprint is in KEYS and its UID matches the
  keyserver's is no longer a text scan. A new read-only
  `release-verify list-keys` lists fingerprints and UIDs in the
  existing throwaway GNUPGHOME (no agent, no keyring, no network,
  private-key material refused), and `keys-sync verdict` classifies it
  as proceed / noop / blocked, failing closed when it cannot compare.
  `render` draws the `# <uid>` KEYS block, the svn / git / generic
  sequence and the upload reminder with the expiry advisory.

Every rendered command quotes its values with the shared shell helpers.
Each backend and outcome gets its own file; audit-report's PR step
moves, byte-identical with its generated adversarial block, to
audit-log-pr.md at the skill root. Golden rules, Hard rules and the
eval headings are unchanged. audit-report 6,657 → 5,136 tokens,
archive-sweep 4,573 → 4,165, keys-sync 4,871 → 4,584.

Evals: the scripted cases become pytest cases named after each former
case (expected records byte-identical); the judgement and injection
cases stay, their fixtures now carrying the real tool output.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_01GTPsZ5aEE5bUVYqp47Dpvv
…refuse what the rules forbid instead of defaulting

Review findings on the three tools:

- archive-sweep: `retention` refuses a trains file whose retention rule
  was not decided (keep_decided false) unless every train names its keep,
  so a free-text rule no longer falls back to keep=1 and archives what
  the rule keeps.
- archive-sweep: `render` refuses a retention output without
  mapping_complete, and any latest-of-line, orphan or pre-release in
  past_retention (Golden rule 2).
- archive-sweep: a trailing -incubating qualifier is a release, not a
  pre-release, so podlings archive their older releases.
- keys-sync: the Step 0 verdict normalises the fingerprint as Step 1
  does (a 0x prefix no longer turns the no-op into a duplicate append).
- keys-sync: Step 2 requires --key-listing and appends only an armour
  holding exactly the key Step 1 checked.
- audit-report: `pr --base` defaults to <default-branch>, not main.

Generated-by: Claude Opus 5
…er fixes into the tool and read S3 and version paths right

Port of the key-checker fixes main landed on the script this series moves
into tools/release-management (#1634, #1635, #1646):

- A key passes only with a signing-capable primary or subkey that itself
  meets the strength floor and is not revoked, invalid, disabled or
  expired; a certify-only primary with no such subkey is blocked
  (`signers` in the check output).
- GPG failures are reported, never read as "key not found": unreadable or
  non-file input, a failed import or listing, and gpg failing to start are
  errors naming the stage and exit code, with the "no key data" exception
  ported as is. release-verify list-keys, which runs gpg for keys-sync,
  now also reports each key's capabilities and subkeys.
- Every test of main's test_check_key.py has a ported equivalent.

Adversarial-review findings:

- archive-sweep reads S3 URLs with promote's whole-host path-style rule,
  so a virtual-hosted bucket whose name starts with "s3" keeps its key.
- A release_dist_url_template with a literal around <version> (v<version>,
  foo-<version>-bin) gives the right release directory and archive URLs.
- keys-sync reads a GitHub KEYS URL as branch (the first segment after
  blob/) plus a path that may include subdirectories, so KEYS can live
  in e.g. dist/; a new optional keys_file_branch names a branch that
  contains '/' (release/1.x). Without it, a subdirectory reading is
  warned; a URL whose file is not KEYS is refused.
- The keyserver's UIDs are compared with KEYS as a set, so a reordered
  keyserver listing is no longer reported as a rolled key.

Generated-by: Claude Opus 5
@potiuk
potiuk force-pushed the perf/release-mgmt-6-audit-archive-keys branch from f33ed9f to 7a45dd7 Compare October 16, 2026 07:37
@potiuk
potiuk merged commit 5fc6a32 into main Oct 16, 2026
14 checks passed
@potiuk
potiuk deleted the perf/release-mgmt-6-audit-archive-keys branch October 16, 2026 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capability:resolve Close-out: invalidate, dedupe, CVE allocate, announcement capability:stats Read-only dashboards, metrics, governance evidence capability:triage Sweep + classify + propose disposition family:docs Docs, MISSION.md, READMEs family:release-management release-* skills family:setup setup-* skills family:tools tools/* substrate:framework-dev Tool substrate: build / validate / eval the framework itself substrate:release Tool substrate: release-artefact helpers (reproducible archive build, lint, comparison)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant