feat(dependabot-policy): add org Dependabot policy analyzer component - #153
Conversation
|
Claude finished @kyle-sexton's task in 4m 13s —— View job PR Review: feat(dependabot-policy): add org Dependabot policy analyzer componentTasks
SummarySolid implementation that mirrors Findings1.
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e22ebab6a3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Two fail-open gaps found in review:
- A conformant `cooldown.default-days` no longer passes when the soak is
defeated. A match-all `cooldown.exclude` ("*") skips the soak for every
dependency, and a `cooldown.include` list inverts it to opt-in; both are now
reported as cooldown-soak-bypassed and are waivable under the cooldown
category. A narrow first-party exclude (melodic-software/*) still passes.
- A `version: 2` config whose `updates` is absent, misspelled, non-array, or
empty is now reported as updates-missing rather than silently auditing an
empty policy.
Adds regression tests; suite is 24/24. Addresses the Codex review findings on
#153.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K3QehVwmWzkBLpKokNCkkt
|
Claude finished @kyle-sexton's task in 2m 22s —— View job Review of fix commit
|
|
Addressed both Codex findings in b799c4d:
Regression tests added; suite is 24/24. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b799c4da75
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Two more fail-open gaps found in review: - A scalar or null item in `updates` was silently skipped; it is now reported as malformed-update-entry, so a broken entry cannot pass unaudited. - The exception schema now ties `waives` to `reason`: tracks-upstream-release may waive only schedule and cooldown, and single-tool-ecosystem may waive only groups. A waiver outside its reason's scope fails closed, so a groups-only justification can no longer license skipping the cooldown soak. Adds regression tests; suite is 26/26. Addresses the Codex review findings on #153. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K3QehVwmWzkBLpKokNCkkt
|
Claude finished @kyle-sexton's task in 2m 17s —— View job Tasks
Review of fix commit
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8f4bf45e3b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Adds a synced analyzer component that enforces the organization Dependabot policy on every `.github/dependabot.yml` updates entry: the weekly schedule, a cooldown soak of at least 7 days, a groups block so related bumps batch into one pull request, and an open-pull-requests limit of at most 5. Part of the volume-control workstream (melodic-software/github-iac#78). Bot-pull-request bursts were a compound cause of the 2026-07-16 job backlog. A default 3-day cooldown now applies with no configuration (the 2026-07-14 Dependabot change); the standard tightens it to 7 and additionally requires grouping and a pull-request cap. A dependabot.yml varies per repository (different ecosystems and directories), so it cannot be byte-synced. This mirrors runner-policy: a byte-identical analyzer, policy, and schema materialize into consumers while each repository owns its config and a locally-owned exception file. - Read-only Node analyzer with canonical values in policy.json, strict Draft 2020-12 policy and exception schemas, a locked ajv/yaml runtime, and a fail-closed ecosystem-keyed exception seam (tracks-upstream-release, single-tool-ecosystem) with required reason and justification and per-waiver consumption checks. - Behavioral test suite (19 cases) and threat model. - Registered in distribution/sync-manifest.yml (unenrolled; per-consumer rollout is separate integration work). - Dogfooded by a new dependabot-policy CI job that audits this repository's own dependabot.yml, which conforms with no exceptions. - [x] npm run test:dependabot-policy — 19/19 passing - [x] npm run lint:dependabot-policy — passes against this repository - [x] npm run lint:runner-policy — still passes with the new CI job present - [x] biome, markdownlint, and sync-manifest validate — clean Refs #150. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K3QehVwmWzkBLpKokNCkkt
Two fail-open gaps found in review:
- A conformant `cooldown.default-days` no longer passes when the soak is
defeated. A match-all `cooldown.exclude` ("*") skips the soak for every
dependency, and a `cooldown.include` list inverts it to opt-in; both are now
reported as cooldown-soak-bypassed and are waivable under the cooldown
category. A narrow first-party exclude (melodic-software/*) still passes.
- A `version: 2` config whose `updates` is absent, misspelled, non-array, or
empty is now reported as updates-missing rather than silently auditing an
empty policy.
Adds regression tests; suite is 24/24. Addresses the Codex review findings on
#153.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K3QehVwmWzkBLpKokNCkkt
Two more fail-open gaps found in review: - A scalar or null item in `updates` was silently skipped; it is now reported as malformed-update-entry, so a broken entry cannot pass unaudited. - The exception schema now ties `waives` to `reason`: tracks-upstream-release may waive only schedule and cooldown, and single-tool-ecosystem may waive only groups. A waiver outside its reason's scope fails closed, so a groups-only justification can no longer license skipping the cooldown soak. Adds regression tests; suite is 26/26. Addresses the Codex review findings on #153. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K3QehVwmWzkBLpKokNCkkt
8f4bf45 to
9f5815c
Compare
|
Claude finished @kyle-sexton's task in 2m 51s —— View job Tasks
Review of HEAD commit
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9f5815cf95
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Two more fail-open gaps found in review: - A semver-specific cooldown override (semver-patch-days: 0, say) takes precedence over default-days for that update type, so a short override reintroduced the burst the soak prevents. Every configured cooldown day-count (default-days and each semver-*-days) now must meet the floor. - An updates entry that is a mapping but omits package-ecosystem or directory/directories produced a bogus key and passed unaudited; it is now reported as incomplete-update-entry and skipped. Adds regression tests; suite is 28/28. Addresses the Codex re-review findings on #153. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K3QehVwmWzkBLpKokNCkkt
|
Claude finished @kyle-sexton's task in 2m 55s —— View job Tasks
Review of fix commit
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 44297e5bdb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…roups Two more policy-bypass vectors found in review: - open-pull-requests-limit: 0 disables version updates entirely rather than capping volume; the limit must now be between 1 and the maximum (pr-limit-disables-updates). - A group whose applies-to is security-updates does not batch version updates, so groups-missing now requires at least one group covering version updates (applies-to absent or version-updates). Adds regression tests; suite is 31/31. Addresses the Codex re-review findings on #153. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K3QehVwmWzkBLpKokNCkkt
|
Claude finished @kyle-sexton's task in 2m 34s —— View job Tasks
Review of fix commit
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 08a40635b6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Two more policy-bypass vectors found in review, both the match-all-wildcard
shape:
- An ignore rule for dependency-name "*" with no version or update-type
narrowing suppresses every update for the ecosystem; it is now reported as
ignore-disables-updates. A narrow ignore (a specific dependency or a
first-party owner) still passes.
- A version-updates group whose exclude-patterns is a match-all ("*") batches
nothing, so groups-missing now also rejects a version group that excludes
every dependency.
A third review comment proposed rejecting a match-all exclude-paths; that is not
a documented dependabot.yml option, so no rule was added.
Adds regression tests; suite is 34/34. Addresses the Codex re-review findings
on #153.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K3QehVwmWzkBLpKokNCkkt
|
Claude finished @kyle-sexton's task in 2m 37s —— View job Tasks
Review of fix commit
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fca21942f5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…racts (#156) ## Summary Register `runner-input` contracts in `components/runner-policy/policy.json` for two floor-conversion reusables, both pinning ci-workflows PR #124's squash-merge commit `3dfb18452a8c6059a22e62456390d84feb10b42f` (the reviewed source that carries the governed `runner` input): - `link-check.yml` — `allowedInputs: ["runner", "args"]`, `allowedCallerPermissions: {contents: read, issues: write}`. - `pulumi-version-drift-check.yml` — `allowedInputs: ["runner"]`, `allowedCallerPermissions: {contents: read, issues: write}`. Each carries `allowedCallerPermissions` because its rolling tracking-issue lane writes issues through the caller `GITHUB_TOKEN`; the waiver is honored only while the call is genuinely selector-routed, and the reusables' own permissions are unchanged. This also corrects the now-false README claim that the Pulumi version-drift monitor "remains fixed to explicit GitHub-hosted images," adds a review note for both contracts, and extends the runner-policy golden fixture (`runner-policy.test.mjs`) to assert them. `hostedExceptionReasons` is **unchanged** — enum-tightening is the separate last wave and must not land before every consumer has dropped its floor exceptions. ## Why Wave 2 of the private-repo hosted-floor elimination (melodic-software/github-iac#78, decision of record 2026-07-16). ci-workflows PR #124 (Wave 1) added the governed `runner` input to these reusables' source; this PR registers the reviewed contracts so each private consumer (Wave 3) can add a `select-runner` job, route the reusable to the fleet, and drop its `hosted-control-plane` / `privileged-control-plane` exception. `policy.json` is the upstream source of truth synced to five consumers, so the contract belongs here. ## Deferred — standards-sync The third floor-conversion reusable, `standards-sync.yml`, is **intentionally not registered here.** Its caller `GITHUB_TOKEN` is read-only (`sync.yml` declares `permissions: contents: read`; all writes go through a minted App token), but a `runner-input` contract that passes secrets must carry `allowedCallerPermissions`, which the validator requires to include at least one `write` (`runner-policy.mjs:186-198`). There is no honest caller write to declare, so registering it would bake a least-privilege inaccuracy into the managed contract. This is a contract-model gap (the secrets waiver is coupled to a write requirement) routed to the runner-policy component owner for a "reviewed secrets + read-only caller" contract shape; `standards-sync` keeps its hosted-only contract until that lands. ## Test plan - [x] `npm run test:runner-policy` — passing with two new golden assertions on the registered contracts: 143/143 on this branch base, and 146/146 verified against a local rebase onto current `main` (post #155 review-tier admission, #153 dependabot-policy component). - [x] `npm run lint:runner-policy` (`GITHUB_REPOSITORY=melodic-software/standards`) — `Runner policy passed` on both the branch base and the current-`main` rebase (the #155 validator). - [x] biome, markdownlint — clean (pre-commit). - [x] Merge SHA finalized to `3dfb184…` (PR #124 squash-merge) across `policy.json` (both contract keys), the golden-fixture constant, and the README review note. ## Related No linked standards issue. Cross-repo references: - Wave 2 of melodic-software/github-iac#78 (private-repo hosted-floor elimination epic). - Depends on melodic-software/ci-workflows#124 (merged) — source of the pinned reusable SHA. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01K3QehVwmWzkBLpKokNCkkt --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Summary
Adds
dependabot-policy, a synced analyzer component that enforces the organization Dependabot policy on every.github/dependabot.ymlupdatesentry: the weekly schedule, a cooldown soak of at least 7 days, agroupsblock so related bumps batch into one pull request, and anopen-pull-requests-limitof at most 5. Part of the volume-control workstream (melodic-software/github-iac#78).A
dependabot.ymlvaries per repository (different ecosystems and directories) and cannot be byte-synced, so this mirrorsrunner-policy: a byte-identical analyzer,policy.json, and schema materialize into consumers while each repository owns its config and a locally-owned exception file. The canonical valuesstandardsalready dogfoods are the target each repository converges to.Shared-component admission evidence
Per
docs/component-lifecycle.md:standardsitself. The newdependabot-policyCI job runs the analyzer (node components/dependabot-policy/dependabot-policy.mjs --root .) against this repository's owndependabot.ymland is aggregated into the requiredci-statusgate.updatesentry batches, soaks, and caps its pull requests, or records a reasoned exception; the gate blocks drift. Rollback: remove the component entry and the CI job.policy.jsonthrough this repository; thedependabot.ymlit checks stays authoritative per consumer. The config cannot be byte-synced (per-repo ecosystems/directories), Dependabot has noextends/shared-config mechanism, and there is no organization control-plane setting for grouping/cooldown/limit — the analyzer is the only boundary that keeps one source of truth for the policy values.runner-policywas rejected for cohesion; there is no finding overlap (runner-policy never inspectsdependabot.yml). Complements the demand-shaping work in ci-workflows#122 and extends the W4 Dependabot batching decision (github-iac#82) with an enforcement gate rather than duplicating it.ajv@8.20.0andyaml@2.9.0, the exact pins already vetted forrunner-policy(byte-identical lockfile tree, 6 packages). No other dependencies.ajvandyamlare MIT-licensed. The analyzer is read-only with no data or credential access. Boundaries, fail-closed behavior, and review triggers are incomponents/dependabot-policy/THREAT-MODEL.md..github/dependabot.yml(/components/dependabot-policy: weekly, cooldown 7, limit 5, grouped), itself conforming to the policy this component enforces.dependabot-policy.test.mjs) covering conformant pass, each rule's violation, thedirectoriesplural form, exception waivers and inventory drift, fail-closed schema, and malformed YAML/JSON. CI runs the same--root .entrypoint consumers use.Enforcement rollout. Blocking from the start: the sole live consumer (
standards) already conforms with no exceptions, so no observation period is needed. Downstream consumers move to blocking in their own integration PRs after a clean run.Dependabot drift (org-wide audit, informational)
Audited
.github/dependabot.ymlacross all 12 active repos (rawgh api). The org is already ~90% converged;standardsis the reference. Remaining drift the standard will catch:dependabot.ymlat all:knowledge-corpus,songwriting(neither has required CI or PR workflows, so both are low priority and out of the enforced set).medley— broad config drift:nugetrunsdailywith no cooldown and no groups;docker,npm(root),npm /tests/e2e,pip, and bothuvroots lackcooldownandgroups; no explicit pull-request limit; commit prefixchorevs the org'sbuild. Only itsgithub-actionsentry conforms.ci-runner-canary: thegithub-actionsentry has nogroupsblock.open-pull-requests-limitis omitted inprovisioning,dotfiles,medley,github-iac; GitHub's default is already 5, so behavior matches — the analyzer accepts an omitted limit and only flags a limit above 5.claude-code-pluginsnpm-rootdaily+ no cooldown (tracks Claude Code releases) andgithub-iacdotnet-sdkdaily+ no cooldown (SDK freshness) map to thetracks-upstream-releaseexception; single-toolpip/uventries map tosingle-tool-ecosystem. These are handled by the locally-owned exception seam, not flagged.Per-consumer rollout (follow-up integration work)
The 9 requires-ci repositories (each has an active
required_status_checksruleset rule).standardsis wired in this PR; each of the other eight needs a separate integration PR: manifest enrollment (materializes the analyzer + policy), a locally-owned.github/dependabot-policy.jsonfor its documented exceptions, the CI job, and a dependabot entry for the analyzer lockfile.medleyandci-runner-canaryalso need theirdependabot.ymlbrought into conformance in that same change.tracks-upstream-releasedotnet-sdktracks-upstream-releasesingle-tool-ecosystemexceptionsThe component is registered in
distribution/sync-manifest.ymlbut enrolled in no target yet, so nothing materializes until each integration PR adds it to that target'smanagedset.knowledge-corpus,songwriting, andci-runner-canaryhave no required CI (the last has no PR workflows); adependabot.ymlfor the first two is optional low-priority follow-up.Note for the reviewer / merge order
This PR and #152 (concurrency-policy) were authored in parallel off
origin/mainand both touch the same shared integration lines — theci.ymlbiomepaths, theci-statusneedslist andRESULTSaggregation,package.jsonscripts,.github/dependabot.yml, and thedistribution/sync-manifest.ymlcomponent block. Whichever merges second will need a trivial rebase to add its job/entry alongside the other's rather than replacing it. The two components are otherwise independent.Test plan
npm run test:dependabot-policy— 19/19 passingnpm run lint:dependabot-policy— passes against this repository (no exceptions needed)npm run lint:runner-policy— still passes with the new CI job presentsync-manifest.sh validate— cleanCloses #150.