Skip to content

feat(dependabot-policy): merge every Dependabot pull request without a human - #793

Merged
kyle-sexton merged 3 commits into
mainfrom
feat/dependabot-autonomous-merge
Oct 17, 2026
Merged

kyle-sexton merged 3 commits into
mainfrom
feat/dependabot-autonomous-merge

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

Refs: melodic-software/claude-code-plugins#7699

Summary

The owner decided (2026-10-16) that no Dependabot pull request waits on a human: "AI is smart enough to know if a dependabot PR should be auto-merged or not. If there's any that you're like, 'Oh, we should hold this back and make a human do it,' then have an AI do it", and, asked whether the merge lane may merge risky Dependabot PRs on an AI review verdict, "Yes, merge on AI verdict". The dependabot-policy component and REVIEW.md still said any other publisher and any major waits for a human to merge. This rewrites them to the tiered autonomous model claude-code-plugins implements.

Fix

  • components/dependabot-policy/README.md: the section "Merge lanes for GitHub Actions bumps" becomes "Merge tiers for Dependabot pull requests" (new anchor #merge-tiers-for-dependabot-pull-requests).
    • Auto tier: every update non-major (first numeric component unchanged, and the second below 1.0), GitHub Actions only from actions/*, github/*, anthropics/*, ci-status not red, update metadata and title agree; merges once ci-status passes.
    • Review tier: everything else; an AI review lane records a head-bound verdict (merge, fix, close with an ignore-entry proposal, escalate; security updates never closed), and the merge lane merges only on merge at the current head with ci-status green.
    • New "Who merges" subsection: merge-queue repositories merge through a lane holding a GitHub App token, quoting GitHub's docs that GITHUB_TOKEN cannot add pull requests to a merge queue. The ci-workflows pr-automerge-dependabot.yml reusable is documented as valid only without a merge queue.
  • REVIEW.md: the Dependabot bullet states the new rule and points at the new anchor.
  • policy.schema.json: publisher tier enum auto | manual becomes auto | review (no entry used manual); automerge-lockstep.test.mjs renamed to match. The lockstep logic is unchanged: it still compares the reusable's PUBLISHER_ALLOWLIST with the tier: auto keys.
  • THREAT-MODEL.md: residual risk now says no person reads a Dependabot pull request before it merges.
  • components/runner-policy/policy.json is unchanged: the pr-automerge-dependabot.yml contracts stay because standards, songwriting, ci-runner and medley still call the reusable, and none of them uses a merge queue (gh api repos/melodic-software/<repo>/rules/branches/main --jq '[.[]|select(.type=="merge_queue")]|length': claude-code-plugins 1, the other four 0).

Verification

  • npm run test:dependabot-policy: 50/50 pass.
  • npm run lint:dependabot-policy: passed. npm run lint:dependabot-automerge-lockstep: matches at both approved SHAs (ab83b01, 594987a). npm run lint:runner-policy: passed.
  • markdownlint, biome, typos, editorconfig and gitleaks (pre-commit) pass on the touched files.
  • The GitHub docs URL returns 200, and the quoted sentence was re-read from the live page this session.

Related

🤖 Generated with Claude Code

https://claude.ai/code/session_01LN2UYVaEoezf3FxZY72hsS

…a human

Replace the manual lane with a review tier: an auto-tier pull request
(every update non-major, Actions only from actions/github/anthropics,
ci-status not red, metadata and title agree) merges once ci-status
passes; any other merges on an AI review lane's merge verdict at its
current head. The publisher tier enum becomes auto | review. The
GITHUB_TOKEN reusable stays documented for repositories without a merge
queue, since GITHUB_TOKEN cannot add a pull request to one.

Refs: melodic-software/claude-code-plugins#7699

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LN2UYVaEoezf3FxZY72hsS
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 17, 2026 02:31
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T02:33:36.281415Z 51f6c69 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@claude

claude Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Claude finished @kyle-sexton's task in 2m 15s —— View job


CI code review: no findings

  • Gather context (PR diff, AGENTS.md, REVIEW.md, pr-body-contract rule)
  • Read changed files in full
  • Check standards-contract resolution ladder for paved paths
  • Review for correctness / maintainability issues
  • Adversarial verification of candidates
  • Post findings (inline + summary)

This is a docs-only rename of the Dependabot merge model (auto/manual → auto/review tiers) across REVIEW.md, the dependabot-policy README, threat model, schema, and lockstep test. I checked:

  • Anchor consistency: the renamed section ## Merge tiers for Dependabot pull requests produces #merge-tiers-for-dependabot-pull-requests, which matches both inbound links (REVIEW.md, THREAT-MODEL.md).
  • No stale manual-tier references: searched the full repo for manual + dependabot/tier context; the only remaining manual hits are unrelated (repin-policy-lockstep, "reviewer's manual disarm", "automated or manual repin") and don't refer to this policy's tiers.
  • Schema/code safety of the enum rename (policy.schema.json: auto|manual → auto|review): grepped components/dependabot-policy/*.mjs for any hardcoded "manual" string or tier comparison other than tier === "auto" — found none, so the rename can't silently break the analyzer or the lockstep check (automerge-lockstep.mjs:65), which only ever tests for "auto".
  • Cross-references from runner-policy/README.md, github-actions-conventions/README.md, package.json, etc. that mention Dependabot or "manual" — none refer to the renamed tier, so nothing else needed updating.
  • Internal consistency of the new Auto/Review tier language in components/dependabot-policy/README.md (including the new "Who merges" subsection and the ci-status-red → Review-tier escalation path for the fix verdict) — coherent with the policy.json shape (autoMerge.ecosystems, autoMerge.publishers) and with the component's stated scope (it documents but doesn't itself enforce the merge lanes, which live in claude-code-plugins).

No paved-path row applies here (docs/policy change, not a new way of doing something already-listed). No correctness or maintainability issues met the high-signal bar for this lane; no security-relevant logic changed (security lane runs separately for this repo).

claude-lane-summary: lane=claude-review sha=51f6c69f2827c1e31366f5bd8908701a30770bc3 run=38995254590

@claude

claude Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Claude finished @kyle-sexton's task in 4m 29s —— View job


Security review complete

  • Gather context (PR diff, changed files)
  • Check skip gate
  • Review changed files for security-relevant issues
  • Check instruction-surface deletions against protected classes
  • Post findings / summary

Scope: REVIEW.md, components/dependabot-policy/README.md, components/dependabot-policy/THREAT-MODEL.md, components/dependabot-policy/automerge-lockstep.test.mjs, components/dependabot-policy/policy.schema.json. All five changes are documentation/policy-text and test-naming updates (manual → review tier rename); no executable merge/review logic is introduced in this repo. components/dependabot-policy/policy.json is untouched, and autoMerge.ecosystems remains ["github-actions"] only, so the no-review auto tier (merge on ci-status alone) still only applies to GitHub Actions bumps from the three allowlisted publishers — unchanged by this PR.

Instruction-surface deletion considered. The diff removes the prior "manual lane" text that required a human to review and merge any Dependabot pull request from a non-allowlisted publisher or any semver-major bump, replacing it with a "review tier" merged solely on an AI review lane's verdict, and broadens scope from GitHub-Actions-only to every Dependabot ecosystem (README.md#L33-L77). This is exactly the "removed guardrail" shape this lane screens for, so I checked it rather than skipping it. I could not consult the protected-class register (not present in this checkout, no WebFetch available in this mode), so this is recognition-by-consequence only, verified adversarially by a separate verifier pass. Conclusion: not reported as a finding. The PR's own THREAT-MODEL.md update already states the identical residual risk verbatim ("No person reads a Dependabot pull request before it merges... a compromised review-tier release that passes ci-status and the review merges too," THREAT-MODEL.md#L130-L135), the PR body quotes the repo owner explicitly authorizing merge-on-AI-verdict, the sequencing gap (policy landing before the AI review lane exists) is explicitly closed by the new text itself ("Until such a repository runs a review lane, its review-tier pull requests stay open"), and the AI-review enforcement mechanism this policy depends on lives entirely outside this diff, in melodic-software/claude-code-plugins (#7699, #7698). There is no undisclosed or unaccounted-for consequence in this diff to flag.

No other security-relevant issues (injection, auth/access-control gaps, secret handling, logic flaws) found in the changed files. GitHub Actions workflow hardening is out of scope for this lane (zizmor's lane) and no workflow files are changed here anyway.

Findings: 0

claude-lane-summary: lane=claude-security-review sha=51f6c69f2827c1e31366f5bd8908701a30770bc3 run=38995254585

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 51f6c69f28

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread REVIEW.md Outdated
@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

claude-review has reviewed this pull request through b3d7ff5; a later push is reviewed from there.

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

claude-security-review has reviewed this pull request through b3d7ff5; a later push is reviewed from there.

…review tier

Add autoMerge.reviewOnly: dependencies a threat model singles out,
scoped by repository, ecosystem and directory, never take the auto
tier, and a merge verdict on one cites the threat-model control it
checked. The distribution parser's yaml is the first entry, and
distribution/THREAT-MODEL.md now states that control in place of human
review.

Refs: melodic-software/claude-code-plugins#7699

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LN2UYVaEoezf3FxZY72hsS
@claude

claude Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Claude finished @kyle-sexton's task in 3m 43s —— View job


CI code review: incremental (since 51f6c69)

  • Gather context (incremental diff, AGENTS.md, REVIEW.md, pr-body-contract rule)
  • Read changed files in full (REVIEW.md, dependabot-policy/README.md, dependabot-policy.test.mjs, policy.json, policy.schema.json, distribution/THREAT-MODEL.md)
  • Trace reviewOnly end-to-end: schema requirement → policy.json entry → dependabot.yml nested root → THREAT-MODEL.md anchor
  • Check for regressions in other tests that construct autoMerge objects
  • Adversarial verification of the candidate finding (review:lane-verifier, confirmed)
  • Post findings (inline + summary)

This commit adds an autoMerge.reviewOnly list so a threat-model-flagged dependency (here, yaml under /distribution) always takes the review tier with a cited control — directly closing the contradiction Codex flagged against distribution/THREAT-MODEL.md. I checked:

  • Schema enforcement: reviewOnly is validated purely structurally via the existing Ajv-based policy.schema.json → validatePolicyStructure path; there's no new logic in dependabot-policy.mjs (confirmed reviewOnly doesn't appear there outside the schema file), which is correct since this component only audits .github/dependabot.yml shape — the merge-tier enforcement itself lives in claude-code-plugins, per the PR body.
  • New test correctness: dependabot-policy.test.mjs's three negative cases (missing control, non-leading-slash directory, missing reviewOnly entirely) all map to real Ajv required/pattern failures given the new schema — verified by reading the schema rather than running the suite (couldn't execute npm test/node --test in this sandbox; permission to run those was not granted — flagging per the task instructions in case --allowedTools should include them for this lane).
  • No regression: the only two inline autoMerge object literals in the test file both already include reviewOnly; every other test loads the real policy.json, which now has the field, so no existing test is left without a required property.
  • Cross-file consistency: policy.json's entry (directory: "/distribution", dependency: "yaml") matches the nested Dependabot root in .github/dependabot.yml:141, and control: "distribution/THREAT-MODEL.md#npm-supply-chain-production-engine" matches the real heading anchor (## npm supply chain (production engine) → npm-supply-chain-production-engine).

1 finding (inline, high-signal): components/dependabot-policy/README.md lines 85-87 — the new "Security-critical dependencies" paragraph lists a reviewOnly entry's scope as repository, ecosystem, directory, but silently omits dependency, which the schema also requires and which is the field that actually narrows an entry to one package (e.g. yaml) rather than every dependency in that directory. As written, a reader (including a future AI review lane) could over-apply the review-tier/control-citation requirement to unrelated npm dependencies bumped in the same directory. Verified adversarially (see inline comment for the fix suggestion).

No other correctness, paved-path, or maintainability issues met the high-signal bar for this lane; no security-relevant logic changed (security lane runs separately for this repo).

claude-lane-summary: lane=claude-review sha=fe6980f7b137e3f82836dceea3006d731d98e414 run=38995849206

@claude

claude Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Claude finished @kyle-sexton's task in 2m 24s —— View job


CI security review: no findings

  • Gather context (incremental diff, custom instructions, prior review state)
  • Read changed files in full (post-diff state)
  • Check instruction-surface deletions/narrowing against protected classes
  • Review for security-relevant issues (logic, trust boundary, schema)
  • Adversarial verification of candidates
  • Post findings (inline + summary)

Scope (incremental since 51f6c69): REVIEW.md, components/dependabot-policy/README.md, components/dependabot-policy/dependabot-policy.test.mjs, components/dependabot-policy/policy.json, components/dependabot-policy/policy.schema.json, distribution/THREAT-MODEL.md. This commit (fe6980f) is the fix for Codex's P1 finding on the prior commit: it introduces autoMerge.reviewOnly, a list of security-critical dependencies (scoped by repository/ecosystem/directory) that always take the review tier regardless of version, each entry citing the threat-model control a merge verdict must answer to.

What I checked:

  • Schema enforcement is real, not decorative. policy.schema.json requires every reviewOnly entry to carry repository, ecosystem, directory, dependency, reviewed, basis, and control; dependabot-policy.mjs compiles this schema with ajv and runs it against policy.json on every invocation, so a reviewOnly entry missing control or using a malformed directory (no leading /) fails closed with ConfigurationError — verified this is actually exercised by the new test, not just asserted (components/dependabot-policy/dependabot-policy.test.mjs:588-629).
  • Cross-file consistency. The control value in policy.json:32 (distribution/THREAT-MODEL.md#npm-supply-chain-production-engine) resolves to the real heading added at distribution/THREAT-MODEL.md:137, and the directory: "/distribution" scope matches the actual nested Dependabot root in .github/dependabot.yml:141. No dangling reference, no silent scope mismatch that would make the entry fail to apply against the real config.
  • No injection or exec surface. This diff is schema/policy JSON plus prose; dependabot-policy.mjs/automerge-lockstep.mjs contain no exec/eval/spawn, and none of the new fields (control, basis, dependency) are used to build paths or shell commands.
  • Instruction-surface narrowing, considered explicitly. distribution/THREAT-MODEL.md previously stated the yaml dependency "requir[es] human review, excluded from any current or future automerge policy" — an unconditional human-review guarantee for a dependency inside the credentialed sync path. This diff replaces that with: always review tier, merged on an AI verdict that must cite this control. That is a real removal of a human-in-the-loop guarantee, so I did not treat it as docs-only. I could not consult the protected-class register (absent from this checkout, as before). Falling back to recognition-by-consequence: this isn't a new, undisclosed removal — it's the same authorization the owner already gave for the whole PR (quoted in the PR body: "AI is smart enough... have an AI do it," "merge on AI verdict yes"), and the prior security-review pass on 51f6c69 already evaluated and accepted that exact risk shape at the general-policy level for the identical reason. The new text is, if anything, stricter than the general review tier (it adds a mandatory control-citation requirement with no equivalent elsewhere), and nothing in this diff silently drops the requirement that the dependency still never reaches the auto tier (policy.json still scopes autoMerge.ecosystems to ["github-actions"] only, so this npm entry can't accidentally auto-merge under current policy regardless). Not reported as a finding.
  • No enforcement gap specific to this change. The requirement that a merge verdict "cite" the control is advisory text consumed by the external AI review lane in claude-code-plugins, not mechanically checked in this repo — but that's the same non-enforcement already accepted for every other review-tier verdict; this diff doesn't introduce a new unenforced claim, it extends an existing one.

Findings: 0
· branch feat/dependabot-autonomous-merge

claude-lane-summary: lane=claude-security-review sha=fe6980f7b137e3f82836dceea3006d731d98e414 run=38995849308

Comment thread components/dependabot-policy/README.md Outdated
@claude

claude Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Claude finished @kyle-sexton's task in 1m 20s —— View job


CI code review: incremental (since fe6980f)

  • Gather context (incremental diff: .claude-lane/incremental.diff)
  • Read changed file in full (components/dependabot-policy/README.md)
  • Check paved-path index for applicable rows
  • Review for correctness / maintainability issues
  • Adversarial verification of candidates
  • Post findings (none to post)

This commit fixes the finding this lane raised on the prior commit (fe6980f): the "Security-critical dependencies" paragraph's scope-field list was missing dependency. The only change is components/dependabot-policy/README.md:87, which now reads "scoped by repository, ecosystem, directory (as in dependabot.yml) and dependency."

Checked:

  • Schema match: policy.schema.json's reviewOnly item schema requires exactly repository, ecosystem, directory, dependency, reviewed, basis, control (L53-L61). The prose now names all four scope fields (reviewed/basis/control are already covered by the following clause), closing the gap exactly.
  • Real-entry match: policy.json's one reviewOnly entry sets dependency: "yaml", consistent with the doc now stating a dependency, not just a directory, is the discriminator.
  • Lint: MD013 (line length) is disabled in this repo's .markdownlint-cli2.jsonc, so the longer line introduces no lint issue; no other markdownlint/prose rule is affected.
  • No regressions elsewhere: this is the only line that changed; nothing else in the file references the scope-field list.

No paved-path row applies (prose wording fix, not a new mechanism). No correctness or maintainability issues found.

Findings: 0

claude-lane-summary: lane=claude-review sha=b3d7ff5dd026729c0f850f278df6821797508b99 run=38996185559
· branch feat/dependabot-autonomous-merge

@kyle-sexton
kyle-sexton merged commit 968d4e7 into main Oct 17, 2026
22 checks passed
@kyle-sexton
kyle-sexton deleted the feat/dependabot-autonomous-merge branch October 17, 2026 02:50
kyle-sexton added a commit that referenced this pull request Oct 17, 2026
…credentialed CI tools (#794)

No related issue: a supply-chain hardening follow-up to #793 for
melodic-software/claude-code-plugins.

## Summary

A hijacked non-major npm release of a claude-code-plugins root
devDependency is tier `auto` today and merges on a green `ci-status`
with no review. Two of those packages run inside a job holding a secret,
so this lists them under `autoMerge.reviewOnly` and syncs `policy.json`
to claude-code-plugins so its classifiers can read it.

## Fix

- `components/dependabot-policy/policy.json`: two `reviewOnly` entries
for `melodic-software/claude-code-plugins`, ecosystem `npm`, directory
`/`:
- `@anthropic-ai/claude-code`: `eval-browser-dispatch.yml:153` runs `npm
ci` on the root lockfile, `:251` runs its `claude` CLI with
`AUTOMATIONS_CLAUDE_CODE_OAUTH_TOKEN` in the step env (`:203`). It is
the only root package with an install script, and
`.github/dependabot.yml:65-66` exempts it from the cooldown.
- `@playwright/cli`: installed in that job, `:163` runs the playwright
it pulls in, `:189` runs `playwright-cli`, and `:45` makes
`Bash(playwright-cli *)` the eval's default tool, so it runs beside the
claude process that holds the secret.
- `distribution/sync-manifest.yml`: a new `dependabot-policy` component
that ships `policy.json` alone to
`.github/standards/dependabot-policy/policy.json`, managed in
claude-code-plugins.
- `components/managed-files-guard/managed-files-guard.yml`: that
destination joins the guard's `paths` union, as its contract test
requires.
- `components/dependabot-policy/README.md`: one paragraph on the sync
component.

Not listed, with the reason: `@biomejs/biome`, `markdownlint-cli2` and
`htmlhint` run in `pr-require-checks.yml` jobs that hold a read-only
`GITHUB_TOKEN` and no secret. In the eval job they sit on PATH and run
only if an eval target's plugin hooks call them.

Every `control` names
`docs/conventions/pr-pipeline/README.md#review-only-dependencies` in
claude-code-plugins, a section the companion PR adds.

## Verification

- `npm run test:dependabot-policy`: 51 pass, 0 fail.
- `npm run lint:dependabot-policy`: "Dependabot policy passed."
- `bash distribution/sync-manifest.test.sh`: 0 FAIL lines.
- `bash harness/shell/run-tests.sh
components/managed-files-guard/managed-files-guard.test.sh`: 1 passed.

## Related

- Companion: melodic-software/claude-code-plugins#7717. Merge order:
this PR first, then the standards sync PR that lands `policy.json` in
claude-code-plugins, then the companion. Before the file arrives, the
companion's classifiers fail closed and send every Dependabot PR to
review with `dependabot-policy-unreadable`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01LN2UYVaEoezf3FxZY72hsS

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant