Skip to content

Add a zero-auth MCP server (Context7) to the PR-review action agent runtime #816

Closed
#826
Closed
Add a zero-auth MCP server (Context7) to the PR-review action agent runtime#816
#826
Labels
dev-leadFor dev-lead agent pickupinitiativeEpic / initiative tracking issue

Description

@don-petry
don-petry
opened on Jun 20, 2026

Story

As a platform owner,
I want a zero-auth MCP server (Context7, for technical/library docs) wired into the PR-review action agent runtime,
so that the review agent gains a real MCP-enriched capability that does not depend on a plan-gated GitHub feature.

Background

Follow-up to the Phase-2 pilot (#681), which closed not-planned on its recommended starter. Findings:

Current wiring (verified, for the implementer)

  • Config file: scripts/engine.sh auto-loads REVIEW_MCP_CONFIG from the conventional path .github/review-mcp.json when present (no thin-caller workflow edit needed). That file today configures only the plan-blocked github / secret_protection server.
  • Allowlist knob: .github/workflows/pr-review.yml sets REVIEW_MCP_ALLOWED_TOOLS: ${{ vars.REVIEW_MCP_ALLOWED_TOOLS || '' }} — i.e. it's driven by the repo variable REVIEW_MCP_ALLOWED_TOOLS (currently mcp__github__*). The engine merges it into the deep + rubber-duck tiers' --allowed-tools (triage stays restricted).
  • Graceful degradation (_emit_mcp_failure_warning, tested in tests/dev-lead/unit/test_engine_mcp.bats) already covers a server being unavailable — confirm Context7-down still completes the review.

Concrete change

1. .github/review-mcp.json — replace the plan-blocked github/secret-scanning server with Context7's zero-auth HTTP endpoint (matches the existing "type": "http" style; no npx install needed in the runner):

 {
   "mcpServers": {
-    "github": {
-      "type": "http",
-      "url": "https://api.githubcopilot.com/mcp/",
-      "headers": {
-        "Authorization": "Bearer ${GH_TOKEN}",
-        "X-MCP-Toolsets": "secret_protection",
-        "X-MCP-Tools": "run_secret_scanning"
-      }
-    }
+    "context7": {
+      "type": "http",
+      "url": "https://mcp.context7.com/mcp"
+    }
   }
 }

Decision: replace rather than add — the github entry's only tool (run_secret_scanning) is plan-blocked, so keeping it just emits an "unavailable" degradation warning on every review. Restore it later if/when advanced_security is licensed. (Alt config form if a runner-local stdio server is ever preferred: {"command":"npx","args":["-y","@upstash/context7-mcp"]} — but the remote HTTP endpoint above is zero-install and zero-auth.)

2. Repo variable REVIEW_MCP_ALLOWED_TOOLS — change from mcp__github__* to:

mcp__context7__*

That permits Context7's tools (mcp__context7__resolve-library-id, mcp__context7__get-library-docs). Optional: Context7 accepts a CONTEXT7_API_KEY header for higher rate limits, but zero-auth works and is the point of this pilot — do not add a secret.

3. Docs — update docs/initiatives/mcp-powered-review.md §5 (and the §2 starter note) to record that the GitHub-MCP secret-scanning starter is plan-gated here (advanced_security unavailable) and Context7 is the actual pilot starter, per #681's outcome.

Acceptance Criteria

  1. .github/review-mcp.json configures the zero-auth Context7 server; the repo variable REVIEW_MCP_ALLOWED_TOOLS is set to mcp__context7__*.
  2. A dry_run=true force-review run on a sample PR shows Context7 connected and its tools permitted (no "not permitted"), mirroring the pilot(mcp): thread REVIEW_MCP_ALLOWED_TOOLS into the review job env #809 verification.
  3. A small sample of PRs is reviewed with Context7 on; qualitative findings captured (e.g. version-aware/library-doc findings the agent surfaced that it couldn't before).
  4. Latency delta vs. server-off recorded using existing tooling (scripts/pr_review_health.sh / daily health workflow) — no new harness.
  5. mcp-powered-review.md updated; a short note records findings + a rollout recommendation. Broader rollout stays a separate, explicit human decision.

Tasks

References

Activity

  1. don-petry
    added
    initiativeEpic / initiative tracking issue
    on Jun 20, 2026
  2. github-actions
    added
    enhancementFeature requests
    on Jun 20, 2026

  3. github-actions commented on Jun 20, 2026

    @github-actions
    github-actionsBot
    on Jun 20, 2026 – with GitHub Actions
    Contributor

    Thanks for the detailed write-up and for linking the full pilot history — the background on the advanced-security gate and the Context7 fallback makes the path forward very clear. Before diving into the wiring work, could you confirm the Context7 MCP server's launch method and endpoint (e.g. npx @upstash/context7-mcp or a hosted URL), since that's the prerequisite for everything else in the task list and will determine how the runtime config needs to look?

  4. don-petry
    added
    dev-leadFor dev-lead agent pickup
    and removed
    enhancementFeature requests
    on Jun 20, 2026

  5. don-petry commented on Jun 20, 2026

    @don-petry
    don-petry
    on Jun 20, 2026
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #816 — Add a zero-auth MCP server (Context7) to the PR-review action agent runtime

    Scope

    Replace the plan-blocked GitHub secret-scanning MCP server in .github/review-mcp.json with the zero-auth Context7 HTTP endpoint, flip the REVIEW_MCP_ALLOWED_TOOLS repo variable to mcp__context7__*, and update the initiative doc to record that the GitHub-MCP secret-scanning starter is plan-gated here while Context7 is the actual pilot starter.

    Implementation Checklist

    • .github/review-mcp.json — replace the github/secret_protection server with the context7 zero-auth http endpoint (https://mcp.context7.com/mcp), matching the committed .github/review-mcp.json.sample.
    • Repo variable REVIEW_MCP_ALLOWED_TOOLS — change mcp__github__* → mcp__context7__* via gh variable set.
    • docs/initiatives/mcp-powered-review.md — update §5 (revised starter) and the §2 starter note: GitHub-MCP secret scanning is plan-gated here (advanced_security unavailable on this plan); Context7 is the actual pilot starter per [Phase 2] Pilot MCP-enriched review on one downstream repo and measure quality + latency #681's outcome. Record findings + rollout recommendation.

    Tests to Write

    • tests/dev-lead/unit/test_engine_mcp.bats — new test asserting the committed review-mcp.json configures the zero-auth context7 server (server present, url is the Context7 endpoint, no Authorization/auth headers, and the plan-blocked github server is gone). Engine wiring itself is unchanged.

    Test Command

    bats tests/dev-lead/unit/test_engine_mcp.bats (full MCP unit suite) + bash .dev-lead/scripts/dev-lead-lint.sh

    Notes on Acceptance Criteria

  6. don-petry commented on Jun 20, 2026

    @don-petry
    don-petry
    on Jun 20, 2026
    CollaboratorAuthor

    Dev-Lead: Implementation Complete

    Plan Execution

    • .github/review-mcp.json — replaced the plan-blocked github/secret_protection server with the zero-auth context7 http endpoint (https://mcp.context7.com/mcp). Matches the committed .github/review-mcp.json.sample exactly.
    • Repo variable REVIEW_MCP_ALLOWED_TOOLS — set mcp__github__* → mcp__context7__* via gh variable set (verified with gh variable get). This permits mcp__context7__resolve-library-id / mcp__context7__get-library-docs; no secret added (zero-auth is the point of the pilot).
    • docs/initiatives/mcp-powered-review.md — added a §2 plan caveat (GitHub-MCP secret scanning gated on advanced_security, unavailable on this plan), rewrote §5 to record the [Phase 2] Pilot MCP-enriched review on one downstream repo and measure quality + latency #681 outcome (plan-gated starter → Context7 as the actual pilot starter), and added §5.1 with findings framing + a rollout recommendation (broader rollout stays a separate human decision).
    • Tests: added 3 assertions to tests/dev-lead/unit/test_engine_mcp.bats pinning the active config to the zero-auth Context7 shape (server present + correct URL, no auth headers, plan-blocked github server gone). Engine wiring (engine.sh) was not touched — the threading was already proven by [Phase 1] Thread an optional MCP config into the Claude agentic review tiers (engine.sh) #677/pilot(mcp): thread REVIEW_MCP_ALLOWED_TOOLS into the review job env #809.

    Test Results

    # tests/dev-lead/unit/test_engine_mcp.bats — TDD red→green for the 3 new tests
ok 16 active config: review-mcp.json configures the zero-auth context7 server
ok 17 active config: context7 server carries no auth (zero-auth, no headers/secret)
ok 18 active config: plan-blocked github secret-scanning server is gone
... 24/24 ok, 0 failures

# Full dev-lead unit suite (23 .bats files)
523 passing, 0 failing

# Repo lint
$ bash .dev-lead/scripts/dev-lead-lint.sh
  [lint] shellcheck on 67 file(s)...
  [lint] validating 4 agent profile(s)... all OK
  [lint] all checks passed   (exit 0)

    Files Changed

    • .github/review-mcp.json: github/secret-scanning server → zero-auth Context7 HTTP endpoint.
    • tests/dev-lead/unit/test_engine_mcp.bats: +3 tests pinning the active Context7 config shape.
    • docs/initiatives/mcp-powered-review.md: §2 plan caveat, §5 rewrite, new §5.1 findings + rollout note.
    • Repo variable REVIEW_MCP_ALLOWED_TOOLS (not a file): mcp__context7__*.

    Notes

    • AC Go-live improvements for PR review agent #2/Add @claude delegation, auto-merge, and rebase handling #3/Optimize review: small-PR and incremental fast paths #4 (live runs): the dry_run=true force-review, the sample-PR qualitative findings, and the latency delta require live GitHub Actions / health-workflow runs and cannot be executed from the implementation sandbox. They will be exercised by the PR-review + daily review-health workflows after merge; §5.1 is written with explicit placeholders for those results to be recorded as they accrue.
    • Graceful degradation (Context7 down): already covered by _emit_mcp_failure_warning and regression-tested in this bats suite (a server-failure marker yields a ::warning::[mcp] naming the server while the verdict still flows through) — no fatal exit, confirming Context7-down still completes the review.
    • Scope: the review prompts (prompts/deep-review.md, prompts/single-review.md) reference mcp__github__run_secret_scanning only conditionally ("when available; exposed only when … enabled for this repo"), so they degrade cleanly with the github server removed and were intentionally left unchanged (out of issue scope).
    • Restore the github secret-scanning entry later if/when advanced_security is licensed on this repo's plan.
  7. don-petry
    added a commit that references this issue on Jun 20, 2026

    fix(review-mcp): revert Context7 transport to type:http (was broken sse)

    3c7c6f9
  8. don-petry
    closed this as completedin #826on Jun 20, 2026
  9. don-petry
    added 2 commits that reference this issue on Jun 20, 2026

    feat: add zero-auth Context7 MCP server to PR-review runtime (#826)

    648b80b

    docs(mcp-review): Context7 A/B eval findings + tool-name fix (#847)

    1ff137e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dev-leadFor dev-lead agent pickupinitiativeEpic / initiative tracking issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions