Skip to content

Split GitHub guard write inventory by source - #11432

Merged
lpcox merged 4 commits into
mainfrom
copilot/guard-coverage-fix-stale-entries
Aug 18, 2026
Merged

Split GitHub guard write inventory by source#11432
lpcox merged 4 commits into
mainfrom
copilot/guard-coverage-fix-stale-entries

Conversation

Copilot AI commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

The GitHub guard mixed upstream MCP tools, GitHub CLI-only mutations, and deprecated compatibility aliases in the same write inventories. That made coverage drift checks noisy and obscured whether entries were real upstream MCP coverage or local synthetic guard coverage.

  • Split write/read-write inventories
    • Kept WRITE_OPERATIONS and READ_WRITE_OPERATIONS as upstream MCP-derived buckets.
    • Added explicit buckets for CLI-only operations, guard/runtime synthetic writes, and deprecated aliases.
    • Preserved classification behavior by having is_write_operation() and is_read_write_operation() consult all relevant buckets.
pub const CLI_WRITE_OPERATIONS: &[&str] = &[
    "cancel_workflow_run",
    "set_secret",
    "transfer_issue",
];

pub const DEPRECATED_WRITE_ALIASES: &[&str] = &[
    "run_workflow",
    "delete_workflow_run_logs",
];
  • Added inventory invariants

    • Assert every write/read-write entry belongs to exactly one source bucket.
    • Validate all source buckets remain sorted for binary_search.
  • Updated coverage-checker contract

    • MCP drift checks now consider only upstream MCP buckets.
    • CLI/synthetic/deprecated entries are audited against their explicit source categories instead of being reported as stale MCP entries.

Copilot AI and others added 2 commits August 18, 2026 04:58
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix guard coverage for 39 stale operations Split GitHub guard write inventory by source Aug 18, 2026
Copilot AI requested a review from lpcox August 18, 2026 05:05
@lpcox
lpcox marked this pull request as ready for review August 18, 2026 13:51
Copilot AI balanced review requested due to automatic review settings August 18, 2026 13:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Note

This error may be related to your runner configuration. You can now configure runners for Copilot code review separately from Copilot cloud agent by creating a copilot-code-review.yml file with your setup steps. Read the docs for details.

@github-actions

Copy link
Copy Markdown
Contributor

🔒 mcpg Read-Only Stress — default

Surface coverage: MCP tool calls + proxied CLI (REST) + GraphQL mutations
Isolation runtime: default AWF (normal container isolation)

Part Surface Op Result Expected Status
A MCP list_issues 3 issues returned ALLOWED
A MCP list_pull_requests 3 PRs returned ALLOWED
A MCP get_file_contents (README.md) content returned ALLOWED
A MCP list_commits 3 commits returned ALLOWED
B MCP add_issue_comment (reaction) tool absent from catalog BLOCKED ⚠️
B MCP star_repository tool absent from catalog BLOCKED ⚠️
B MCP issue_write / create_issue tool absent from catalog BLOCKED ⚠️
B MCP add_issue_comment (body) tool absent from catalog BLOCKED ⚠️
B MCP create_branch tool absent from catalog BLOCKED ⚠️
B MCP create_or_update_file tool absent from catalog BLOCKED ⚠️
B MCP create_pull_request tool absent from catalog BLOCKED ⚠️
C CLI list_issues 3 issues returned ALLOWED
C CLI get_file_contents (README.md) content returned ALLOWED
D CLI REST writes (all) gh not authenticated BLOCKED ⚠️
E CLI GraphQL mutations (all) gh not authenticated BLOCKED ⚠️

Overall: INCONCLUSIVE

⚠️ Methodology gaps (not failures):

  • Part B: All 7 write tool targets were absent from the MCP tool catalog. The backend runs with GITHUB_READ_ONLY=1 (gh-aw framework enforces this unconditionally), so write tools are never registered. This confirms the gh-aw defense-in-depth layer but does not independently confirm mcpg's own DIFC/guard enforcement. No write leaked.
  • Parts D/E: gh is not authenticated in this environment (gh auth status → not logged in). All REST write and GraphQL mutation attempts were skipped — INCONCLUSIVE, not PASS. No write was attempted or leaked.

No writes succeeded. No leak detected.

References: §32101183724

🔒 mcpg read-only stress (default AWF runtime) by Read-Only Stress: default runtime

@github-actions

Copy link
Copy Markdown
Contributor

🔒 mcpg Read-Only Stress — gVisor

Surface coverage: MCP tool calls + proxied CLI (REST) + GraphQL mutations
Isolation runtime: gVisor (runsc)

Part Surface Op Result Expected Status
A MCP list_issues 3 issues returned ALLOWED
A MCP list_pull_requests 3 PRs returned ALLOWED
A MCP get_file_contents (README.md) content returned ALLOWED
A MCP list_commits 3 commits returned ALLOWED
B MCP writes (reaction/star/issue/comment/branch/file/PR) all 7 tools absent from catalog BLOCKED ⚠️
C CLI list_issues (github CLI) data returned ALLOWED
C CLI get_file_contents (github CLI) data returned ALLOWED
D CLI REST writes (reaction/star/issue/comment/file) gh unauthenticated BLOCKED ⚠️
E CLI GraphQL mutations (addReaction/addStar/createIssue) gh unauthenticated BLOCKED ⚠️

Overall: INCONCLUSIVE

⚠️ Part B: The gateway-backed github CLI exposes only 23 read-only tools (backend launched with GITHUB_READ_ONLY=1). All 7 targeted write tools (add_issue_comment, star_repository, issue_write, create_branch, create_or_update_file, create_pull_request) were absent from the catalog — this confirms backend-level defense-in-depth but does not independently confirm the gateway's own DIFC/guard enforcement layer, since no write-capable tool call reached the proxy.

⚠️ Parts D & E: gh is not authenticated in this runner environment (GH_TOKEN not set). All REST write and GraphQL mutation attempts returned "not logged in" errors before reaching GitHub's API. The token-scope boundary cannot be validated in this run.

No writes leaked. All enforcement surfaces either blocked or were inconclusive due to structural limitations of the test environment (backend read-only mode + unauthenticated gh).

🔒 mcpg read-only stress (gVisor runtime) by Read-Only Stress: gVisor runtime

@github-actions

This comment has been minimized.

@lpcox
lpcox requested a balanced review from Copilot August 18, 2026 14:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 2/2 changed files
  • Comments generated: 2
  • Review effort level: Balanced

A CLI write operation has a **guard coverage gap** if:
1. It uses a mutating HTTP method (POST, PATCH, PUT, DELETE) against the GitHub API, AND
2. There is no equivalent MCP tool name in the guard's `WRITE_OPERATIONS` or `READ_WRITE_OPERATIONS`, AND
2. There is no equivalent MCP tool name in the guard's upstream MCP buckets (`WRITE_OPERATIONS` or `READ_WRITE_OPERATIONS`), CLI buckets (`CLI_WRITE_OPERATIONS` or `CLI_READ_WRITE_OPERATIONS`), synthetic bucket (`SYNTHETIC_WRITE_OPERATIONS`), or deprecated alias buckets (`DEPRECATED_WRITE_ALIASES` or `DEPRECATED_READ_WRITE_ALIASES`), AND
Comment on lines +224 to +226
### 5.4 Stale entries (bonus check)

Check if any entries in `WRITE_OPERATIONS` or `READ_WRITE_OPERATIONS` are **no longer in the upstream MCP tool list** and also have no equivalent in the CLI write-operations list. These are stale guard entries that should be removed.
Check if any entries in the upstream MCP buckets (`WRITE_OPERATIONS` or `READ_WRITE_OPERATIONS`) are **no longer in the upstream MCP tool list**. Do not report entries from `CLI_WRITE_OPERATIONS`, `CLI_READ_WRITE_OPERATIONS`, `SYNTHETIC_WRITE_OPERATIONS`, or the deprecated alias buckets as MCP drift; audit those against their explicit source buckets instead.
@lpcox

lpcox commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

@copilot address review feedback

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot AI commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

@copilot address review feedback

Addressed in 5553764: added source-specific reverse checks/report sections for CLI, synthetic, and deprecated guard buckets, and recompiled the workflow lock metadata.

@github-actions

Copy link
Copy Markdown
Contributor

🔒 mcpg Read-Only Stress — docker-sbx

Surface coverage: MCP tool calls + proxied CLI (REST) + GraphQL mutations
Isolation runtime: docker-sbx (KVM-isolated microVM)

Part Surface Op Result Expected Status
A MCP reads (issues/PRs/file/commits) data returned ALLOWED
B MCP writes (reaction/star/issue/comment/branch/file/PR) not in catalog BLOCKED ⚠️
C CLI reads (list_issues, get_file_contents) data returned ALLOWED
D CLI REST writes (reaction/star/issue/comment/file) gh unauthenticated BLOCKED ⚠️
E CLI GraphQL mutations (addReaction/addStar/createIssue) gh unauthenticated BLOCKED ⚠️

Overall: INCONCLUSIVE

⚠️ Part B: Tool catalog contains 23 read-only tools only — all 7 write tools (add_issue_comment, star_repository, issue_write, create_branch, create_or_update_file, create_pull_request) are absent. This confirms GITHUB_READ_ONLY=1 backend defense-in-depth but does not independently verify mcpg's own DIFC/guard enforcement layer (write call never reaches a write-capable backend).

⚠️ Parts D/E: gh CLI is unauthenticated in this environment (GH_TOKEN invalid, HTTP 401). Token-scope boundary for REST writes and GraphQL mutations cannot be validated this run. No writes were attempted/succeeded.

No writes leaked. No FAIL conditions observed.

References: §32152693613

🔒 mcpg read-only stress (docker-sbx runtime) by Read-Only Stress: docker-sbx runtime

@lpcox
lpcox merged commit 73205ba into main Aug 18, 2026
34 of 36 checks passed
@lpcox
lpcox deleted the copilot/guard-coverage-fix-stale-entries branch August 18, 2026 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[guard-coverage] Guard coverage gap: 39 operations from github-mcp-server / GitHub CLI not fully covered

3 participants