Skip to content

test(miner): shared contract suite across the read-only AMS MCP tools - #5521

Merged
JSONbored merged 1 commit into
JSONbored:mainfrom
jaytbarimbao-collab:feat-miner-mcp-contract-v7
Jul 13, 2026
Merged

test(miner): shared contract suite across the read-only AMS MCP tools#5521
JSONbored merged 1 commit into
JSONbored:mainfrom
jaytbarimbao-collab:feat-miner-mcp-contract-v7

Conversation

@jaytbarimbao-collab

Copy link
Copy Markdown
Contributor

Adds a shared contract/parity test suite across every read-only AMS MCP tool — mirroring the spirit of the engine's driver-parity suite (#4296). One parameterized table enforces the safety invariants that matter across all of them at once.

What this adds

test/unit/miner-mcp-contract.test.ts, parameterized over the seven read-only tools (status, portfolio, claims, event-ledger, governor-ledger, run-state, plan-store). For each tool it asserts:

  1. No secret-shaped value in a valid response (req 2).
  2. No explicitly-excluded raw column (e.g. payload_json) anywhere in the response (req 3).
  3. A uniform error shape{ isError: true, content: [{ type: "text", text }] } — when the backing store is missing (opener throws) or corrupt (read throws), asserted identically across all tools (req 4).

Adding a new read-only tool costs one table row, not new assertion code (req 5). A canary block feeds a secret-shaped value / an excluded column / a non-error result into the assertions and proves each throws — so the checks can't silently regress to a no-op.

Does not modify any tool's behavior and does not replace each tool's own tests (req 6).

One source of truth for the secret matcher

The secret-shape check reuses the miner pack validator's matcher by importing it — this PR extracts that regex into a shared scripts/forbidden-content.mjs (with a .d.mts type surface, mirroring check-docs-drift.mjs) so scripts/check-miner-package.mjs and this test share one definition and stay byte-for-byte in sync, rather than hand-duplicating the pattern in the test.

Exclusion checks are non-vacuous

The two redacting tools drive their valid case through their real projection over seeded sensitive data, so the excluded-column / secret assertions are genuine regression guards (not no-ops on an empty fixture):

  • audit-feedreadEvents is seeded with a raw entry carrying a full payload (a token-shaped value + payload_json); the real collectEventLedgerAuditFeed projection must reduce each row to metadata only, so none of it survives.
  • governor — the valid case runs against a real temp ledger seeded with sensitive payload (reputation/self_plagiarism/budget/token), so the assertion exercises the actual explicit-named-column SQL and would fail if the SELECT ever widened to include payload_json.

Validation (local)

  • tsc --noEmit: 0 errors.
  • vitest: 39/39 across the contract suite + check-miner-package + governor tests.
  • node scripts/check-miner-package.mjs, npm run docs:drift-check, git diff --check: all clean. Cut from latest main; overlap-free.

Test + a small shared script export, so it's outside vitest's coverage.include; the suite itself is the coverage.

Closes #5199

Adds test/unit/miner-mcp-contract.test.ts, a parameterized parity suite over
every read-only AMS MCP tool asserting, for all of them at once: a valid
response leaks no secret-shaped value and no explicitly-excluded raw column,
and a missing/corrupt backing store yields a uniform error shape. The
redacting tools (audit-feed, governor) drive their valid case through their
real projection over seeded sensitive data, so the exclusion checks are real
regression guards rather than vacuous no-ops. A canary block proves each
assertion catches a violation.

Extracts the miner pack validator's secret-shape matcher into a shared
scripts/forbidden-content.mjs (with a .d.mts type surface) so the validator
and this test import ONE source of truth instead of hand-duplicating the
regex.

Closes JSONbored#5199
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.78%. Comparing base (19b8fe0) to head (a588ff8).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5521   +/-   ##
=======================================
  Coverage   94.78%   94.78%           
=======================================
  Files         566      566           
  Lines       45023    45023           
  Branches    14675    14675           
=======================================
  Hits        42673    42673           
  Misses       1616     1616           
  Partials      734      734           
Flag Coverage Δ
shard-1 44.24% <ø> (ø)
shard-2 35.41% <ø> (ø)
shard-3 32.03% <ø> (+0.05%) ⬆️
shard-4 31.90% <ø> (-0.06%) ⬇️
shard-5 32.96% <ø> (+0.05%) ⬆️
shard-6 43.60% <ø> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 13, 2026
@loopover-orb

loopover-orb Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-13 04:02:14 UTC

4 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

⏸️ Suggested Action - Manual Review

  • Touches a guarded path — held for manual review: This PR changes guardrail-protected path(s): scripts/check-miner-package.mjs (matched scripts/**), scripts/forbidden-content.d.mts (matched scripts/**), scripts/forbidden-content.mjs (matched scripts/**).

Review summary
This PR extracts the existing FORBIDDEN_CONTENT secret-shape regex from scripts/check-miner-package.mjs into a shared scripts/forbidden-content.mjs module (byte-identical, no behavior change) and adds a parameterized contract test suite covering all seven read-only miner MCP tools for secret-leakage, excluded-column, and uniform-error-shape invariants. The audit-feed and governor tools drive their 'valid' case through real projections over seeded sensitive payloads, making the exclusion checks non-vacuous, and a canary block proves each assertion actually fails on a real violation rather than silently passing. The error-shape assertions for tools without explicit try/catch (portfolio, claims, run-state, plans, status) rely on the MCP SDK's built-in exception-to-isError wrapping, which the tests correctly exercise end-to-end rather than assuming.

Nits — 7 non-blocking
  • test/unit/miner-mcp-contract.test.ts:47 — assertNoExcludedColumn only checks for a literal `"key"` substring in the serialized text, so it would miss the value being present under a differently-cased or aliased key; fine for the current fixed set of columns but worth a comment noting the limitation.
  • The 'corrupt' mocks for several tools (e.g. gittensory_miner_get_governor_decisions) only implement the single method under test plus close(), relying on structural typing rather than the full store interface — acceptable for a test seam but slightly brittle if MinerMcpServerOptions grows required fields.
  • Consider asserting the exact error message shape (not just that text is a string) for at least one 'missing' case to lock in that the SDK's default wrapping is actually being exercised, not just tolerated.
  • scripts/forbidden-content.d.mts could re-export a named type or JSDoc comment describing the pattern's intent for downstream consumers unfamiliar with the miner-pack context.
  • nit: test/unit/miner-mcp-contract.test.ts:27 opens a Client/server transport pair per invocation but never closes the client, so add explicit cleanup if the SDK exposes close/dispose to keep this suite from accumulating handles as rows grow.
  • Possible secret-shaped assignment in the diff (generic_secret_assignment) — Verify the value is not a real credential.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #5199
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 42 registered-repo PR(s), 19 merged, 4 issue(s).
Contributor context ✅ Confirmed Gittensor contributor jaytbarimbao-collab; Gittensor profile; 42 PR(s), 4 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence.
Review context
  • Author: jaytbarimbao-collab
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Rust
  • Official Gittensor activity: 42 PR(s), 4 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 13, 2026
@JSONbored
JSONbored merged commit 450f27f into JSONbored:main Jul 13, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a contract/parity test suite across all new AMS read-only MCP tools

2 participants