Skip to content

feat(analytics): track MCP compatibility adoption - #185

Merged
JSONbored merged 2 commits into
JSONbored:mainfrom
oktofeesh1:codex/mcp-compat-adoption
Jun 1, 2026
Merged

feat(analytics): track MCP compatibility adoption#185
JSONbored merged 2 commits into
JSONbored:mainfrom
oktofeesh1:codex/mcp-compat-adoption

Conversation

@oktofeesh1

Copy link
Copy Markdown
Contributor

Summary

Closes #139.

Adds privacy-safe MCP compatibility adoption tracking across the CLI/MCP request path, private analytics APIs, and the operator dashboard.

What changed

  • Added bounded MCP package/client/protocol telemetry parsing with unsafe header rejection.
  • Sent static MCP package/client/version headers from the CLI API client.
  • Recorded MCP telemetry on MCP success and error paths, including compatibility status.
  • Added aggregate MCP compatibility adoption summaries for private operator/API analytics.
  • Surfaced stale/incompatible MCP client counts and version buckets in the analytics dashboard.
  • Regenerated the app OpenAPI snapshot for the new private analytics endpoint.
  • Added regression tests for no-leak invariants, endpoint clamps, summary aggregation, CLI headers, and MCP error telemetry.

Why

MCP adoption needs version distribution and stale/incompatible client visibility without storing tokens, local paths, source contents, or raw actor/session identifiers in product analytics.

Validation

  • npm run test:unit -- test/unit/mcp-compatibility.test.ts test/unit/product-usage-mcp-adoption.test.ts
  • npm run test:unit -- test/unit/mcp-server-telemetry.test.ts
  • npm run test:unit -- test/unit/mcp-cli.test.ts
  • npm run test:integration -- test/integration/api.test.ts
  • npm run test:ci
  • Codex Security diff scan: no reportable findings; every final-diff worklist row completed and the markdown/HTML report rendered locally.

Notes

@oktofeesh1
oktofeesh1 requested a review from JSONbored as a code owner June 1, 2026 12:23
@dosubot dosubot Bot added the size:XXL label Jun 1, 2026
@superagent-security superagent-security Bot added the pr:flagged PR flagged for review by security analysis. label Jun 1, 2026

@superagent-security superagent-security Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superagent found 1 security concern(s).

Comment thread src/db/repositories.ts
@superagent-security superagent-security Bot removed the pr:flagged PR flagged for review by security analysis. label Jun 1, 2026
@oktofeesh1
oktofeesh1 force-pushed the codex/mcp-compat-adoption branch 2 times, most recently from 4abf89d to 31117bc Compare June 1, 2026 18:20
@superagent-security superagent-security Bot added the pr:flagged PR flagged for review by security analysis. label Jun 1, 2026

@superagent-security superagent-security Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superagent found 1 security concern(s).

Comment thread src/api/routes.ts
@oktofeesh1
oktofeesh1 force-pushed the codex/mcp-compat-adoption branch from 31117bc to ca7be59 Compare June 1, 2026 18:28
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent found 1 security concern(s).

P1: New private analytics endpoints lack operator role authorization

Private analytics endpoints omit operator role checks.

Add requireAppRole(c, ["operator"]) guards to both analytics routes before serving data.

AI prompt
Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior.

<file name="src/api/routes.ts">
<violation number="1" location="src/api/routes.ts:810">
<priority>P1</priority>
<title>New private analytics endpoints lack operator role authorization</title>
<evidence>The new handlers `GET /v1/app/analytics/mcp-compatibility` and `GET /v1/app/analytics/daily-rollups` are registered without any `requireAppRole` or identity check, while the adjacent `/v1/app/operator-dashboard` explicitly enforces `requireAppRole(c, ["operator"])`. This leaves aggregate adoption data and daily rollups accessible without the intended operator authorization.</evidence>
<recommendation>Add `const forbidden = await requireAppRole(c, ["operator"]); if (forbidden) return forbidden;` at the top of both new analytics route handlers, consistent with the existing operator-dashboard pattern.</recommendation>
</violation>
</file>

@oktofeesh1
oktofeesh1 force-pushed the codex/mcp-compat-adoption branch from ca7be59 to c6f67a1 Compare June 1, 2026 18:44
@dosubot dosubot Bot added size:XL and removed size:XXL labels Jun 1, 2026
@superagent-security superagent-security Bot removed the pr:flagged PR flagged for review by security analysis. label Jun 1, 2026
Emit bounded MCP package/client telemetry, aggregate compatibility adoption into private analytics APIs and the operator dashboard, and keep the summary output aggregate-only.

Add regression coverage for unsafe header rejection, MCP success/error telemetry, CLI telemetry headers, aggregate adoption summaries, API day clamps, and token/local-path no-leak invariants.
Remove the short-name skip from product usage actor redaction and match actor tokens at segment boundaries so short logins are still removed without corrupting unrelated words.

Add a regression test for two-character actor leakage across repo names, target keys, metadata values, and metadata keys.
@oktofeesh1
oktofeesh1 force-pushed the codex/mcp-compat-adoption branch from c6f67a1 to 7dd641a Compare June 1, 2026 19:34
@dosubot dosubot Bot added size:L and removed size:XL labels Jun 1, 2026

@JSONbored JSONbored left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oktofeesh1 This is ready from my side. The MCP compatibility adoption tracking is now green and merge-clean against current main.

A few notes:

  • The API/OpenAPI/MCP surfaces are updated together.
  • CLI telemetry, compatibility service behavior, MCP server telemetry, and adoption aggregation all have targeted tests.
  • This should be merged after the rollup foundation if we are keeping the analytics stack ordered cleanly.

@dosubot dosubot Bot added the lgtm label Jun 1, 2026
@JSONbored
JSONbored merged commit 40c019c into JSONbored:main Jun 1, 2026
3 checks passed
@JSONbored JSONbored added gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. and removed feature gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. labels Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(analytics): add MCP package and API compatibility adoption tracking

2 participants