Skip to content

mcp(telemetry): capture the remote server's tool exceptions under mcp_tool/error_code like the other two sinks #10037

Description

@JSONbored

⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.

Context

All three MCP servers capture a thrown tool error as a PostHog exception. Two of them attach the same
two grouping properties; the remote attaches neither.

Stdio — packages/loopover-mcp/lib/telemetry.ts:195:

client.captureException(call.error instanceof Error ? call.error : new Error(String(call.error)), MCP_TELEMETRY_DISTINCT_ID, {
  mcp_tool: telemetry.tool,
  error_code: telemetry.errorCode,
});

Miner — packages/loopover-miner/lib/mcp-dispatch-telemetry.ts:60:

captureMinerPostHogError(call.error, { mcp_tool: telemetry.tool, error_code: telemetry.errorCode });

(captureMinerPostHogError passes that context straight through to captureException
packages/loopover-miner/lib/posthog.ts:68-75.)

Remote — src/mcp/dispatch-telemetry-sink.ts:86:

captureException: (error, call) => {
  if (!isWorkerPostHogConfigured(env)) return;
  // `mcp_tool` + `error_code` are the grouping properties: an exception dashboard broken down by
  // tool and cause is the thing an operator can act on, unlike a stack-only view.
  defer(capturePostHogWorkerError(env, error, { path: `mcp.tool/${call.tool}`, method: call.errorCode ?? "unknown_error" }));
},

The comment names mcp_tool and error_code. The call does not emit them.
capturePostHogWorkerError (src/api/worker-posthog.ts:103) has a fixed HTTP-request shape and maps its
context onto request properties:

client.captureException(err, WORKER_ERROR_DISTINCT_ID, {
  environment: ...,
  request_path: scrubString(context.path),
  request_method: context.method,
});

So a remote MCP tool exception lands in PostHog with request_path: "mcp.tool/loopover_x" and
request_method: "forbidden" — an error code smuggled into a field named "method". In a project that
receives all three surfaces (which is the stated point of the shared contract —
packages/loopover-contract/src/telemetry.ts:4, "Three servers, one shape"), an exception breakdown by
mcp_tool silently omits every exception from the hosted server, and an error_code breakdown omits them
too. The remote is also the surface with the most tool traffic, so the omission is the majority of the data.

This is not covered: test/unit/mcp-dispatch-telemetry-sink.test.ts:74 and :87 assert only that
captureException is or is not invoked, never what properties reach PostHog.

Requirements

  • The remote sink must emit mcp_tool and error_code as exception properties, with the same values and
    the same names the stdio and miner sinks use, so one PostHog breakdown covers all three surfaces.
  • error_code must be the resolved closed-set code (call.errorCode), defaulting to "unknown_error"
    when absent — the same default the current line already applies.
  • The two existing gates stay exactly as they are: isWorkerPostHogConfigured(env)
    (WORKER_POSTHOG_API_KEY, deliberately separate from POSTHOG_API_KEY — see the header note at
    src/mcp/dispatch-telemetry-sink.ts:7), and the defer/waitUntil scheduling.
  • The capture must stay best-effort and must never throw into the tool caller.
  • What must NOT change: capturePostHogWorkerError's existing behaviour for its HTTP callers
    (createWorkerPostHogErrorMiddleware, src/api/worker-posthog.ts:146) — those must keep emitting
    request_path/request_method unchanged, and scrubString must keep being applied to the path.
  • What must NOT change: the WORKER_ERROR_DISTINCT_ID anonymity posture; no per-actor identity is added.

⚠️ Required pattern: match packages/loopover-mcp/lib/telemetry.ts:195 property-for-property. Extend
capturePostHogWorkerError with an optional extra-properties argument (scrubbed the same way as the
existing ones) and pass { mcp_tool, error_code } from src/mcp/dispatch-telemetry-sink.ts:90. What
does NOT satisfy this issue: (a) constructing a hand-built $exception payload in the MCP sink instead
of going through capturePostHogWorkerError — the header at src/mcp/dispatch-telemetry-sink.ts:13
records why that is rejected; (b) renaming request_path/request_method globally, which breaks the
HTTP middleware's existing dashboards; (c) deleting the misleading comment and leaving the properties
as they are.

Deliverables

  • capturePostHogWorkerError in src/api/worker-posthog.ts accepts additional exception properties
    and merges them into the captureException call, with the existing environment/request_path/
    request_method properties unchanged for callers that pass none.
  • src/mcp/dispatch-telemetry-sink.ts:90 passes { mcp_tool: call.tool, error_code: call.errorCode ?? "unknown_error" }.
  • A regression test at test/unit/mcp-dispatch-telemetry-sink.test.ts named for this bug that
    configures WORKER_POSTHOG_API_KEY, invokes sink.captureException(new Error("boom"), call) with
    a call carrying errorCode: "forbidden", and asserts the captured exception's properties contain
    mcp_tool equal to the tool name and error_code equal to "forbidden".
  • A test in the same file asserting the default arm: a call with no errorCode captures
    error_code: "unknown_error".
  • A test asserting createWorkerPostHogErrorMiddleware's captures still carry request_path and
    request_method and do not carry mcp_tool.

All Deliverables above are required in a single PR. A PR that satisfies only some of them — for
example passing the new properties without a test that reads them off the captured event, or changing
capturePostHogWorkerError without asserting the HTTP middleware is unaffected — does not resolve
this issue.

Test Coverage Requirements

This repo enforces 99%+ Codecov patch coverage, branch-counted. vitest.config.ts's
coverage.include covers src/**/*.ts (line 78), so both touched files are measured and gated.
The change introduces or touches these branches, each needing both arms: the new optional
extra-properties parameter (present vs absent), the existing call.errorCode ?? "unknown_error"
nullish arm, and isWorkerPostHogConfigured(env)'s configured/unconfigured gate, which
test/unit/mcp-dispatch-telemetry-sink.test.ts:87 already exercises and must keep exercising.

Expected Outcome

One PostHog breakdown by mcp_tool and error_code covers exceptions from the remote, stdio, and miner
servers instead of two of the three, and the comment at src/mcp/dispatch-telemetry-sink.ts:88 describes
what the code actually sends.

Links & Resources

  • src/mcp/dispatch-telemetry-sink.ts:86 — the remote sink's captureException
  • src/api/worker-posthog.ts:103capturePostHogWorkerError and its fixed request-shaped properties
  • packages/loopover-mcp/lib/telemetry.ts:195 — the stdio sink's properties
  • packages/loopover-miner/lib/mcp-dispatch-telemetry.ts:60 — the miner sink's properties
  • packages/loopover-contract/src/telemetry.ts:4 — "Three servers, one shape"

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions