Repository navigation
fix(mcp): reword injected descriptions so directory review stops flagging them - #1191
Conversation
What: reworded the descriptions of the injected conversation_id and llm_model arguments and the get_more_tools and send_feedback virtual tools (plus the feedback_type property) so each states what the field is and that it is recorded for analytics. Removed behavioral instructions and broad-triggering lines. Descriptions only: names, annotations, schemas and the canned get_more_tools reply are unchanged. Adds a patch sampo changeset. Why: the ChatGPT and Claude directory reviews flag descriptions that tell the model how to behave beyond one call or ask to be triggered broadly. Tests: - pytest posthog/test/mcp (PYTHONPATH=worktree): 665 passed, 10 skipped - new parameterized test_advertised_descriptions_carry_no_behavioral_instructions failed 6/6 on the old text, passes now - ruff format --check / ruff check on posthog/mcp and posthog/test/mcp: clean Reviewer note: test_units asserted the old conversation_id clause "rest of the conversation, including after later user messages"; its comment says ChatGPT drops the handle without it. The new wording drops that clause as specified, so session continuity on ChatGPT is worth watching.
…ption posthog-js#5112 added "for the rest of the conversation, including after later user messages" because ChatGPT otherwise drops the handle on the first call of each new user message. Keep that semantics; it describes the field, not unrelated model behaviour. The parallel-calls and different-task instructions stay out.
posthog-python Compliance ReportDate: 2026-11-05T12:44:14.442165+00:00 ✅ All Tests Passed!121/121 tests passed Capture_V1 Tests✅ 95/95 tests passed View Details
Capture_Ai Tests✅ 5/5 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
Feature_Flags_Local_Evaluation Tests✅ 4/4 tests passed View Details
|
SDK performance: posthog-pythonBenchmarks completed. No changes flagged for review. Flags require at least a 5% change in the median and all six pairs agreeing on direction. This is a noise filter, not proof of a performance change or a CI gate. View all results
Values are medians of paired measurements. Not flagged does not mean unchanged. Unavailable means a measurement could not be provided, not zero. Estimated allocation measurements retain their instrumentation uncertainty. RSS measures the whole process, not just SDK allocations; observed peaks are sampled checkpoints. Paired ranges in the diagnostics describe variability, not confidence intervals. Each shard uses its own runner; compare Before PR against With PR within a row, not across shards. Compared 102 metric rows; 102 rows shown in all results. Omitted or shortened data remain in |
Why
Customers who instrument an MCP server with MCP analytics are getting flagged or rejected by the ChatGPT app directory and the Claude Connectors Directory over text this SDK adds to their tools. Anthropic rejects descriptions that "tell Claude to behave in ways unrelated to the tool's function" or "interfere with Claude calling other tools". OpenAI rejects descriptions that "recommend overly broad triggering" and tool metadata that seeks protected context such as the system prompt.
Our injected descriptions did all three.
conversation_idtold the model not to make parallel calls and to carry the id across unrelated tasks.llm_modelpointed the model at its system prompt.get_more_toolsasked to be called "whenever your task might benefit … even if existing tools could work as a fallback".What
This PR changes description strings only. Each description now says what the field is and that it is recorded for analytics. The analytics disclosure stays everywhere. Names, schemas,
requiredlists, annotations, defaults, thecontextdescription and the cannedget_more_toolsreply are unchanged.conversation_idllm_modelget_more_toolssend_feedbackfeedback_typeThe PR also adds a parameterized test. It lists tools through the instrumented
tools/listwith every model-visible feature on, and fails if any advertised description contains a phrase reviewers flag. The same wording and test are going into all five MCP SDKs: posthog-js#5573, posthog-python#1191, posthog-ruby#330, posthog-go#414, posthog-rs#332.Testing
pytest posthog/test/mcp: 665 passed, 10 skipped. The new test failed 6/6 on the old text first.ruff format --checkandruff checkonposthog/mcpandposthog/test/mcp: clean.Notes for the reviewer
conversation_idkeeps the cross-message clause from posthog-js#5112, since ChatGPT otherwise drops the handle at each new user message. It no longer says "do not issue parallel tool calls until you have it". That could mint more than one handle at the start of a conversation. On PostHog's own server, 40-60% of handles on Claude surfaces and 5% on OpenAI's hosted client are reused today. That's the number to watch after this ships.get_more_toolswording should produce fewer, more relevant missing-capability reports. That's the intent, but it is a data change.readOnlyHint: falseon the virtual tools (it would make Claude ask permission before each call), renaming_mcp_instructions, retiringget_more_tools, and an opt-out mode for directory submissions.