Skip to content

Commit f3116ff

Browse files
fix(mcp): reword injected descriptions so directory review stops flagging them (#1191)
* fix(mcp): reword injected argument descriptions for directory review What: reworded the descriptions of the injected conversation_id and llm_model arguments and the get_more_tools and send_feedback virtual tools (plus the feedback_type property) so each states what the field is and that it is recorded for analytics. Removed behavioral instructions and broad-triggering lines. Descriptions only: names, annotations, schemas and the canned get_more_tools reply are unchanged. Adds a patch sampo changeset. Why: the ChatGPT and Claude directory reviews flag descriptions that tell the model how to behave beyond one call or ask to be triggered broadly. Tests: - pytest posthog/test/mcp (PYTHONPATH=worktree): 665 passed, 10 skipped - new parameterized test_advertised_descriptions_carry_no_behavioral_instructions failed 6/6 on the old text, passes now - ruff format --check / ruff check on posthog/mcp and posthog/test/mcp: clean Reviewer note: test_units asserted the old conversation_id clause "rest of the conversation, including after later user messages"; its comment says ChatGPT drops the handle without it. The new wording drops that clause as specified, so session continuity on ChatGPT is worth watching. * fix(mcp): keep the cross-message clause in the conversation_id description posthog-js#5112 added "for the rest of the conversation, including after later user messages" because ChatGPT otherwise drops the handle on the first call of each new user message. Keep that semantics; it describes the field, not unrelated model behaviour. The parallel-calls and different-task instructions stay out.
1 parent 5febc73 commit f3116ff

6 files changed

Lines changed: 60 additions & 20 deletions

File tree

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
pypi/posthog: patch
3+
---
4+
5+
Reworded the descriptions of the injected `conversation_id` and `llm_model` arguments and the `get_more_tools` and `send_feedback` virtual tools so they describe the field instead of instructing the model. This changes your published tool definitions; if your server is listed in the ChatGPT or Claude directories, those platforms may re-review the updated tools.

‎posthog/mcp/constants.py‎

Lines changed: 5 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -28,18 +28,14 @@
2828
)
2929

3030
DEFAULT_CONVERSATION_ID_DESCRIPTION = (
31-
"Pass the exact conversation_id from the server's previous response, unchanged. The server "
32-
"provides it on the first call — never invent one, and do not issue parallel tool calls until "
33-
"you have it. Keep passing the same conversation_id for the rest of the conversation, including "
34-
"after later user messages or on a different task; do not reset it when the user starts a new "
35-
"request."
31+
"The conversation_id this server returned earlier in this conversation, unchanged, including "
32+
"after later user messages. Omit it if you have none. Recorded for the server developer's "
33+
"analytics."
3634
)
3735

3836
DEFAULT_MODEL_PARAMETER_DESCRIPTION = (
39-
"The exact model identifier you (the assistant) are running as, taken from your "
40-
'system prompt or environment (e.g. "claude-opus-4-8", "gpt-5.2"). Used for '
41-
'analytics only. If you do not know your model identifier with certainty, pass "unknown" '
42-
"— never guess."
37+
'The model identifier you are running as, for example "claude-opus-4-8" or "gpt-5.2". '
38+
'Pass "unknown" if you are not certain. Recorded for the server developer\'s analytics.'
4339
)
4440

4541
POSTHOG_MCP_ANALYTICS_SOURCE = "posthog_mcp_analytics"

‎posthog/mcp/feedback.py‎

Lines changed: 4 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,9 @@
2828
_TRUNCATION_SUFFIX = "..."
2929

3030
_DEFAULT_FEEDBACK_DESCRIPTION = (
31-
"Send feedback about this server to its developers. Most important: report a missing "
32-
"capability whenever no available tool fits your task, even if you can work around it "
33-
'(feedback_type "missing_capability"). Also welcome: a tool that failed or confused you, '
34-
"an unhelpful error, or something that worked well. This records the feedback; it does "
35-
"not add or change tools. Do not include user PII or sensitive content."
31+
"Send feedback about this server to its developers: a capability you needed that no tool "
32+
"provides, a tool that failed or misled you, or something that worked well. Records the "
33+
"feedback; it does not add or change tools. Leave out personal or sensitive details."
3634
)
3735

3836
_SEND_FEEDBACK_RESULT_TEXT = (
@@ -47,8 +45,7 @@
4745
"description": (
4846
"What kind of feedback this is. Use 'missing_capability' when the tool you "
4947
"needed does not exist in the tool list - nothing failed, the capability is "
50-
"absent (this is the most valuable report; send it even if you found a "
51-
"workaround). Use 'issue' when an existing tool behaved badly: it failed, "
48+
"absent. Use 'issue' when an existing tool behaved badly: it failed, "
5249
"returned a confusing error, or its description or schema misled you. Use "
5350
"'praise' when something worked notably well. Use 'other' for anything else."
5451
),

‎posthog/mcp/tools.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,8 +39,8 @@ def build_report_missing_descriptor(name: str = GET_MORE_TOOLS_NAME) -> Dict[str
3939
return {
4040
"name": name,
4141
"description": (
42-
"Check for additional tools whenever your task might benefit from specialized "
43-
"capabilities - even if existing tools could work as a fallback."
42+
"Report that the user's request needs a capability this server does not have. "
43+
"Records the report for the server's developers; it does not return or add tools."
4444
),
4545
"inputSchema": {
4646
"type": "object",

‎posthog/test/mcp/test_feedback.py‎

Lines changed: 43 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,7 @@ async def _list_tools_lowlevel(server):
8888
def test_descriptor_defaults():
8989
descriptor = get_feedback_tool_descriptor()
9090
assert descriptor["name"] == SEND_FEEDBACK_TOOL_NAME
91-
assert "missing capability" in descriptor["description"]
91+
assert "a capability you needed that no tool provides" in descriptor["description"]
9292
assert descriptor["inputSchema"]["required"] == ["feedback_type", "summary"]
9393
assert set(descriptor["inputSchema"]["properties"]) == {
9494
"feedback_type",
@@ -540,6 +540,48 @@ async def test_fastmcp_coexists_with_report_missing():
540540
assert _events(client, "$mcp_tool_call") == []
541541

542542

543+
def _descriptions(node):
544+
if isinstance(node, dict):
545+
for key, value in node.items():
546+
if key == "description" and isinstance(value, str):
547+
yield value
548+
yield from _descriptions(value)
549+
elif isinstance(node, list):
550+
for item in node:
551+
yield from _descriptions(item)
552+
553+
554+
@pytest.mark.parametrize(
555+
"phrase",
556+
[
557+
"parallel tool calls",
558+
"system prompt",
559+
"even if existing tools",
560+
"Most important",
561+
"even if you can work around it",
562+
"send it even if",
563+
],
564+
)
565+
async def test_advertised_descriptions_carry_no_behavioral_instructions(phrase):
566+
server = make_fastmcp()
567+
instrument(
568+
server,
569+
FakeClient(),
570+
MCPAnalyticsOptions(
571+
report_missing=True, collect_feedback=True, capture_model=True
572+
),
573+
)
574+
575+
list_handler = server._mcp_server.request_handlers[mcp_types.ListToolsRequest]
576+
result = await list_handler(mcp_types.ListToolsRequest(method="tools/list"))
577+
tools = result.root.tools
578+
assert {"add", "get_more_tools", "send_feedback"} <= {t.name for t in tools}
579+
descriptions = list(_descriptions([t.model_dump() for t in tools]))
580+
581+
assert descriptions
582+
assert not [d for d in descriptions if phrase.lower() in d.lower()]
583+
584+
543585
# --- on_feedback -----------------------------------------------------------------
544586

545587

‎posthog/test/mcp/test_units.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -235,7 +235,7 @@ def test_add_conversation_id_adds_property():
235235
assert out["properties"]["conversation_id"]["type"] == "string"
236236
# Without this clause, ChatGPT drops the handle on each new user message and every message becomes its own session.
237237
assert (
238-
"rest of the conversation, including after later user messages"
238+
"including after later user messages"
239239
in out["properties"]["conversation_id"]["description"]
240240
)
241241

0 commit comments

Comments
 (0)