Repository navigation
Conversation
…rage Claude web and desktop silently drop a tool whose serialized inputSchema reaches 16,384 chars. Only the exec tool had a budget test, but tools mode registers every tool on its own and 42 schemas are already over. The count of oversized schemas can now only go down, and nested property description coverage cannot fall below its current level. Both checks live in the existing schema snapshot test and use counts and percentages only, so no tool names are copied into the test.
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
|
FLASH MODE - Faster, but stupid, use regular ReviewHog for a heavy review 🦔 PostHog Review reviewed this pull requestFound 0 must fix, 1 should fix, 0 consider. Published 1 finding (view the review). |
|
FLASH MODE - Faster, but stupid, use regular ReviewHog for a heavy review |
![posthog[bot]](https://avatars.githubusercontent.com/in/1203049?s=60&v=4){kind=small}
HostHog preview —
|
|
Risk: No findings Test-only change: it refactors the snapshot test's tool-loading into a shared helper and adds two ratchet checks (count of tool schemas at or over the claude.ai 16,384-char inputSchema limit, and a floor on described nested properties), plus a new exported constant that only the test imports. No production code paths, authentication, parsing, or trust boundaries are altered, so there is no security risk introduced. Sentinel reviewed |
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
🤖 CI report✅ Trunk lane — non-backend lane (
|
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. 🧰 Additional context used📚 Code guidelines (7)📝 WalkthroughWalkthroughThe change adds 🚥 Pre-merge checks | ✅ 1✅ Passed checks (1 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Note
Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.
🟡 Other comments (1)
services/mcp/tests/unit/tool-schema-snapshots.test.ts-156-183 (1)
156-183: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winRecurse into object-valued
additionalPropertiesschemas.
countNestedPropertiescountsproject.assetsbut never visits itsadditionalPropertiesschema. This excludes fields such ascontentinside each asset value from both totals, allowing many undescribed nested fields to escape the 55% gate.Suggested fix
const items = schema.items && typeof schema.items === 'object' ? [schema.items as Record<string, unknown>] : [] - return [...branches, ...items] + const additionalProperties = + schema.additionalProperties && typeof schema.additionalProperties === 'object' + ? [schema.additionalProperties as Record<string, unknown>] + : [] + return [...branches, ...items, ...additionalProperties]
ℹ️ Review info
⚙️ Run configuration
- Configuration used: Repository: PostHog/posthog/.coderabbit.yaml
- Review profile: QUIET
- Plan: Enterprise
- Run ID:
02809386-b048-4e02-a52c-019c03fd2a15
📒 Files selected for processing (2)
services/mcp/src/lib/constants.tsservices/mcp/tests/unit/tool-schema-snapshots.test.ts
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 4 remain after this review.
| it('keeps the number of tool schemas over the claude.ai registry limit from growing', async () => { | ||
| const tools = await loadSnapshotTools() | ||
| const oversized = tools | ||
| .map((tool) => ({ | ||
| name: tool.name, | ||
| size: JSON.stringify(z.toJSONSchema(tool.schema, { io: 'input', reused: 'inline' })).length, | ||
| })) | ||
| .filter(({ size }) => size >= CLAUDE_REGISTRY_INPUT_SCHEMA_CHAR_LIMIT) | ||
| .sort((a, b) => b.size - a.size) | ||
|
|
||
| const offenders = oversized.map(({ name, size }) => `- ${name}: ${size} chars`).join('\n') | ||
|
|
||
| expect( | ||
| oversized.length, | ||
| `${oversized.length} tool schemas are at or over ${CLAUDE_REGISTRY_INPUT_SCHEMA_CHAR_LIMIT} chars, ` + | ||
| `above the ratchet of ${OVERSIZED_SCHEMA_COUNT}. Claude web and desktop silently drop such tools in tools mode. ` + | ||
| `Shrink the schema with exclude_params, include_params or param_overrides in the product tools.yaml, ` + | ||
| `or shorten serializer help_text. Largest schemas:\n${offenders}` | ||
| ).toBeLessThanOrEqual(OVERSIZED_SCHEMA_COUNT) | ||
| expect( | ||
| oversized.length, | ||
| `Only ${oversized.length} tool schemas are at or over the limit. Lower OVERSIZED_SCHEMA_COUNT ` + | ||
| `from ${OVERSIZED_SCHEMA_COUNT} to ${oversized.length} so the count only goes down.` | ||
| ).toBe(OVERSIZED_SCHEMA_COUNT) | ||
| }) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
sed -n '82,116p' services/mcp/tests/unit/tool-schema-snapshots.test.ts
sed -n '190,232p' services/mcp/tests/unit/tool-schema-snapshots.test.ts
sed -n '125,170p' services/mcp/src/hono/tool-executor.tsRepository: PostHog/posthog
Length of output: 5580
🏁 Script executed:
git diff --unified=20 61cdf3ae4eace53a0084f7f82c98a2dfbe526389 acdfdca02d364038ed4b326b83716272ea747cff -- services/mcp/tests/unit/tool-schema-snapshots.test.ts
printf '\\n--- tools/index.ts ---\\n'
sed -n '130,225p' services/mcp/src/tools/index.ts
printf '\\n--- executor advertised builder ---\\n'
sed -n '145,235p' services/mcp/src/hono/tool-executor.ts
printf '\\n--- local prepareToolList refs/tests ---\\n'
rg -n -C 4 -- 'prepareToolList|inputSchema|rawInputSchema' services/mcp/src services/mcp/tests/unit --glob '*.ts'Repository: PostHog/posthog
Length of output: 42199
🏁 Script executed:
git diff --unified=20 61cdf3ae4eace53a0084f7f82c98a2dfbe526389 acdfdca02d364038ed4b326b83716272ea747cff -- services/mcp/tests/unit/tool-schema-snapshots.test.ts
printf '\n--- tools/index.ts ---\n'
sed -n '130,225p' services/mcp/src/tools/index.ts
printf '\n--- executor advertised builder ---\n'
sed -n '145,235p' services/mcp/src/hono/tool-executor.ts
printf '\n--- local prepareToolList refs/tests ---\n'
rg -n -C 4 -- 'prepareToolList|inputSchema|rawInputSchema' services/mcp/src services/mcp/tests/unit --glob '*.ts'Repository: PostHog/posthog
Length of output: 41961
🏁 Script executed:
python3 - <<'PY'
import json
from pathlib import Path
root = Path('services/mcp/tests/unit/__snapshots__/tool-schemas')
if not root.exists():
print(f'MISSING {root}')
else:
rows = []
for path in root.glob('*.json'):
try:
schema = json.loads(path.read_text())
except Exception as exc:
print(f'UNREADABLE {path}: {exc}')
continue
size = len(json.dumps(schema, separators=(',', ':'), ensure_ascii=False))
rows.append((size, path.name))
print(f'{root}: {len(rows)} snapshots')
for size, name in sorted(rows, key=lambda row: abs(row[0] - 16384))[:20]:
print(f'{size:>6} {name}')
print('at-or-over:', sum(size >= 16384 for size, _ in rows))
PY
printf '\n--- catalog schema construction ---\n'
nl -ba services/mcp/src/hono/tool-catalog.ts | sed -n '72,95p;115,180p;205,232p'
printf '\n--- prepareToolList test and binding ---\n'
nl -ba services/mcp/tests/unit/instructions-formatter-snapshot.test.ts | sed -n '178,194p'
rg -n -F -- 'new PostHogMCP' services/mcp/tests/unit/instructions-formatter-snapshot.test.ts
printf '\n--- threshold constant ---\n'
rg -n -F -- 'CLAUDE_REGISTRY_INPUT_SCHEMA_CHAR_LIMIT' services/mcp/src/lib/constants.ts services/mcp/tests/unit/tool-schema-snapshots.test.tsRepository: PostHog/posthog
Length of output: 8160
🌐 Web query:
@posthog/mcp 0.21.2 PostHogMCP prepareToolList implementation inject context inputSchema source
💡 Result:
I could confirm what `prepareToolList` is intended to do, but not the **exact source implementation for `@posthog/mcp@0.21.2`** from the available indexed sources.
The current package documentation says `prepareToolList()` adds analytics fields such as `context` to advertised tool schemas; it also describes optional virtual-tool injection. It notes that `prepareToolCall()` uses the original tool descriptor to determine which fields are SDK-owned, so it can remove injected fields without stripping fields declared by the application. ([npmjs.com](https://www.npmjs.com/package/%40posthog/mcp?utm_source=openai))
A release note for an older version (`0.4.0`) confirms that `prepareToolList` injected `context` into tool input schemas, but that is **not evidence of the exact implementation in 0.21.2**. ([newreleases.io](https://newreleases.io/project/github/PostHog/posthog-js/release/%40posthog%2Fmcp%400.4.0?utm_source=openai))
So the version-specific details you asked for—how 0.21.2 clones or mutates tools, edits `inputSchema`, and tracks injected context—remain unresolved by these sources. The current package listing is newer (`0.21.4`), not 0.21.2. ([npmjs.com](https://www.npmjs.com/package/%40posthog/mcp?utm_source=openai))
Citations:
- 1: https://www.npmjs.com/package/%40posthog/mcp?utm_source=openai
- 2: https://newreleases.io/project/github/PostHog/posthog-js/release/%40posthog%2Fmcp%400.4.0?utm_source=openai
- 3: https://www.npmjs.com/package/%40posthog/mcp?utm_source=openai
🏁 Script executed:
python3 - <<'PY'
import io
import re
import tarfile
import urllib.request
url = 'https://registry.npmjs.org/@posthog%2fmcp/-/mcp-0.21.2.tgz'
try:
with urllib.request.urlopen(url, timeout=15) as response:
archive = response.read()
print(f'Fetched exact package tarball: {len(archive)} bytes')
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tf:
hits = []
for member in tf.getmembers():
if not member.isfile() or not member.name.endswith(('.js', '.cjs', '.mjs', '.ts', '.map')):
continue
data = tf.extractfile(member).read()
text = data.decode('utf-8', errors='replace')
for match in re.finditer(r'prepareToolList', text):
hits.append((member.name, text, match.start()))
if not hits:
print('No prepareToolList text in package JS/TS/source maps')
for name, text, pos in hits:
print(f'\n--- {name} around prepareToolList ---')
print(text[max(0, pos-1200):min(len(text), pos+3500)])
except Exception as exc:
print(f'PACKAGE FETCH/READ FAILED: {type(exc).__name__}: {exc}')
PY
printf '\n--- compatibility helper declaration ---\n'
rg -n -C 5 -- 'toJsonSchemaCompat' services/mcp/src services/mcp/tests
printf '\n--- query-llm-traces-list snapshot header and schema ---\n'
nl -ba services/mcp/tests/unit/__snapshots__/tool-schemas/query-llm-traces-list.json | sed -n '1,45p'
printf '\n--- nearest schema serialized length and root fields ---\n'
python3 - <<'PY'
import json
from pathlib import Path
p = Path('services/mcp/tests/unit/__snapshots__/tool-schemas/query-llm-traces-list.json')
s = json.loads(p.read_text())
print('name:', p.name)
print('compact chars:', len(json.dumps(s, separators=(',', ':'), ensure_ascii=False)))
print('root keys:', list(s))
print('root additionalProperties:', s.get('additionalProperties'))
print('root schema:', s.get('$schema'))
PYRepository: PostHog/posthog
Length of output: 42461
🏁 Script executed:
python3 - <<'PY'
import io
import re
import tarfile
import urllib.request
url = 'https://registry.npmjs.org/@posthog%2fmcp/-/mcp-0.21.2.tgz'
try:
with urllib.request.urlopen(url, timeout=15) as response:
archive = response.read()
print(f'Fetched exact package tarball: {len(archive)} bytes')
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tf:
hits = []
for member in tf.getmembers():
if not member.isfile() or not member.name.endswith(('.js', '.cjs', '.mjs', '.ts', '.map')):
continue
text = tf.extractfile(member).read().decode('utf-8', errors='replace')
for match in re.finditer(r'prepareToolList', text):
hits.append((member.name, text, match.start()))
if not hits:
print('No prepareToolList text in package JS/TS/source maps')
for name, text, pos in hits:
print(f'\n--- {name} around prepareToolList ---')
print(text[max(0, pos-1200):min(len(text), pos+3500)])
except Exception as exc:
print(f'PACKAGE FETCH/READ FAILED: {type(exc).__name__}: {exc}')
PY
printf '\n--- compatibility helper declaration ---\n'
rg -n -C 5 -- 'toJsonSchemaCompat' services/mcp/src services/mcp/tests
printf '\n--- query-llm-traces-list snapshot header and schema ---\n'
nl -ba services/mcp/tests/unit/__snapshots__/tool-schemas/query-llm-traces-list.json | sed -n '1,45p'
printf '\n--- nearest schema serialized length and root fields ---\n'
python3 - <<'PY'
import json
from pathlib import Path
p = Path('services/mcp/tests/unit/__snapshots__/tool-schemas/query-llm-traces-list.json')
s = json.loads(p.read_text())
print('name:', p.name)
print('compact chars:', len(json.dumps(s, separators=(',', ':'), ensure_ascii=False)))
print('root keys:', list(s))
print('root additionalProperties:', s.get('additionalProperties'))
print('root schema:', s.get('$schema'))
PYRepository: PostHog/posthog
Length of output: 41829
🏁 Script executed:
printf '%s\n' '--- PostHog client construction ---'
rg -n -C 8 -- 'getPostHogClient|new PostHogMCP' services/mcp/src
printf '%s\n' '--- MCP service dependency versions ---'
rg -n -C 2 -- '"@modelcontextprotocol/sdk"|@modelcontextprotocol/sdk' services/mcp/package.json pnpm-lock.yaml
python3 - <<'PY'
import io
import tarfile
import urllib.request
def package_files(url):
with urllib.request.urlopen(url, timeout=15) as response:
data = response.read()
tf = tarfile.open(fileobj=io.BytesIO(data), mode='r:gz')
return tf
tf = package_files('https://registry.npmjs.org/@posthog%2fmcp/-/mcp-0.21.2.tgz')
wanted = ('context-parameters', 'model-parameters', 'conversation-id', 'output-instructions')
for member in tf.getmembers():
if member.isfile() and member.name.startswith('package/src/') and any(x in member.name for x in wanted):
print(f'\n--- exact @posthog/mcp@0.21.2 {member.name} ---')
print(tf.extractfile(member).read().decode('utf-8', errors='replace'))
PYRepository: PostHog/posthog
Length of output: 42271
🏁 Script executed:
printf '%s\n' '--- PostHog client construction ---'
rg -n -C 8 -- 'getPostHogClient|new PostHogMCP' services/mcp/src
printf '%s\n' '--- MCP SDK package pin ---'
rg -n -C 2 -- '@modelcontextprotocol/sdk' services/mcp/package.json pnpm-lock.yaml
python3 - <<'PY'
import io
import re
import tarfile
import urllib.request
from pathlib import Path
def read_tarball(url):
with urllib.request.urlopen(url, timeout=15) as response:
data = response.read()
return tarfile.open(fileobj=io.BytesIO(data), mode='r:gz')
posthog = read_tarball('https://registry.npmjs.org/@posthog%2fmcp/-/mcp-0.21.2.tgz')
wanted = ('context-parameters', 'model-parameters', 'conversation-id', 'output-instructions')
for member in posthog.getmembers():
if member.isfile() and member.name.startswith('package/src/') and any(x in member.name for x in wanted):
print(f'\n--- exact @posthog/mcp@0.21.2 {member.name} ---')
print(posthog.extractfile(member).read().decode('utf-8', errors='replace'))
lock = Path('pnpm-lock.yaml').read_text()
versions = sorted(set(re.findall(r'@modelcontextprotocol/sdk@([0-9]+\.[0-9]+\.[0-9]+)', lock)))
print('\nResolved @modelcontextprotocol/sdk versions:', versions)
for version in versions:
url = f'https://registry.npmjs.org/@modelcontextprotocol%2fsdk/-/sdk-{version}.tgz'
try:
sdk = read_tarball(url)
except Exception as exc:
print(f'SDK {version} fetch failed: {type(exc).__name__}: {exc}')
continue
for member in sdk.getmembers():
if member.isfile() and 'zod-json-schema-compat' in member.name and member.name.endswith(('.js', '.ts')):
print(f'\n--- @modelcontextprotocol/sdk@{version} {member.name} ---')
print(sdk.extractfile(member).read().decode('utf-8', errors='replace'))
PYRepository: PostHog/posthog
Length of output: 42228
🏁 Script executed:
printf '%s\n' '--- configured MCP analytics client ---'
nl -ba services/mcp/src/lib/posthog/client.ts | sed -n '18,48p'
python3 - <<'PY'
import io
import tarfile
import urllib.request
url = 'https://registry.npmjs.org/@posthog%2fmcp/-/mcp-0.21.2.tgz'
with urllib.request.urlopen(url, timeout=15) as response:
archive = response.read()
tf = tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz')
wanted = (
'package/src/extensions/context-parameters.ts',
'package/src/extensions/analytics-parameters.ts',
'package/src/extensions/constants.ts',
)
for name in wanted:
member = tf.getmember(name)
print(f'\n--- exact @posthog/mcp@0.21.2 {name} ---')
print(tf.extractfile(member).read().decode('utf-8', errors='replace'))
PYRepository: PostHog/posthog
Length of output: 14513
🏁 Script executed:
python3 - <<'PY'
import json
from pathlib import Path
path = Path('services/mcp/tests/unit/__snapshots__/tool-schemas/query-llm-traces-list.json')
schema = json.loads(path.read_text())
compact = lambda value: json.dumps(value, separators=(',', ':'), ensure_ascii=False)
print('snapshot chars:', len(compact(schema)))
print('root keys:', list(schema))
print('root properties:', list(schema.get('properties', {})))
print('raw required:', schema.get('required', []))
schema.pop('$schema', None)
context = "Explain in 15-25 words, in third person, why this tool is called and how it supports the user's goal. For analytics only. You MUST describe only the abstract purpose of the tool call. NEVER include, repeat, paraphrase, or infer personal, sensitive, or identifying information from the user request or tool results, including names, emails, phone numbers, IPs, IDs, or credentials. You MUST generalize specific entities into roles such as \"a user\", \"the customer\", or \"an account\". Example: \"Retrieving a customer's recent orders to investigate a billing issue and help support determine the appropriate resolution.\""
model = 'The exact model identifier you (the assistant) are running as, taken from your system prompt or environment (e.g. \"claude-opus-4-8\", \"gpt-5.2\"). Used for analytics only. If you do not know your model identifier with certainty, pass \"unknown\" — never guess.'
for name, description in (('context', context), ('llm_model', model)):
if name in schema.get('properties', {}):
print('COLLISION:', name)
schema['properties'][name] = {'type': 'string', 'description': description}
schema.setdefault('required', [])
if name not in schema['required']:
schema['required'].append(name)
print('chars after root $schema removal + context + llm_model:', len(compact(schema)))
print('margin relative to 16384:', 16384 - len(compact(schema)))
PY
printf '\n--- local declaration for reachable tool ---\n'
rg -n -C 8 -- 'query-llm-traces-list|queryLlmTracesList' services/mcp/src/tools
printf '\n--- production conversion and response binding ---\n'
nl -ba services/mcp/src/hono/tool-catalog.ts | sed -n '77,93p;143,176p'
nl -ba services/mcp/src/hono/tool-executor.ts | sed -n '145,163p;165,184p'Repository: PostHog/posthog
Length of output: 8623
🏁 Script executed:
printf '%s\n' '--- AssistantTracesQuery declaration ---'
rg -n -C 22 -- 'AssistantTracesQuery' services/mcp/src/tools/generated/ai_observability.ts
python3 - <<'PY'
import io
import re
import tarfile
import urllib.request
from pathlib import Path
lock = Path('pnpm-lock.yaml').read_text()
versions = sorted(set(re.findall(r'(?m)^ zod@([0-9]+\.[0-9]+\.[0-9]+)', lock)))
print('Resolved Zod versions:', versions)
for version in versions:
url = f'https://registry.npmjs.org/zod/-/zod-{version}.tgz'
try:
with urllib.request.urlopen(url, timeout=15) as response:
data = response.read()
tf = tarfile.open(fileobj=io.BytesIO(data), mode='r:gz')
except Exception as exc:
print(f'Zod {version} fetch failed: {type(exc).__name__}: {exc}')
continue
for member in tf.getmembers():
if member.isfile() and member.name.endswith('to-json-schema.ts'):
text = tf.extractfile(member).read().decode('utf-8', errors='replace')
if 'reused' in text or 'toJSONSchema' in text:
print(f'\n--- Zod {version} {member.name}: reuse/default excerpts ---')
lines = text.splitlines()
for i, line in enumerate(lines):
if 'reused' in line or ('target' in line and 'draft' in line):
lo, hi = max(0, i - 3), min(len(lines), i + 5)
print('\n'.join(f'{n+1}: {lines[n]}' for n in range(lo, hi)))
PYRepository: PostHog/posthog
Length of output: 16834
🏁 Script executed:
python3 - <<'PY'
import json
from collections import Counter
from pathlib import Path
p = Path('services/mcp/tests/unit/__snapshots__/tool-schemas/query-llm-traces-list.json')
root = json.loads(p.read_text())
counts = Counter()
def walk(node):
if isinstance(node, dict):
counts.update(node.keys())
for value in node.values():
walk(value)
elif isinstance(node, list):
for value in node:
walk(value)
walk(root)
for key in ('$ref', '$defs', 'definitions', 'prefixItems', 'pattern', 'patternProperties', 'additionalItems', 'anyOf', 'oneOf', 'allOf', 'type'):
print(f'{key}: {counts[key]}')
PY
printf '\n--- schema component declarations ---\n'
nl -ba services/mcp/src/tools/generated/ai_observability.ts | sed -n '2680,2860p']]Repository: PostHog/posthog
Length of output: 390
🏁 Script executed:
python3 - <<'PY'
import json
from collections import Counter
from pathlib import Path
p = Path('services/mcp/tests/unit/__snapshots__/tool-schemas/query-llm-traces-list.json')
root = json.loads(p.read_text())
counts = Counter()
def walk(node):
if isinstance(node, dict):
counts.update(node.keys())
for value in node.values():
walk(value)
elif isinstance(node, list):
for value in node:
walk(value)
walk(root)
for key in ('$ref', '$defs', 'definitions', 'prefixItems', 'pattern', 'patternProperties', 'additionalItems', 'anyOf', 'oneOf', 'allOf', 'type'):
print(f'{key}: {counts[key]}')
PY
printf '\n--- schema component declarations ---\n'
nl -ba services/mcp/src/tools/generated/ai_observability.ts | sed -n '2680,2860p'Repository: PostHog/posthog
Length of output: 11253
🏁 Script executed:
python3 - <<'PY'
import json
from pathlib import Path
root = Path('services/mcp/tests/unit/__snapshots__/tool-schemas')
limit = 16384
context = "Explain in 15-25 words, in third person, why this tool is called and how it supports the user's goal. For analytics only. You MUST describe only the abstract purpose of the tool call. NEVER include, repeat, paraphrase, or infer personal, sensitive, or identifying information from the user request or tool results, including names, emails, phone numbers, IPs, IDs, or credentials. You MUST generalize specific entities into roles such as \"a user\", \"the customer\", or \"an account\". Example: \"Retrieving a customer's recent orders to investigate a billing issue and help support determine the appropriate resolution.\""
model = 'The exact model identifier you (the assistant) are running as, taken from your system prompt or environment (e.g. \"claude-opus-4-8\", \"gpt-5.2\"). Used for analytics only. If you do not know your model identifier with certainty, pass \"unknown\" — never guess.'
conversation = "Pass the exact conversation_id from the server's previous response, unchanged. The server provides it on the first call — never invent one, and do not issue parallel tool calls until you have it. Keep passing the same conversation_id for the rest of the conversation, including after later user messages or on a different task; do not reset it when the user starts a new request."
def compact(value):
return json.dumps(value, separators=(',', ':'), ensure_ascii=False)
rows = []
for path in root.glob('*.json'):
schema = json.loads(path.read_text())
before = len(compact(schema))
advertised = json.loads(json.dumps(schema, ensure_ascii=False))
advertised.pop('$schema', None)
advertised.pop('additionalProperties', None)
blocked = any(key in advertised for key in ('$ref', 'oneOf', 'allOf', 'anyOf'))
props = advertised.setdefault('properties', {})
required = advertised.setdefault('required', [])
if not blocked:
for name, description in (('context', context), ('llm_model', model)):
if name not in props:
props[name] = {'type': 'string', 'description': description}
if name not in required:
required.append(name)
if 'conversation_id' not in props:
props['conversation_id'] = {'type': 'string', 'description': conversation}
after = len(compact(advertised))
rows.append((path.name, before, after))
print('snapshots:', len(rows))
print('baseline over limit:', sum(before >= limit for _, before, _ in rows))
print('projected over limit:', sum(after >= limit for _, _, after in rows))
print('newly over limit:')
for name, before, after in sorted((r for r in rows if r[1] < limit <= r[2]), key=lambda r: r[2], reverse=True):
print(f'{name}: {before} -> {after}')
PYRepository: PostHog/posthog
Length of output: 315
Measure the prepared inputSchema
loadSnapshotTools() measures each Zod schema before production converts it to inputSchema and before prepareToolList() adds analytics fields. The reachable query-llm-traces-list schema measures 15,900 characters here, but the advertised schema is 17,287 characters after production conversion and the default context, llm_model, and conversation_id fields. The current ratchet therefore counts 42 oversized schemas while the advertised schemas contain 44. Claude can silently drop at least this tool.
Measure toMcpInputSchema(tool.schema) after getPostHogClient().prepareToolList() instead.
Suggested fix
- size: JSON.stringify(z.toJSONSchema(tool.schema, { io: 'input', reused: 'inline' })).length,
+ size: JSON.stringify(tool.inputSchema).length,Build tool.inputSchema with toMcpInputSchema(tool.schema) and pass those descriptors through prepareToolList() before this map.
|
Folded into #111786. Same reviewer, so one review instead of two. The ReviewHog finding here is fixed there. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: acdfdca02d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const oversized = tools | ||
| .map((tool) => ({ | ||
| name: tool.name, | ||
| size: JSON.stringify(z.toJSONSchema(tool.schema, { io: 'input', reused: 'inline' })).length, |
There was a problem hiding this comment.
This sizes the raw Zod conversion, but tools/list first applies toMcpInputSchema and then prepareToolList, which injects analytics fields such as context. Consequently, a raw schema just below 16,384 characters can pass here while its final serialized inputSchema exceeds Claude's cap and the tool is silently dropped. The existing exec budget test already demonstrates the required pattern by measuring the post-injection entry.
AGENTS.md reference: AGENTS.md:L238-L240
Useful? React with 👍 / 👎.
| // dashboard-widgets and experiment setup context. Other flag-gated tools (logs-alerts, | ||
| // visual-review, etc.) stay off to keep the surface stable. | ||
| // agent-feedback is always_available and no longer flag-gated, so it appears regardless. | ||
| const featureFlags = { |
There was a problem hiding this comment.
loadSnapshotTools enables only this hard-coded subset and deliberately leaves other production families, including visual-review and logs-alerts, disabled. When one of those omitted flags is enabled for a team, an oversized tool or a nested-description regression remains invisible to both new checks even though that tool is advertised in production. Use the full catalog or enumerate all flag-gated variants for the budget checks rather than reusing the intentionally partial snapshot surface.
Useful? React with 👍 / 👎.
| const items = | ||
| schema.items && typeof schema.items === 'object' ? [schema.items as Record<string, unknown>] : [] | ||
| return [...branches, ...items] |
There was a problem hiding this comment.
schemaChildren follows unions and array items but ignores object-valued additionalProperties. Current snapshots contain nested fields in that shape, such as inputs.additionalProperties.properties in the error-tracking schemas, so adding an undocumented map-value field or removing its description never changes either the numerator or denominator of this coverage metric. Include schema-valued additionalProperties in the recursive children.
Useful? React with 👍 / 👎.
Problem
Claude web and desktop silently drop an MCP tool whose serialized
inputSchemareaches 16,384 characters.In tools mode every tool is registered on its own, and 42 tool schemas are already at or over that size.
Only the
exectool has a budget test, so any schema can grow further with no signal.Nested description coverage has the same gap.
About 56% of properties below the top level have a description, and nothing stops that share from dropping as schemas grow.
Changes
exclude_params,include_paramsorparam_overrides, or addhelp_texton the serializer and regenerate.CLAUDE_REGISTRY_INPUT_SCHEMA_CHAR_LIMITinservices/mcp/src/lib/constants.ts.Size is measured on the schema as the snapshot test already builds it (
JSON.stringifyof the input JSON Schema). Thecontextproperty that the analytics SDK adds at registration is not included, so the real size is slightly larger.The exec budget test keeps its own literal 16,384. That test carries a do-not-modify note, so it was left alone.
How did you test this code?
Automated only: the schema snapshot test passes locally and typecheck is clean.
Test rationale: The checks extend
tests/unit/tool-schema-snapshots.test.ts, which already iterates every tool schema. They catch a new oversized schema or a drop in nested descriptions, which no existing test covers outsideexec. The test runs on the same schemas as the snapshot, so it adds one extra schema build and no new harness.Release status
Docs update
None.
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Agent: Claude Code. Code written by a Claude Sonnet 5.5 (claude-sonnet-5-5) subagent, orchestrated by Claude Opus 5.5 (claude-opus-5-5).
/writing-tests,/reviewing-with-coderabbit,/writing-pr-descriptions.--deep, test-only diff): no findings.