Description
Found an O(N²) in @ai-sdk/mcp's SSE parsing on large MCP tool responses. Fix is up at rexxars/eventsource-parser#28.
I know its a fix on another repo/maintainer but mostly just wanted to give you a heads up. I had already patched it locally to unblock myself, but I'd rather fix it at the source (I don't like to do one-offs haha) so the whole community benefits, especially folks doing tool calls that return larger datasets. Hoping this helps other people hitting the same wall, and happy to find ways to contribute to this community! Let me know if you have any suggestions on it or anything else!
Context
Ran into this while profiling some big MCP tool calls. @ai-sdk/mcp's streamable-HTTP transport pipes responses through EventSourceParserStream (from @ai-sdk/provider-utils, backed by eventsource-parser). If a tool returns a big payload as one SSE event, which is the common shape for MCP servers that dump bulk data as a single content[] block, the incompleteLine + chunk concat inside feed() goes quadratic.
Not the same case as #5862 / rexxars/eventsource-parser#19. That one got fixed in 3.0.1 by optimizing splitLines, but the concat pattern itself was left alone. So streams with one very long line still hit O(N²).
Root cause
In eventsource-parser@3.0.7 feed():
const input = incompleteLine === "" ? chunk : incompleteLine + chunk;
incompleteLine = processLines(input);
When one SSE event spans many chunks (no \n\n until the very end), incompleteLine grows unbounded. Each chunk pays O(total_seen). Across N chunks, O(N²).
My fix at rexxars/eventsource-parser#28 swaps the string accumulator for a fragment array and defers the concat until a terminator actually arrives. Hot path (empty buffer) stays byte-for-byte identical to the original.
Repro
Public mcp-clickhouse + ClickHouse playground, no auth needed:
CLICKHOUSE_HOST=play.clickhouse.com CLICKHOUSE_PORT=443 CLICKHOUSE_USER=play \
CLICKHOUSE_PASSWORD='' CLICKHOUSE_SECURE=true \
CLICKHOUSE_MCP_SERVER_TRANSPORT=http CLICKHOUSE_MCP_BIND_PORT=8765 \
CLICKHOUSE_MCP_AUTH_DISABLED=true \
uvx --from mcp-clickhouse mcp-clickhouse &
Then:
import { createMCPClient } from "@ai-sdk/mcp";
const c = await createMCPClient({
transport: { type: "http", url: "http://127.0.0.1:8765/mcp" },
});
const tools = await c.tools();
const t0 = performance.now();
await tools.run_query.execute(
{ query: "SELECT * FROM uk_price_paid LIMIT 1000000" },
{ toolCallId: crypto.randomUUID(), messages: [] }
);
console.log(`${Math.round(performance.now() - t0)} ms`);Benchmarks
Same query, 1M rows / ~280 MB / one content block:
| version |
total |
rss peak |
eventsource-parser@3.0.6 |
93 s |
3.9 GB |
eventsource-parser@3.0.7 (upstream perf refactor) |
101 s |
2.3 GB |
eventsource-parser@3.0.7 + rexxars/eventsource-parser#28 |
7 s |
2.0 GB |
Scaling before the fix is ~N^1.9 in payload size. After, linear. 14× speedup on this shape, peak memory roughly halves.
AI SDK Version
- @ai-sdk/mcp: 1.0.35
- @ai-sdk/provider-utils: 4.0.23
Code of Conduct
Description
Found an O(N²) in
@ai-sdk/mcp's SSE parsing on large MCP tool responses. Fix is up at rexxars/eventsource-parser#28.I know its a fix on another repo/maintainer but mostly just wanted to give you a heads up. I had already patched it locally to unblock myself, but I'd rather fix it at the source (I don't like to do one-offs haha) so the whole community benefits, especially folks doing tool calls that return larger datasets. Hoping this helps other people hitting the same wall, and happy to find ways to contribute to this community! Let me know if you have any suggestions on it or anything else!
Context
Ran into this while profiling some big MCP tool calls.
@ai-sdk/mcp's streamable-HTTP transport pipes responses throughEventSourceParserStream(from@ai-sdk/provider-utils, backed byeventsource-parser). If a tool returns a big payload as one SSE event, which is the common shape for MCP servers that dump bulk data as a singlecontent[]block, theincompleteLine + chunkconcat insidefeed()goes quadratic.Not the same case as #5862 / rexxars/eventsource-parser#19. That one got fixed in 3.0.1 by optimizing
splitLines, but the concat pattern itself was left alone. So streams with one very long line still hit O(N²).Root cause
In
eventsource-parser@3.0.7feed():When one SSE event spans many chunks (no
\n\nuntil the very end),incompleteLinegrows unbounded. Each chunk pays O(total_seen). Across N chunks, O(N²).My fix at rexxars/eventsource-parser#28 swaps the string accumulator for a fragment array and defers the concat until a terminator actually arrives. Hot path (empty buffer) stays byte-for-byte identical to the original.
Repro
Public
mcp-clickhouse+ ClickHouse playground, no auth needed:Then:
Benchmarks
Same query, 1M rows / ~280 MB / one content block:
eventsource-parser@3.0.6eventsource-parser@3.0.7(upstream perf refactor)eventsource-parser@3.0.7+ rexxars/eventsource-parser#28Scaling before the fix is ~N^1.9 in payload size. After, linear. 14× speedup on this shape, peak memory roughly halves.
AI SDK Version
Code of Conduct