Poor performance when using Gemini image output with `streamText`

[wong2]

Description

I tried using the Gemini image output, but found that when used with streamText, the response was very slow (tens of seconds to 1 minute).

To figure out where the issue was, I tried using generateText, and this time I got the result quickly (within seconds).

I also tried to call the API (/v1beta/models/gemini-2.0-flash-exp:streamGenerateContent?alt=sse) with curl directly, which also response very fast. So I guess there's something happening inside streamText

Here's the minimal test code:

Code example

import { google } from '@ai-sdk/google'
import { generateText, streamText } from 'ai'

const model = google('gemini-2.0-flash-exp')

const result = await generateText({
  model,
  providerOptions: {
    google: { responseModalities: ['TEXT', 'IMAGE'] },
  },
  prompt: 'Generate an image of a black cat',
})

console.log('non stream result', result)

const streamResult = streamText({
  model,
  providerOptions: {
    google: { responseModalities: ['TEXT', 'IMAGE'] },
  },
  prompt: 'Generate an image of a black cat',
})

for await (const chunk of streamResult.fullStream) {
  console.log(chunk)
}

AI provider

@ai-sdk/google: 1.2.1

Additional context

Node.js 23

[wong2]

After some debugging, I made a new discovery. This issue seems to be related to EventSourceParserStream. The call happens in createEventSourceResponseHandler.

ai/packages/provider-utils/src/response-handler.ts

Lines 99 to 103 in 3802590

	value: response.body
	.pipeThrough(new TextDecoderStream())
	.pipeThrough(new EventSourceParserStream())
	.pipeThrough(
	new TransformStream<EventSourceMessage, ParseResult<T>>({

Because the SSE response is large when Gemini generates images, EventSourceParserStream takes quite a bit of time to parse it (depending on CPU performance).

[wong2]

I've created an issue in eventsource-parser repo: rexxars/eventsource-parser#19

[lgrammel]

Have a fix that should improve performance for all streams.

[lgrammel]

Please try @ai-sdk/google@1.2.2 once released.

[STRRL]

I could still reproduce this issue with @ai-sdk/google@1.2.2, how about on your side? @wong2

[rexxars]

eventsource-parser@3.0.1 is a ton faster (approx 220x according to my benchmarks) thanks to @wong2's bug report, but I see it is no longer used as of #5351 / @ai-sdk/google@1.2.2.

[wong2]

@lgrammel Can you use the new eventsource-parser in AI SDK?

[lgrammel]

@wong2 no - I replaced it with a internal implementation that is more tailored to our own usage. I love the great work @rexxars has put into the eventsource-parser lib, and it has been invaluable for us - that said one goal of the AI SDK is to reduce dependencies and library size where possible, and we don't need the full event source parser functionality since we only use it in stream transformations, and I prefer having the code in the AI SDK repo vs a dependency in case there a urgent bugs.

[wong2]

@lgrammel The problem still exists in @ai-sdk/google@1.2.3, maybe it's a bit better than before, but it's still slow. Maybe you can apply some of @rexxars' optimizations to your implementation?

[lgrammel]

@wong2 I can take another look; the main performance optimization is not rescanning already scanned chunks which both me and @rexxars implemented