This repository contains a simple benchmark of
eventsource-parser@3.0.1eventsource-parser@3.0.0- Custom implementation of
ai(source)
Based on this issue by @wong2: rexxars/eventsource-parser#19
@rexxars mentioned that the performance of eventsource-parser was greatl in version 3.0.1 improved (comment) based on the above bug report, but that the ai package diverged from the eventsource-parser prior the performance improvements.
eventsource-parser@3.0.1 is roughly 100x faster than eventsource-parser@3.0.0 and ai-sdk-eventsource-parser is roughly 10x slower than eventsource-parser@3.0.1.
$ node benchmark.js
┌─────────┬─────────────────────────────┬─────────────────────┬───────────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ (index) │ Task name │ Latency avg (ns) │ Latency med (ns) │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├─────────┼─────────────────────────────┼─────────────────────┼───────────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0 │ 'ai-sdk-eventsource-parser' │ '38988637 ± 2.44%' │ '38276813 ± 2067041' │ '26 ± 2.28%' │ '26 ± 1' │ 64 │
│ 1 │ 'eventsource-parser-3.0.0' │ '357692048 ± 0.86%' │ '355302188 ± 7421604' │ '3 ± 0.83%' │ '3 ± 0' │ 64 │
│ 2 │ 'eventsource-parser-3.0.1' │ '3281850 ± 1.97%' │ '3390104 ± 149084' │ '307 ± 2.01%' │ '295 ± 12' │ 64 │
└─────────┴─────────────────────────────┴─────────────────────┴───────────────────────┴────────────────────────┴────────────────────────┴─────────┘
The repository contains a recorded response from Gemini. The response can be updated by running
GOOGLE_API_KEY=... node record.js
It updates the recorded/image-of-a-black-cat-chunks.txt file which is used in the benchmark.