Conversation
Signed-off-by: Lin Sun <lin.sun@solo.io>
Signed-off-by: Lin Sun <lin.sun@solo.io>
Deploying agentproxy with
|
| Latest commit: |
eadee09
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://9f128279.agentproxy.pages.dev |
| Branch Preview URL: | https://lin-perf.agentproxy.pages.dev |
linsun
changed the title
artberger
approved these changes
Aug 14, 2026
|
|
||
| ### Visualized results | ||
|
|
||
| {{< reuse-image src="img/blog/agentgateway-vs-litellm-rust-mode/image1.png" width="624px" >}} |
Collaborator
There was a problem hiding this comment.
Image 1 and 8, the y-axis doesn't seem to have numbers, says either 0 reqs/s or K reqs/s, not sure if it is missing a number before k, like maybe for image8 20K and 40K?
| rust: true | ||
| ``` | ||
|
|
||
| I also made sure I was using the latest LiteLLM build available for the test, LiteLLM `1.98.0`, which is newer than `1.94.0`. |
Collaborator
There was a problem hiding this comment.
which is newer than
1.94.0.
not sure why the 1.94 is important to know? maybe say why or just remove
and then also, maybe add what version of agentgateway you were using?
Suggested change
| I also made sure I was using the latest LiteLLM build available for the test, LiteLLM `1.98.0`, which is newer than `1.94.0`. | |
| I also made sure I was using the latest LiteLLM build available for the test, LiteLLM `1.98.0`, and the latest agentgateway version, `1.4.1`. |
| | agentgateway | **35,502 QPS** | **0.863 ms** | **1.644 ms** | **1.972 ms** | | ||
| | LiteLLM (rust: true) | 984 QPS | 32.139 ms | 48.528 ms | 71.451 ms | | ||
|
|
||
| agentgateway handled **over 36× more requests per second** while maintaining sub-2 ms P99 latency. |
Collaborator
There was a problem hiding this comment.
Typically we use sentence-case for agentgateway.
Suggested change
| agentgateway handled **over 36× more requests per second** while maintaining sub-2 ms P99 latency. | |
| Agentgateway handled **over 36× more requests per second** while maintaining sub-2 ms P99 latency. |
Comment on lines
+112
to
+113
| | agentgateway | 199% | 482% | **26 MB** | 34 MB | | ||
| | LiteLLM (rust: true) | 69% | 204% | **2.15 GB** | 2.15 GB | |
Collaborator
There was a problem hiding this comment.
Suggested change
| | agentgateway | 199% | 482% | **26 MB** | 34 MB | | |
| | LiteLLM (rust: true) | 69% | 204% | **2.15 GB** | 2.15 GB | | |
| | agentgateway | 199% | 482% | **26 MiB** | 34 MiB | | |
| | LiteLLM (rust: true) | 69% | 204% | **2.15 GiB** | 2.15 GiB | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
an updated blog using LiteLLM's Rust mode.