Skip to content

[NVIDIA][GB300] update DSR1 FP8 GB300 TRTLLM image to latest #1767

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cquil11 merged 11 commits into from
Jun 17, 2026
+12 −2
Merged

[NVIDIA][GB300] update DSR1 FP8 GB300 TRTLLM image to latest #1767

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
fa2ebe5
dsr1-fp8-gb300-dynamo-trt: pin image to tensorrtllm-runtime:1.3.0-dev…
xinli-sw Jun 15, 2026
f6f4136
dsr1-fp8-gb300-dynamo-trt: pin image to tensorrtllm-runtime:1.3.0-dev…
xinli-sw Jun 15, 2026
4c442b6
perf changelog update
xinli-sw Jun 15, 2026
2c91c4d
Merge branch 'main' into update/dsr1-gb300-mtp-fp8
xinli-sw Jun 15, 2026
d3b807c
change runner
xinli-sw Jun 15, 2026
9031353
Merge branch 'main' into update/dsr1-gb300-mtp-fp8
xinli-sw Jun 16, 2026
3a9d26c
perf change log
xinli-sw Jun 16, 2026
7c4c645
fix perf change log
xinli-sw Jun 16, 2026
728eb32
Enhance gsm8k accuracy fix description in changelog
xinli-sw Jun 16, 2026
f867bd4
Refine accuracy fix description for DSR1 TRTLLM
xinli-sw Jun 17, 2026
4b8d282
Merge branch 'main' into update/dsr1-gb300-mtp-fp8
xinli-sw Jun 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6587,10 +6587,10 @@ dsr1-fp4-gb300-dynamo-sglang:
dp-attn: true

dsr1-fp8-gb300-dynamo-trt:
image: nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:0.8.1.post2
image: nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.3.0-dev.1-cuda13
model: deepseek-ai/DeepSeek-R1-0528
model-prefix: dsr1
runner: gb300
runner: gb300-nv
precision: fp8
framework: dynamo-trt
multinode: true
Expand Down
10 changes: 10 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3909,3 +3909,13 @@
description:
- "Use the Marlin MoE backend for MiniMax-M3 B200/B300 TP-only vLLM configurations by adding --moe-backend marlin when expert parallelism is disabled."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1809

- config-keys:
- dsr1-fp8-gb300-dynamo-trt
description:

@functionstackx functionstackx Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xinli-sw can u add an new more of an description of what was broken and what was the fix?

- "Fix gsm8k accuracy at 88% instead of 95% for a single point."
- "In previous submission, there was an numeric issue causing accuracy degradation and performance anomaly in some MTP points at certain concurrency."
- "This issue is now fixed in the latest TRTLLM release."
- "Also update all configs for DSR1 TRTLLM FP8 to reflect latest released image usage"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1767