[TomerBN-Nvidia]

What does this PR do?

Updates only the vLLM launch portion of the existing Super 3.5 external server script:

• applies the measured no-MTP TP4/DP1 full-rack server arguments after the existing role arrays, including max-num-seqs=512 per engine;
• restores the measured BF16 KV-cache behavior instead of the eval config's FP8 override;
• launches five independent prefill engines and eleven independent decode engines, then registers all 16 endpoints with the vLLM router;
• switches the router to round_robin and permits 8,192 concurrent requests.

The allocation, evaluation, CSV export, mounts, srun, cleanup, and sbatch behavior are unchanged.

This configuration was measured on a 16-node GB300 rack as five TP4/DP1 prefill engines plus eleven TP4/DP1 decode engines with EP enabled and no MTP, using the vLLM 0.27.1 evaluation image. The zero-error concurrency point delivered 29,382.41 output tok/s (6,144/6,144 successful requests); the absolute-throughput point delivered 31,712.79 output tok/s at concurrency 8,192.

Validation

• bash -n benchmarks/nemotron_3.5_super/sbatch_external_vllm.sh
• shellcheck -e SC2269,SC2086,SC2016 -x benchmarks/nemotron_3.5_super/sbatch_external_vllm.sh (the excluded informational findings predate this change)
• git diff --check

Runtime inputs used for the measured setup

• NUM_PREFILL_NODES=5
• NUM_DECODE_NODES=11
• MODEL=nvidia/NVIDIA-Nemotron-3.5-Super-120B-A12B-SourceOfTruth
• MODEL_REVISION=db2e4b63a1658dff80b6978ba4ffccda5cc8d166
• VLLM_ROUTER_BINARY=/lustre/fsw/portfolios/nemotron/projects/nemotron_compress_dev/common/bin/vllm-router-d2ba586533de
• vLLM image: /lustre/fsw/portfolios/nemotron/projects/nemotron_compress_dev/common/containers/super-vl-evals-v0271-pr38-7c4ca090b.sqsh

cc @bxyu-nvidia — this updates the external vLLM script you shared while preserving its evaluation workflow.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.