Add new FW (TRT) and precision support

[kedarpotdar-nv]

Overview

This PR adds TensorRT-LLM (TRT-LLM) as a new inference framework for LLaMA 70B benchmarking on NVIDIA H200 and B200 GPUs, alongside the existing vLLM framework. This enables direct performance comparison between vLLM and TRT-LLM on the same hardware.

Key Features

• Multi-framework support: vLLM and TRT-LLM for LLaMA 70B
• Precision support: FP8 (default) and FP4 (future-ready)
• Unified plotting: All frameworks and hardware on single performance plots
• Framework-specific Docker images: Prevents image conflicts between frameworks
• Clean job naming: Clear identification in GitHub Actions UI

🔧 Core Workflow Updates

.github/workflows/benchmark-tmpl.yml

• Added framework and precision as required inputs
• Updated RESULT_FILENAME to include framework and precision
• Modified result processing to pass framework and precision to process_result.py
• Fixed job naming to prevent duplication
• Added hardware extraction logic for proper result processing

.github/workflows/70b-tmpl.yml

• Added bmk-h200-trt and bmk-b200-trt jobs for TRT-LLM
• Configured TRT-LLM jobs with nvidia/tensorrt-llm Docker image
• Set precision to empty string for FP8 (default)
• Updated all jobs to include framework and precision inputs

🚀 New Benchmark Scripts

benchmarks/70b_h200_trt_slurm.sh and benchmarks/70b_b200_trt_slurm.sh

• TRT-LLM server setup for H200 using mpirun trtllm-serve
• Inline llama-config.yml configuration
• Client benchmarking with benchmark_serving.py

🔄 Launcher Script Updates

Updated SLURM Launchers
runners/launch_h200-nv.sh
runners/launch_h200-cw.sh
runners/launch_h200-nb.sh
runners/launch_b200-nv.sh
Key improvements:

• Framework-specific SQSH file naming to prevent Docker image conflicts
• Dynamic script selection based on framework (VLLM/SGLang use base scripts, TRT uses _trt scripts)
• Proper MODEL_CODE environment variable passing to containers
• Framework reset logic for VLLM and SGLang to use default script names

📊 Result Processing & Visualization

utils/process_result.py

• Added framework and precision command-line arguments
• Updated output data structure to include framework and precision
• Default precision handling (empty string → 'fp8')

utils/plot_perf.py

• Added distinct colors for TRT-LLM results:
  h200-trt: dark green
  b200-trt: gray
• Unified plotting: all frameworks and hardware on single plots
• Updated plot titles and legend handling
• Model-specific plot generation

🧪 Testing Configuration

.github/workflows/workflow-scheduler.yml

• Commented out concurrency and schedule blocks for manual testing
• Disabled DSR1 jobs as requested

[kimbochen]

Thank you for the PR. lgtm