vLLM Reaches 25K Total TPS/GPU on Qwen3.5 (GB200 NVL72 PD serving)
blog
Fields
id
sources/bench-nv-vllm-qwen35-25k-tps-blogPublisher
vLLM project
Kind
blog
Published
2026-08-06
Accessed
2026-10-03
notes
vLLM PD-serving post on a GB200 NVL72 cluster (NVLink72), model Qwen3.5-397B-A17B-NVFP4, ISL/OSL 8192/1024, concurrency swept 64..5120. Headline is a pareto-frontier figure (25K total TPS/GPU); the exact single point is only visible in a PNG chart, so the record uses the prose headline with the concurrency sweep range stated. Recipes, container digest, Dynamo and srt-slurm versions are all published. Authored by vLLM/NVIDIA/Inferact contributors on NVIDIA-run GB200 hardware, so measured_by is vendor-adjacent but recorded as vendor-published.
Notes
vLLM PD-serving post on a GB200 NVL72 cluster (NVLink72), model Qwen3.5-397B-A17B-NVFP4, ISL/OSL 8192/1024, concurrency swept 64..5120. Headline is a pareto-frontier figure (25K total TPS/GPU); the exact single point is only visible in a PNG chart, so the record uses the prose headline with the concurrency sweep range stated. Recipes, container digest, Dynamo and srt-slurm versions are all published. Authored by vLLM/NVIDIA/Inferact contributors on NVIDIA-run GB200 hardware, so measured_by is vendor-adjacent but recorded as vendor-published.
Cross-references
This record references no other records.