inference/research
verifiedhigh · 0.85updated 2026-10-03

vLLM Reaches 25K Total TPS/GPU on Qwen3.5 (GB200 NVL72 PD serving)

blog

Fields

id
sources/bench-nv-vllm-qwen35-25k-tps-blog
Publisher
vLLM project
Kind
blog
Published
2026-08-06
Accessed
2026-10-03
notes
vLLM PD-serving post on a GB200 NVL72 cluster (NVLink72), model Qwen3.5-397B-A17B-NVFP4, ISL/OSL 8192/1024, concurrency swept 64..5120. Headline is a pareto-frontier figure (25K total TPS/GPU); the exact single point is only visible in a PNG chart, so the record uses the prose headline with the concurrency sweep range stated. Recipes, container digest, Dynamo and srt-slurm versions are all published. Authored by vLLM/NVIDIA/Inferact contributors on NVIDIA-run GB200 hardware, so measured_by is vendor-adjacent but recorded as vendor-published.

Notes

vLLM PD-serving post on a GB200 NVL72 cluster (NVLink72), model Qwen3.5-397B-A17B-NVFP4, ISL/OSL 8192/1024, concurrency swept 64..5120. Headline is a pareto-frontier figure (25K total TPS/GPU); the exact single point is only visible in a PNG chart, so the record uses the prose headline with the concurrency sweep range stated. Recipes, container digest, Dynamo and srt-slurm versions are all published. Authored by vLLM/NVIDIA/Inferact contributors on NVIDIA-run GB200 hardware, so measured_by is vendor-adjacent but recorded as vendor-published.

Cross-references

This record references no other records.