PD Serving of Qwen3.8-2.4T on GB300 NVL72 (vLLM)
blog
Fields
id
sources/bench-nv-vllm-qwen38-pd-blogPublisher
vLLM project
Kind
blog
Published
2026-09-21
Accessed
2026-10-03
notes
vLLM PD-serving post on a GB300 NVL72 cluster, model Qwen3.8-2.4T-A95B NVFP4, ISL/OSL 8192/1024, concurrency 1..2560. Headline Pareto points are given in prose: 5000 total token throughput per GPU and 180 generated tokens/s/user (interactivity). GSM8K accuracy 95% per configuration. Container image, vLLM revision, Dynamo, srt-slurm and AIPerf versions published; recipes public in NVIDIA/srt-slurm-recipes.
Notes
vLLM PD-serving post on a GB300 NVL72 cluster, model Qwen3.8-2.4T-A95B NVFP4, ISL/OSL 8192/1024, concurrency 1..2560. Headline Pareto points are given in prose: 5000 total token throughput per GPU and 180 generated tokens/s/user (interactivity). GSM8K accuracy 95% per configuration. Container image, vLLM revision, Dynamo, srt-slurm and AIPerf versions published; recipes public in NVIDIA/srt-slurm-recipes.
Cross-references
This record references no other records.