Deploying DeepSeek on GB200 NVL72 with PD and Large Scale EP (Part I), SGLang

Fields

id

sources/bench-nv-sglang-gb200-deepseek-pd-blog

URL

https://lmsys.org/blog/2025-06-16-gb200-part-1/

Publisher

SGLang project (LMSYS)

Kind

blog

Published

2025-06-16

Accessed

2026-10-03

notes

SGLang post: DeepSeek-V3/R1 671B on GB200 NVL72, PD disaggregation + large-scale expert parallelism. Headline 7,583 decode tok/s per GPU for 2000-token input, 2.7x vs H100 per GPU, 2.5-3.4x across sequence lengths. 12 of 14 GB200 nodes used for decode, 2 for prefill. Reproduction instructions referenced at sgl-project/sglang issue 7227.

Notes

SGLang post: DeepSeek-V3/R1 671B on GB200 NVL72, PD disaggregation + large-scale expert parallelism. Headline 7,583 decode tok/s per GPU for 2000-token input, 2.7x vs H100 per GPU, 2.5-3.4x across sequence lengths. 12 of 14 GB200 nodes used for decode, 2 for prefill. Reproduction instructions referenced at sgl-project/sglang issue 7227.

Cross-references

This record references no other records.