Deploying DeepSeek with PD Disaggregation and Large-Scale Expert Parallelism on 96 H100 GPUs, SGLang

Fields

id

sources/bench-nv-sglang-h100-deepseek-ep-blog

URL

https://lmsys.org/blog/2025-05-05-large-scale-ep/

Publisher

SGLang project (LMSYS)

Kind

blog

Published

2025-05-05

Accessed

2026-10-03

notes

SGLang post: DeepSeek-V3 671B FP8 on 12 nodes x 8 H100 (Atlas Cloud) with PD disaggregation + large-scale EP. 52.3k input tok/s and 22.3k output tok/s per node for 2000-token inputs. Reproduction instructions at sgl-project/sglang issue 6017.

Notes

SGLang post: DeepSeek-V3 671B FP8 on 12 nodes x 8 H100 (Atlas Cloud) with PD disaggregation + large-scale EP. 52.3k input tok/s and 22.3k output tok/s per node for 2000-token inputs. Reproduction instructions at sgl-project/sglang issue 6017.

Cross-references

This record references no other records.