Pipeline Parallelism in SGLang: Scaling to Million-Token Contexts and Beyond (LMSYS Blog)

SGLang Chunked Pipeline Parallelism (CPP) with async P2P communication and dynamic chunking. DeepSeek-V3.1 on H20: PP4 TP8 yields 3.31x prefill throughput vs TP8 (chunk size 12K), outperforming TP32 by 30.5%. Up to 67.9% TTFT reduction with 82.8% strong scaling efficiency. Qwen3-235B-A22B-FP8 on H20 with PP8: up to 81% TTFT reduction for ultra-long prompts. Compatible with PD disaggregation and HiCache.