SGLang: Fast and Expressive LLM Inference with RadixAttention (LMSYS blog post)

VENDOR BENCHMARK CLAIM. SGLang authors claim 'up to 5 times higher throughput compared to existing systems, namely Guidance and vLLM' on Llama-7B and Mixtral-8x7B on A10G GPUs. SKEPTICAL CONSIDERATIONS: (1) This is a self-benchmark by the SGLang authors - they chose the workloads, hardware, and baseline configurations. (2) The comparison uses vLLM v0.2.5 (January 2024) - vLLM has had many releases since then with significant performance improvements. (3) The 5x figure is 'up to' - best case, not typical. (4) The benchmark focuses on KV-cache-reuse workloads (few-shot, chat, self-consistency) which is exactly where SGLang's RadixAttention excels; it does not benchmark general throughput where vLLM's continuous batching may dominate. (5) No third-party reproduction of the 5x figure is cited. This is a vendor self-benchmark that should be treated as an upper bound under favorable conditions, not a general performance claim.