AMD ROCm Blog: Efficient LLM Serving with MTP: DeepSeek V3 and SGLang on AMD Instinct GPUs

Published September 11, 2025 by Chang Liu, Andy Luo, and Anshul Gupta. Details multi-token prediction (NextN module) speculative decoding serving DeepSeek V3 on 8x AMD Instinct MI300X 192GB GPUs with ROCm 6.3.1 and SGLang v0.5.0rc0. Measures end-to-end latency across concurrency levels 1, 2, 4, 8, 16, 32, 64 on Random and ShareGPT datasets. Demonstrates batch dilution effect: speedup decreases from 2.11x at batch/concurrency 1 to 1.25x at batch/concurrency 64 on Random dataset, and 1.80x at batch 4 to 1.36x at batch 64 on ShareGPT.