vLLM 0.9.x ROCm optimizations: AITER block-scaled GEMM, fused MoE, MLA
benchmark
Fields
id
sources/amd-vllm-09x-rocmPublisher
AMD ROCm Blogs
Kind
benchmark
Published
2025-06-28
Accessed
2026-10-04
notes
Measured on MI300X with ROCm 6.3.2, vLLM 0.9.x. AITER block-scaled FP8 GEMM: 19.4% request throughput increase, 18.6% output token throughput increase. AITER Biased Group TopK: 4.9% request throughput, 6.1% TTFT reduction. AITER fused MoE and MLA integrations. Docker: rocm/vllm-dev:nightly_0624.
Notes
Measured on MI300X with ROCm 6.3.2, vLLM 0.9.x. AITER block-scaled FP8 GEMM: 19.4% request throughput increase, 18.6% output token throughput increase. AITER Biased Group TopK: 4.9% request throughput, 6.1% TTFT reduction. AITER fused MoE and MLA integrations. Docker: rocm/vllm-dev:nightly_0624.
Cross-references
This record references no other records.