vLLM ROCm attention backends: 7 backends, 3-path routing, and measured speedups

Fields

id

sources/amd-vllm-rocm-attn-backends

URL

https://blog.vllm.ai/2026/02/27/rocm-attention-backend.html

Publisher

vLLM Blog (AMD AI Framework team)

Kind

benchmark

Published

2026-02-27

Accessed

2026-10-04

notes

Measured speedups on MI300X/MI325X/MI355X with ROCm 7.0, PyTorch 2.9.0a0, vLLM 0.14.0rc2. ROCM_AITER_FA delivers 2.7-4.4x higher TPS than ROCM_ATTN on MHA models. ROCM_AITER_MLA delivers 1.2-1.5x higher TPS than TRITON_MLA on DeepSeek MLA. 3-path routing (prefill/extend/decode) with batch reordering and shuffled KV cache layout gives 15-20% decode throughput improvement. Hardware: EPYC 9654/9575F, 8x MI300X/MI325X/MI355X, Ubuntu 22.04, kernel 5.15.0-116.

Notes

Measured speedups on MI300X/MI325X/MI355X with ROCm 7.0, PyTorch 2.9.0a0, vLLM 0.14.0rc2. ROCM_AITER_FA delivers 2.7-4.4x higher TPS than ROCM_ATTN on MHA models. ROCM_AITER_MLA delivers 1.2-1.5x higher TPS than TRITON_MLA on DeepSeek MLA. 3-path routing (prefill/extend/decode) with batch reordering and shuffled KV cache layout gives 15-20% decode throughput improvement. Hardware: EPYC 9654/9575F, 8x MI300X/MI325X/MI355X, Ubuntu 22.04, kernel 5.15.0-116.

Cross-references

This record references no other records.