inference/research
verifiedhigh · 0.90majorupdated 2026-10-03

On RDNA, models with head_dim != 128 silently fall back to Triton paged attention and decode collapses at long context

[major] kernel

Fields

id
gotchas/rdna-triton-paged-attn-decode-cliff
Class
kernel
Severity
major
Concepts
gfx1100
Symptom
Qwen3.6-27B on gfx1100 decodes at 12.1 tok/s at 518 tokens of context but 4.2 tok/s at 32K. Linear-attention layers keep constant cost; only the 16 full-attention layers degrade. Profiling puts essentially all the growth in kernel_paged_attention_2d, which grows 28.3x from 356.664 us at ctx 1024 to 10095.188 us at ctx 32768.
Root cause
use_rocm_custom_paged_attention in vllm/platforms/rocm.py gates the custom paged-attention kernel on the RDNA branch with head_size == 128 and block_size == 16. This model reports head_dim=256 with a measured KV block size of 784, so the custom kernel is never eligible and the Triton 2D fallback runs instead.
Workaround
Check head_dim against 128 and the measured KV block size against 16 before assuming an RDNA card is slow. The reporter verified the upstream split-KV decode kernel fixes it on gfx1100, gfx1151 and gfx1201, because the on_gfx12x() gate was widened inside the PR itself on 2026-08-10; nothing local is needed on current versions.
notes
Do not misattribute this to the linear/SSM layers - in the reported profile their decode kernels cost the same at 1K and 32K (0.95x), which is what the architecture promises. Verify on the version you run before concluding the fix landed.

Notes

Do not misattribute this to the linear/SSM layers - in the reported profile their decode kernels cost the same at 1K and 32K (0.95x), which is what the architecture promises. Verify on the version you run before concluding the fix landed.

Cross-references