AMD Radeon RX 7900 XTX 24GB, llama.cpp ROCm: Llama-2-7B Q4_0 decode tok/s (community measured)
118.9 decode tok/s (TG, single-stream; PP 2424)
Fields
id
benchmarks/comm-rx7900xtx-llama2-7b-q4-decodeMetric
decode_tok_s
Value
118.9
Unit
decode tok/s (TG, single-stream; PP 2424)
Model
Llama-2-7B Q4_0 (GGUF)
Methodology
COMMUNITY-MEASURED (llama.cpp ROCm backend). Llama-2-7B Q4_0: TG 118.9 tok/s (vs RTX 3090 136.1, RTX 4090 162.1), PP 2424. Yi-34B Q4_0 TG 32.5 t/s (vs 3090 ~35). Roughly 12-15% behind the 3090 on llama.cpp decode.
Measured by
third_party
Reproducible
yes
notes
The only non-Nvidia 24GB card worth a look, and only at <=$850 used (~$800-950 used; new remaining stock $1,400-1,800 - AVOID new). 24GB GDDR6, 384-bit, ~960 GB/s. llama.cpp ROCm + Vulkan work; ExLlamaV2 fine; vLLM is the historical weak point (multi-GPU llama.cpp segfaults noted, bitsandbytes/FA2 gaps). ROCm 7.x improved things through 2026. Buy it only if Linux+llama.cpp-only and cheaper than a 3090.
Notes
The only non-Nvidia 24GB card worth a look, and only at <=$850 used (~$800-950 used; new remaining stock $1,400-1,800 - AVOID new). 24GB GDDR6, 384-bit, ~960 GB/s. llama.cpp ROCm + Vulkan work; ExLlamaV2 fine; vLLM is the historical weak point (multi-GPU llama.cpp segfaults noted, bitsandbytes/FA2 gaps). ROCm 7.x improved things through 2026. Buy it only if Linux+llama.cpp-only and cheaper than a 3090.