AMD RX 7900 XTX 24GB, llama.cpp ROCm: Yi-34B Q4_0 decode tok/s (community measured)
32.5 decode tok/s (single-stream, tg128-class llama-bench run)
Fields
id
benchmarks/comm-rx7900xtx-yi34b-q40-decodeMetric
decode_tok_s
Value
32.5
Unit
decode tok/s (single-stream, tg128-class llama-bench run)
Model
Yi-34B Q4_0 (GGUF)
Methodology
COMMUNITY-MEASURED (llama.cpp discussion, RX 7900 XTX ROCm vs RTX 3090/4090 thread, cited by research-consumer-gpus.md). Yi-34B Q4_0 on the 24GB 7900 XTX (gfx1100, ~960 GB/s GDDR6): 32.5 tok/s decode. The same thread's RTX 3090 reference is ~35 tok/s on the same model, i.e. the AMD card lands roughly 12-15% behind the 3090 on llama.cpp decode despite a slightly wider memory bus - ROCm/kernel maturity, not bandwidth, is the gap.
Measured by
third_party
Reproducible
yes
notes
Gap-fill for the Oct-14 Phase-A ingest: research-consumer-gpus.md cited this 32.5 t/s Yi-34B row but the repo only had the 7B llama.cpp numbers for the 7900 XTX. 34B dense Q4 is the largest model class that fits fully in the 24GB VRAM, so this is the 'largest well-fit' datapoint for the card.
Notes
Gap-fill for the Oct-14 Phase-A ingest: research-consumer-gpus.md cited this 32.5 t/s Yi-34B row but the repo only had the 7B llama.cpp numbers for the 7900 XTX. 34B dense Q4 is the largest model class that fits fully in the 24GB VRAM, so this is the 'largest well-fit' datapoint for the card.