RX 7900 XTX llama2-7B Q4_0 token-gen (ROCm)
118.9 tok/s TG batch-1
Fields
id
benchmarks/comm-rx-7900-xtx-llama2-7b-q4-tgMetric
decode_tok_s
Value
118.9
Unit
tok/s TG batch-1
Model
Llama-2-7B
Methodology
llama.cpp llama-bench ROCm backend, Q4_0 GGUF, batch-1 decode (llama.cpp discussion measured)
Measured by
third_party
Thermal state
unknown
notes
vs 3090 136.1, 4090 162.1 - ~12-15% behind 3090. PP 2,424. Yi-34B Q4_0 32.5 t/s.
Notes
vs 3090 136.1, 4090 162.1 - ~12-15% behind 3090. PP 2,424. Yi-34B Q4_0 32.5 t/s.