RX 7900 XTX (GPU0) llama.cpp ROCm 7.1.1: Llama-2-7B Q4_0 FA-off pp512
3487.15 tok/s (pp512, per-request, batch 1, one GPU)
Fields
id
benchmarks/rocm711-rx7900xtx-gpu0-llama27b-q40-nofa-pp512Metric
prefill_tok_s
Value
3,487.15
Unit
tok/s (pp512, per-request, batch 1, one GPU)
Model
llama-2-7b Q4_0 (3.56 GiB, 6.74B)
Methodology
Same run and stack as rocm711-rx7900xtx-gpu0-llama27b-q40-nofa-tg128. llama-bench pp512, ngl 99, fa 0, device 0. Reported 3487.15 +/- 39.07. Per-request prompt processing, not aggregate. Device 1 scored 3550.31 +/- 52.29 on the same test.
Measured by
third_party
Reproducible
yes
Thermal state
unknown
notes
Flash attention on the same card raised pp512 to 3785.07 +/- 32.27 and tg128 to 126.09 +/- 0.28. Those cells are not recorded separately; the no-FA cell is the one comparable to the #15021 scoreboard.
Notes
Flash attention on the same card raised pp512 to 3785.07 +/- 32.27 and tg128 to 126.09 +/- 0.28. Those cells are not recorded separately; the no-FA cell is the one comparable to the #15021 scoreboard.