NVIDIA A100, llama.cpp Vulkan (2257758, coopmat2): Llama-2-7B Q4_0 FA tg128 t/s
170.56 tok/s per-request tg128 (FA); ±0.02
Fields
id
benchmarks/a100-vulkan-llama27b-q40-fa-tg128Metric
decode_tok_s
Value
170.56
Unit
tok/s per-request tg128 (FA); ±0.02
Model
Llama-2-7B Q4_0 GGUF
Methodology
THIRD-PARTY MEASURED (llama.cpp discussion #10879, commit 2257758, coopmat2). Llama-2-7B Q4_0, -fa 1, tg128 = 170.56 ± 0.02 t/s. Same variant caveat as the no-FA row.
Measured by
third_party
Reproducible
yes
notes
Same 'Nvidia A100' unspecified-SKU caveat; pp512 FA = 7064.40 t/s.
Notes
Same 'Nvidia A100' unspecified-SKU caveat; pp512 FA = 7064.40 t/s.