Intel Arc Pro B70, llama.cpp Vulkan (b863507): Llama-2-7B Q4_0 tg128 t/s
112.02 tok/s per-request tg128 (no FA); ±1.08
Fields
id
benchmarks/b70-vulkan-llama27b-q40-nofa-tg128Metric
decode_tok_s
Value
112.02
Unit
tok/s per-request tg128 (no FA); ±1.08
Model
Llama-2-7B Q4_0 GGUF
Methodology
THIRD-PARTY MEASURED (llama.cpp discussion #10879 Vulkan scoreboard, commit b863507). Llama-2-7B Q4_0, -ngl 99 -fa 0, tg128 = 112.02 ± 1.08 t/s, pp512 = 3379.00 ± 47.92. Per-request, batch-1 — the affordable single-user figure.
Measured by
third_party
Reproducible
yes
notes
Vulkan backend on BMG — the single-user decode path (vs the vLLM batched-serving MLPerf numbers).
Notes
Vulkan backend on BMG — the single-user decode path (vs the vLLM batched-serving MLPerf numbers).