Intel Arc B580: DeepSeek-R1-Distill-Qwen 14B Q4_K_M decode tok/s, batch 1 (llama.cpp Vulkan)
35.6 tok/s
Fields
id
benchmarks/local-arc-b580-r1-qwen-14b-q4-decode-b1Metric
decode_tok_s
Value
35.6
Unit
tok/s
Model
DeepSeek-R1-Distill-Qwen 14B
Methodology
llama-bench token-generation row tg128, batch size 1, single stream. Command as posted: `llama-bench -m DeepSeek-R1-Distill-Qwen-14B-Q4_K_M.gguf -ngl 99 --threads 8 -p 512,1024 -n 128,256` (model 8.37 GiB, 14.77B params). Intel Arc B580 12GB, Ryzen 9 9950X, Windows 11 24H2, mainline llama.cpp Vulkan backend: tg128 = 35.59 +/- 0.02 tok/s. The reporter's IPEX-LLM SYCL build measured tg128 35.98 +/- 0.73 and a plain SYCL build 19.56 +/- 0.03.
Measured by
third_party
Reproducible
yes
notes
THE 14B DATA POINT FOR THE ARC B580, and the one that shows the 12GB VRAM gate: 14B Q4_K_M is 8.37 GiB of weights, which fits the B580's 12GB with ~3.5GB of headroom for KV, while the next size class up does not. Decode falls from 67.2 tok/s (7B) to 35.6 tok/s (14B) on the same card - roughly the ratio of the two models' weight bytes, as memory-bandwidth-bound decode predicts. Same source and same caveats as the 7B record: single reporter, single card, verbatim llama-bench table, third_party measurement. NO PRICE IS ATTACHED: the B580's cheapest non-cloud price is ~$200 (newegg-refreshed-arc-b580), so on a tok/s-per-dollar read this row and the 7B row are what a buyer of the cheapest accelerator actually gets.
Notes
THE 14B DATA POINT FOR THE ARC B580, and the one that shows the 12GB VRAM gate: 14B Q4_K_M is 8.37 GiB of weights, which fits the B580's 12GB with ~3.5GB of headroom for KV, while the next size class up does not. Decode falls from 67.2 tok/s (7B) to 35.6 tok/s (14B) on the same card - roughly the ratio of the two models' weight bytes, as memory-bandwidth-bound decode predicts. Same source and same caveats as the 7B record: single reporter, single card, verbatim llama-bench table, third_party measurement. NO PRICE IS ATTACHED: the B580's cheapest non-cloud price is ~$200 (newegg-refreshed-arc-b580), so on a tok/s-per-dollar read this row and the 7B row are what a buyer of the cheapest accelerator actually gets.