Intel Arc B580: DeepSeek-R1-Distill-Qwen 7B Q4_K_M decode tok/s, batch 1 (llama.cpp Vulkan)

Fields

id

benchmarks/local-arc-b580-r1-qwen-7b-q4-decode-b1

Metric

decode_tok_s

Value

67.2

Unit

tok/s

Model

DeepSeek-R1-Distill-Qwen 7B

Methodology

llama-bench token-generation row tg128, batch size 1 (llama-bench -n 128 is a single-stream generation run). Command as posted: `llama-bench -m DeepSeek-R1-Distill-Qwen-7B-Q4_K_M.gguf -t 16 -p 512,1024,2048 -n 128,256,512`. Intel Arc B580 12GB, Ryzen 9 9950X, 48GBx4 DDR5-6200, Windows 11 24H2, on the mainline llama.cpp Vulkan backend: tg128 = 67.21 +/- 2.23 tok/s. The same reporter's IPEX-LLM portable SYCL build measured tg128 66.48 +/- 0.22, and a plain mainline SYCL build 36.40 +/- 0.08 - i.e. Vulkan and IPEX-LLM SYCL are within noise here, and both roughly double the plain-SYCL figure.

Measured by

third_party

Reproducible

yes

notes

FIRST DECODE FIGURE IN THE CORPUS FOR THE ARC B580, the cheapest card in the priced set ($200-330). Community measurement posted in an Intel IPEX-LLM GitHub issue (TabNahida), with the verbatim llama-bench table; measured_by is 'third_party' because the schema has no 'community' member. THE HEADLINE IS THE BACKEND SPREAD: Vulkan 67.2 and IPEX-LLM SYCL 66.5 differ by ~1%, while mainline SYCL is 36.4 - the same silicon nearly doubles depending on which Intel software path you build, which is the concrete form of gotcha intel-arc-sycl-vulkan-underutilization. In the SAME thread another user (Sketchfellow) reported the OPPOSITE ordering on an Arc A770 Mobile (IPEX 48.9 > SYCL 24.1 > Vulkan 40.7 at 8B), so the backend ranking is NOT portable across Intel parts - do not generalise this B580 ordering to other Arc SKUs. Context caveat stated in-thread: Vulkan generation degrades faster than SYCL/IPEX as context grows. n=1 reporter, one card; treat as an order-of-magnitude anchor, not a spec.

Notes

FIRST DECODE FIGURE IN THE CORPUS FOR THE ARC B580, the cheapest card in the priced set ($200-330). Community measurement posted in an Intel IPEX-LLM GitHub issue (TabNahida), with the verbatim llama-bench table; measured_by is 'third_party' because the schema has no 'community' member. THE HEADLINE IS THE BACKEND SPREAD: Vulkan 67.2 and IPEX-LLM SYCL 66.5 differ by ~1%, while mainline SYCL is 36.4 - the same silicon nearly doubles depending on which Intel software path you build, which is the concrete form of gotcha intel-arc-sycl-vulkan-underutilization. In the SAME thread another user (Sketchfellow) reported the OPPOSITE ordering on an Arc A770 Mobile (IPEX 48.9 > SYCL 24.1 > Vulkan 40.7 at 8B), so the backend ranking is NOT portable across Intel parts - do not generalise this B580 ordering to other Arc SKUs. Context caveat stated in-thread: Vulkan generation degrades faster than SYCL/IPEX as context grows. n=1 reporter, one card; treat as an order-of-magnitude anchor, not a spec.

Cross-references

Intel Arc B580: DeepSeek-R1-Distill-Qwen 7B Q4_K_M decode tok/s, batch 1 (llama.cpp Vulkan)

Fields

Notes

Cross-references

Accelerators (1)

Engine (1)

Format (1)

Interconnect (1)