llama.cpp issue #21517: Q8_0 and IQ4_NL decode 4x slower than Q4_0 on Intel Arc Pro B70
forum
Fields
id
sources/noncuda-llamacpp-21517-sycl-q8-reorderPublisher
ggml-org/llama.cpp
Kind
forum
Published
2026-04-06
Accessed
2026-10-03
notes
2x Arc Pro B70 (BMG-G31, 32GB, 608 GB/s each, PCIe Gen4 x8), llama.cpp 25eec6f, oneAPI 2025.3.3. Full quant sweep on Qwen3.5-27B: Q4_0 23.67 t/s (57% of peak bandwidth), Q4_K_M 20.56 (53%), IQ4_NL 5.85 (14%), Q8_0 4.88 (21%). IQ4_NL and Q4_0 are the same size, so the bottleneck is kernel efficiency, not data volume. Affects SYCL and Vulkan equally and persists across two GPUs. Root cause per PR #21527: Q8_0 was never added to the SYCL reorder optimization framework introduced in PR #12035.
Notes
2x Arc Pro B70 (BMG-G31, 32GB, 608 GB/s each, PCIe Gen4 x8), llama.cpp 25eec6f, oneAPI 2025.3.3. Full quant sweep on Qwen3.5-27B: Q4_0 23.67 t/s (57% of peak bandwidth), Q4_K_M 20.56 (53%), IQ4_NL 5.85 (14%), Q8_0 4.88 (21%). IQ4_NL and Q4_0 are the same size, so the bottleneck is kernel efficiency, not data volume. Affects SYCL and Vulkan equally and persists across two GPUs. Root cause per PR #21527: Q8_0 was never added to the SYCL reorder optimization framework introduced in PR #12035.
Cross-references
This record references no other records.