A single llama.cpp commit regressed Vulkan Q4_K_M decode speed with no prompt-processing change
[minor] kernel
Fields
id
gotchas/vulkan-q4km-speed-regressionClass
kernel
Severity
minor
Concepts
vulkan
Symptom
Bisected to first bad commit adc5dd92e8aea98f5e7ac84f6e1bc15de35130b5 on Windows 11 with dual Radeon PRO W7800, Vulkan SDK 1.3.283, Qwen2.5-14B-Instruct-Q4_K_M at -ngl 99. Decode (tg) fell 42.12 -> 36.38 tok/s (23.74 -> 27.49 ms/token) while prompt processing and sampling were flat (103.62 vs 103.04 tok/s pp; 12410 vs 12356 tok/s sampling).
Root cause
A change in the Vulkan k-quant decode path introduced at adc5dd92. The reporter observed degradation across several architectures with Q4_K_M (Qwen2.5 7B, Qwen2.5 14B, Command R v01, with an unconfirmed report for Phi 4), and not on smaller models. The specific code change was not identified in the issue.
Workaround
Stay on a known-good commit or bisect around adc5dd92 for your own build; verify with llama-bench comparing identical seeds and flags across commits rather than comparing to numbers from another machine.
notes
Useful methodological point: pp flat + tg regressed localizes the change to the decode-specific kernel, not the general backend. Reported under label bug-unconfirmed / stale upstream, so treat the exact scope as narrow evidence rather than a general Vulkan-k-quant verdict.
Notes
Useful methodological point: pp flat + tg regressed localizes the change to the decode-specific kernel, not the general backend. Reported under label bug-unconfirmed / stale upstream, so treat the exact scope as narrow evidence rather than a general Vulkan-k-quant verdict.