inference/research
verifiedhigh · 0.90updated 2026-10-03

llama.cpp issue #26663: Vulkan token generation 5-7x slower than HIP on gfx1201 for models with hidden_size >= 4096

forum

Fields

id
sources/qgotcha-llamacpp-26663-vulkan-hidden-size-cliff
Publisher
llama.cpp GitHub issues
Kind
forum
Accessed
2026-10-03
notes
Clean llama-bench matrix with effective bandwidth computed from file size: 424 GB/s at hidden 2560 vs 67-98 GB/s at hidden 4096/5120, on a 640 GB/s card. Same files run at full speed on the HIP build, ruling out the GGUF itself.

Notes

Clean llama-bench matrix with effective bandwidth computed from file size: 424 GB/s at hidden 2560 vs 67-98 GB/s at hidden 4096/5120, on a 640 GB/s card. Same files run at full speed on the HIP build, ruling out the GGUF itself.

Cross-references

This record references no other records.