inference/research
verifiedhigh · 0.85updated 2026-10-03

llama.cpp issue #27734: ~78% Vulkan decode cliff at 131072 context from suballocation fragmentation

forum

Fields

id
sources/noncuda-llamacpp-27734-vulkan-suballoc-cliff
Publisher
ggml-org/llama.cpp
Kind
forum
Published
2026-08-26
Accessed
2026-10-03
notes
RX 7900 XTX / Win11 / build 10603: 40.4 tok/s at ctx 65536 and 40.8 at 98304, then 8.9 tok/s at ctx 131072; GGML_VK_SUBALLOCATION_BLOCK_SIZE=4294967296 restores 40.3 tok/s. GGML_VK_MEMORY_LOGGER=1 shows 21.55 GiB fully in VRAM split into 30+ ~1018 MiB VkBuffers. A second reporter reproduced the fragmentation on an RTX 5070 Ti; a maintainer-side comment (tczank) reproduced the fragmentation on Linux/RADV but NOT the throughput cliff, so the causal claim is contested.

Notes

RX 7900 XTX / Win11 / build 10603: 40.4 tok/s at ctx 65536 and 40.8 at 98304, then 8.9 tok/s at ctx 131072; GGML_VK_SUBALLOCATION_BLOCK_SIZE=4294967296 restores 40.3 tok/s. GGML_VK_MEMORY_LOGGER=1 shows 21.55 GiB fully in VRAM split into 30+ ~1018 MiB VkBuffers. A second reporter reproduced the fragmentation on an RTX 5070 Ti; a maintainer-side comment (tczank) reproduced the fragmentation on Linux/RADV but NOT the throughput cliff, so the causal claim is contested.

Cross-references

This record references no other records.