inference/research
verifiedhigh · 0.90updated 2026-10-04

LMCache#4463: Silent KV cache corruption on vLLM 0.26 fused/packed layout

forum

Fields

id
sources/src5-gotcha-lmcache-4463-fused-kv-corruption
Publisher
LMCache
Kind
forum
Published
2026-08-08
Accessed
2026-10-04
notes
LMCache in-process connector silently returns wrong KV data for models with fused/packed KV layout (K and V share trailing per-head dimension). No exception or warning. use_gpu_connector_v3=True fails differently with tensor size mismatch. LMCacheMPConnector works as workaround. Demonstrates silent KV cache corruption with layout mismatch.

Notes

LMCache in-process connector silently returns wrong KV data for models with fused/packed KV layout (K and V share trailing per-head dimension). No exception or warning. use_gpu_connector_v3=True fails differently with tensor size mismatch. LMCacheMPConnector works as workaround. Demonstrates silent KV cache corruption with layout mismatch.

Cross-references

This record references no other records.