inference/research
verifiedhigh · 0.85updated 2026-10-03

LMCache documentation

repo

Fields

id
sources/pd-lmcache-docs
Publisher
LMCache
Kind
repo
Accessed
2026-10-03
notes
Positions LMCache as 'a KV cache management layer for LLM inference', vendor-neutral, and explicitly as a standalone daemon process so 'KV cache will not be lost even if the inference engine crashes (i.e., no fate-sharing with engines)'. KV-transfer path statement: 'PD disaggregation and KV transfer: Support KV cache transfer from prefill workers to decode workers over NVLink, RDMA, or TCP through transport layers such as NIXL.' Storage backends listed: CPU RAM, local disk (SSD), Redis/Valkey, Mooncake, InfiniStore, S3-compatible object storage, NIXL, GDS. Also documents non-prefix KV reuse via CacheBlend selective recompute, which is the deliberate quality-vs-latency trade against exact-prefix reuse.

Notes

Positions LMCache as 'a KV cache management layer for LLM inference', vendor-neutral, and explicitly as a standalone daemon process so 'KV cache will not be lost even if the inference engine crashes (i.e., no fate-sharing with engines)'. KV-transfer path statement: 'PD disaggregation and KV transfer: Support KV cache transfer from prefill workers to decode workers over NVLink, RDMA, or TCP through transport layers such as NIXL.' Storage backends listed: CPU RAM, local disk (SSD), Redis/Valkey, Mooncake, InfiniStore, S3-compatible object storage, NIXL, GDS. Also documents non-prefix KV reuse via CacheBlend selective recompute, which is the deliberate quality-vs-latency trade against exact-prefix reuse.

Cross-references

This record references no other records.