Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving

Serving a long-context chat workload needs to keep prefill and decode from interfering while also holding a KV cache far larger than GPU memory.

Prefill and decode run in separate clusters; the KV cache itself becomes the scheduling currency and lives in a disaggregated pool spread across otherwise-idle CPU DRAM and SSD in the GPU cluster. The KVCache-centric scheduler trades cache-hit rate against latency SLOs to maximise effective throughput.

NOT RECORDED — deliberately null-like text rather than a number. The arXiv abstract states the architecture and the KVCache-centric scheduler but I did not verify a headline multiple with its baseline and hardware settings; the evaluation numbers depend on Moonshot's internal cluster and workload, which are not reproducible by a reader. Per this repo's rule, a bare multiple without baseline and settings is worse than an honest gap. Read the paper's evaluation section for the comparison against their previous FastChat-derived pipeline.

This is the one paper in the slice that is unambiguously in production: the arXiv abstract states Mooncake is the serving platform for Kimi. Venue caveat, important: the arXiv comment field is EMPTY, so the arXiv copy says nothing about venue; Crossref puts this title in ACM Transactions on Storage (DOI 10.1145/3773772), NOT at a systems conference. An OSDI '25 paper shares the name 'Mooncake' but is from Ant Group and is a different system — do not merge them. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. Peer-reviewed venue confirmed as ACM Transactions on Storage 2024 (same). This is an additive track annotation; `venue` is unchanged at 'other' and no status change is implied by the track alone. VENUE RECLASSIFIED 2026-10-10 (crawler/w11-venue-enum): venue 'other' -> 'acm-tos', venue_kind 'journal'. Confirmed by record notes and agreeing publisher/index deposit in data/sources/.