ESS: An Offload-Centric Latent-Cache Management Architecture for DeepSeek-V3.2-Exp

Even after DeepSeek Sparse Attention improves overall throughput, the DECODE stage of prefill-decode disaggregation remains a bottleneck: the Latent-Cache grows LINEARLY with sequence length while GPU memory is fixed, so feasible batch size is pinned by cache size and decode throughput collapses with it.

An OFFLOAD-CENTRIC design that SELECTIVELY OFFLOADS THE LATENT-CACHE TO CPU MEMORY while keeping latency-critical components resident on GPU. The selective part is the contribution: offloading everything is a latency disaster and offloading nothing does nothing, so the paper's result is a partition of the cache - which parts go to host and which stay on device - chosen for latency sensitivity. Because the offload frees GPU memory, BATCH-SIZE SCALING DECOUPLES FROM GPU MEMORY, which is the actual bottleneck being relieved: it is not a faster kernel, it is a larger feasible batch.

Paper's own claim, and READ THE METHOD BEFORE THE NUMBER: 'high-fidelity SIMULATIONS' show 69.4% throughput improvement at 32K context length and up to 123% at 128K. This is a SIMULATED result, not a measured one on hardware, and the percentage is relative to a DSA-enabled disaggregated baseline. The repo's own benchmarking rules (AGENTS.md, 'Adding a benchmark record') require exact engine build, model, quantization, batch size and lengths for a throughput claim; a simulation cannot supply all of those, so this number is recorded with its method attached and must not be compared against measured tok/s from any other record in this repo.

RECORDED AS arxiv-preprint WITH venue_track 'unknown': arXiv 2512.10576's comment field is EMPTY, there is no journal_ref, and no proceedings entry or OpenReview record was found for this title. WHY IT IS IN THE BATCH DESPICALLY OF BEING A DECEMBER-2025 PREPRINT WITH SIMULATED NUMBERS: it is the clearest published statement of the constraint that binds any disaggregated deepseek-V3.2 deployment in late 2026 - that after sparse attention, the binding limit is the DECODE stage, and that the limit is cache capacity rather than compute. That reframes prefill/decode disaggregation: disaggregation does not remove the decode cache bottleneck, it relocates it, and the relocation has to be solved with an offload policy rather than with more decode GPUs. The 123%-at-128K figure is also the honest counterweight to the repo's existing PowerInfer/InfiniGen offload records, which are measured on consumer GPUs - ESS's point is that offload pressure now also exists on datacenter parts running MoE with a latent cache. Filed as an inference-serving system record, not as an engine record, since no released engine artifact is claimed. VENUE RE-CHECKED 2026-10-05 (wave 6, agent w6-papers) - STILL NO VENUE, and the wave that could not verify this is now three waves old, so the negative is re-run rather than inherited. Wave 4 reached the same answer from the OpenReview index; this pass adds two corpora it did not sweep. ACL Anthology search for this title returns no paper page of its own - only unrelated PDFs that CITE it, which is the strongest available evidence that no ACL-family proceedings entry exists. Crossref bibliographic query for the exact title returns no proceedings-article or journal-article of this paper either; the hits are other papers on adjacent topics (a different elastic KV paper, a different memory-disaggregated caching system, a different concurrent-LoRA serving system), none of them this work. So the answer is arxiv-preprint with venue_track 'unknown', and it is a POSITIVE finding rather than an unresearched gap: this is a widely cited paper with no peer-reviewed venue, which is a fact about the literature that a reader needs to know. The claim that matters for a deployment reader is the one already in the record above - the binding constraint it names - that after sparse attention the limit is DECODE-stage cache capacity rather than compute - is a statement about deployed DeepSeek-V3.2 systems and does not depend on the paper having been refereed. arxiv_id and every other field unchanged; only `updated` moved, because `updated` is the date the record was last verified against its sources.