ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification

Adaptive KV compression that picks salient tokens degrades sharply at high compression ratios because salient-token identification is inaccurate.

Two parts: (1) a strong quantization BASELINE via CHANNEL-SEPARABLE TOKENWISE quantization - token-wise (not fine-grained group-wise) scales so the scale metadata overhead is far smaller, which is what makes high compression ratios reachable at all; (2) NORMALIZED ATTENTION SCORE as the saliency metric, exploiting the LOWER-TRIANGLE structure of the causal attention matrix, plus an efficient approximation that decouples the scoring from the full attention computation.

Paper's own claim (abstract): prior adaptive approaches 'exhibit significant performance degradation at high compression ratios due to inaccuracies in identifying salient tokens'; ZipCache fixes this with the channel-separable tokenwise scheme and normalized attention score. COMPRESSION RATIO: adaptive/mixed-precision KV (not a fixed bit width - salient tokens keep higher precision), so no single ratio applies. QUALITY COST: no paired accuracy delta in the abstract, so speedup_reported states the qualitative claim only.

SLUG RENAMED 2026-10-04 (parent agent): was 'zipcache', colliding with the quantization record of the same id, which made every bare-slug cross-reference ambiguous. Inbound refs ([]) all mean the quantization record, so none needed repointing; papers are cited via `sources`, never by bare slug. arXiv ID 2405.14256 verified via /abs/. Venue NeurIPS 2024 from Semantic Scholar; year = venue year 2024. Repo already has source qlab-zipcache-paper. Distinct from KVQuant: this one varies PRECISION PER TOKEN rather than using a uniform sub-4-bit format. arXiv ID and title verified by fetching https://arxiv.org/abs/2405.14256 and reading the citation_title meta tag on 2026-10-03. Venue and year from the Semantic Scholar graph API batch endpoint (paper/arXiv:2405.14256) unless the record's notes say otherwise. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: neurips has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.