Low-rank key projection and KV compression: research-only vs deployed

Low-rank key projection and KV compression aim to reduce the KV cache by projecting to a lower-dimensional space. MLA (Multi-head Latent Attention) is the deployed extreme of this: it compresses K and V JOINTLY through one shared down-projection W_DKV, so every head's K and V live in the same d_c-dimensional subspace. MLA is deployed in DeepSeek-V2/V3/V4. GQA/MQA reduce cache by reducing KV head count - GQA is universally deployed, MQA is widely used. Low-rank key projection (separate from MLA) is research-only: no production engine ships a general low-rank KV projection. KV quantization (fp8/int8/int4) is deployed: FlashMLA supports fp8/fp4 KV cache on Blackwell. KV compression via token merging (e.g., PyramidKV, SnapKV, H2O) is research-only: they reduce the number of cached tokens but require a selection pass and are not in production engines.

Deployment status summary: DEPLOYED: MLA (DeepSeek-V2/V3/V4), GQA (most modern models), MQA (many models), KV quantization (fp8 in FlashMLA, int8/int4 in various engines). RESEARCH-ONLY: low-rank key projection (general), token merging (PyramidKV, SnapKV, H2O), NSA, DSA (only in DeepSeek-V3.2 with FlashMLA), YOCO, RetNet, Kimi Linear. The key insight: MLA is the only low-rank compression that is deployed because it preserves quality while achieving 85x cache reduction. GQA/MQA are deployed because they are simple and preserve quality. Token merging and general low-rank projection lose quality and are not in production. SageAttention is NOT in vLLM or SGLang LLM serving path - it is research-only. Sources: arch-deepseek-v2-mla, arch-nsa, arch-retnet, arch-kimi-linear, kern-flashmla-github. === SOURCE DECLARED 2026-10-10 (w10-xref-consistency). === This record's notes cite 'arch-deepseek-v2-mla' by id in the closing Sources line, and that source file exists and was read - but it was never added to this record's sources[], so a claim was carrying a citation the record-level source list did not back. AGENTS.md rule 2 requires every cited source to be resolvable, and it was; the reverse is the one that failed: a source a record cites is a source the record must declare. Added to sources[]. NOTE ON THE NEAR-DUPLICATE: this record also carries attn-deepseek-v2-mla-paper, a SEPARATE source record for the same arXiv 2405.04434, entered by an earlier pass under a different id with a different publisher string ('arXiv' vs 'DeepSeek-AI (arXiv)'). Both are kept rather than merged, because merging sources is a data/sources/ edit outside this wave's write scope; the duplication is recorded here so a future sources pass can consolidate them instead of rediscovering it. No number in this record changed.