DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads (Yao et al., MIT Han Lab)

Head-wise KV sparsity (retrieval vs streaming heads); an alternative to quantizing the KV cache, and combinable with it.