KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization (Hooper et al.)
paper
Fields
id
sources/qlab-kvquant-paperPublisher
arXiv
Kind
paper
Published
2024-01-31
Accessed
2026-10-03
notes
v6. Per-channel key quant, pre-RoPE key quant, non-uniform datatypes, per-vector dense-and-sparse; <0.1 PPL degradation at 3-bit.
Notes
v6. Per-channel key quant, pre-RoPE key quant, non-uniform datatypes, per-vector dense-and-sparse; <0.1 PPL degradation at 3-bit.
Cross-references
This record references no other records.