Google Research blog: TurboQuant extreme KV compression

Amir Zandieh and Vahab Mirrokni, Google Research, 2026-03-24. Vendor explainer for TurboQuant (arXiv:2504.19874, ICLR 2026), PolarQuant and QJL. States that TurboQuant reaches perfect downstream needle-in-a-haystack results while reducing KV memory by a factor of at least 6x, that it quantizes the KV cache to 3 bits with no training and no accuracy compromise on Gemma and Mistral, and that 4-bit TurboQuant computes attention logits up to 8x faster than 32-bit unquantized keys on H100, relative to a JAX baseline. The LongBench chart is captioned on Llama-3.1-8B-Instruct with bit-widths in brackets; the numeric bar heights are in the figure, not in the prose. Vendor claim: mark numbers taken from this page vendor_claim.