QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks

Weight-only PTQ at <=4 bits was not yet Pareto-optimal, and generic random orthogonal incoherence transforms are slow and lack tight guarantees.

Three changes to QuIP: (1) the randomized Hadamard transform replaces the generic random orthogonal matrix - faster (O(n log n), and hardware-friendly butterfly structure) with better theoretical incoherence bounds; (2) VECTOR quantization with hardware-efficient codebooks built on the E8 lattice, which gives the optimal packing of the unit ball in 8 dimensions and exploits the ball-shaped sub-Gaussian distribution incoherent weights have; (3) a final fine-tuning stage to recover fidelity to the original model.

Paper's own claim (abstract): outperforms existing PTQ methods at <=4 bits per weight, 'enables new behaviors in PTQ scaling', and supports fast inference. COMPRESSION RATIO: up to 8x at the 2-bit extreme. QUALITY COST: no numeric perplexity or accuracy delta appears in the abstract, so speedup_reported stays qualitative here rather than inventing a number. OpenReview search independently confirms the ICML 2024 venue.

SLUG RENAMED 2026-10-04 (parent agent): was 'quip-sharp', colliding with the quantization record of the same id, which made every bare-slug cross-reference ambiguous. Inbound refs ([]) all mean the quantization record, so none needed repointing; papers are cited via `sources`, never by bare slug. arXiv ID 2402.04396 verified via /abs/; v2. Venue ICML 2024 confirmed THREE ways: the arXiv comment field ('ICML 2024'), Semantic Scholar, and an OpenReview search returning ICML.cc/2024/Conference. Year = venue year 2024. E8-lattice codebooks need a bespoke decode kernel (quip-sharp); that is the barrier to production use, not the numerics. arXiv ID and title verified by fetching https://arxiv.org/abs/2402.04396 and reading the citation_title meta tag on 2026-10-03. Venue and year from the Semantic Scholar graph API batch endpoint (paper/arXiv:2402.04396) unless the record's notes say otherwise. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: icml has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.