QuIP: 2-Bit Quantization of Large Language Models With Guarantees

Below 3 bits, weight quantization error explodes because weights are spread-out and the Hessian directions that matter for rounding are badly aligned with the coordinate axes.

Quantization with Incoherence Processing, in two steps: (1) incoherence processing - multiply the weights and the inverse-Hessian by random orthogonal matrices (folded into neighbouring weight matrices so there is no inference-time cost) so weights become near-Gaussian with even magnitudes and the rounding-sensitive directions are unaligned with the axes; (2) adaptive rounding that greedily minimizes a quadratic proxy of the layer-output error, with closed-form clipping thresholds per coordinate. Ships the first theoretical analysis for an LLM-scale quantization algorithm (which also retroactively explains OPTQ).

Paper's own claim (abstract): incoherence preprocessing improves SEVERAL EXISTING quantization algorithms (not just its own), and yields the first LLM quantization methods producing 'viable' results at only TWO bits per weight. COMPRESSION RATIO: 8x at 2 bits/weight. QUALITY COST: the abstract deliberately says 'viable results' rather than 'no loss' - the 2-bit configuration is usable, not clean, and the paper's own framing is that this is the first demonstration rather than a production-grade point.

arXiv ID 2307.13304 verified via /abs/. Venue NeurIPS 2023 from Semantic Scholar. Year = venue year 2023 (v1 2023-07-25). Sufficiently superseded by QuIP# that this is reference material rather than an actionable recipe. arXiv ID and title verified by fetching https://arxiv.org/abs/2307.13304 and reading the citation_title meta tag on 2026-10-03. Venue and year from the Semantic Scholar graph API batch endpoint (paper/arXiv:2307.13304) unless the record's notes say otherwise. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: neurips has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.