Extreme Compression of Large Language Models via Additive Quantization

2-3 bit weight quantization of LLMs: existing methods degrade badly below 3 bits because uniform scalar grids cannot represent the weight distribution.

Multi-Codebook Quantization (information-retrieval Additive Quantization) adapted to LLM weights: represent each weight vector as a SUM of selected codeword entries from several learned codebooks rather than a scalar on a uniform grid. Two innovations over classic AQ: the codebook assignment is INPUT-ADAPTIVE (a small network picks codewords per block of inputs rather than per weight in isolation), and codebook parameters are jointly optimized ACROSS transformer blocks instead of per-matrix. Ships fast GPU and CPU decode kernels.

Paper's own claim (abstract): 'the first scheme that is Pareto optimal in terms of accuracy-vs-model-size when compressing to less than 3 bits per parameter', significantly improving on all known schemes in the 2-bit regime; and the fast implementations 'match or outperform optimized FP16 implementations for speed, while executing in a much smaller memory footprint'. COMPRESSION RATIO: up to ~8x at 2 bits/weight. QUALITY COST: the outperform-FP16-for-speed claim is the notable part - at 2-3 bits the decoder can be FASTER than FP16, not just smaller, because bandwidth drops faster than arithmetic rises.

SLUG RENAMED 2026-10-04 (parent agent): was 'aqlm', colliding with the quantization record of the same id, which made every bare-slug cross-reference ambiguous. Inbound refs (['scheme']) all mean the quantization record, so none needed repointing; papers are cited via `sources`, never by bare slug. arXiv ID 2401.06118 verified via /abs/; v4. Venue ICML 2024 confirmed by the arXiv comment field ('ICML, 2024') and Semantic Scholar. Year = venue year 2024. adoption=in-upstream-engine: HuggingFace transformers ships AqlmConfig (src/transformers/utils/quantization_config.py, QuantizationMethod.AQLM), but vLLM's quantization registry does NOT list aqlm - so the support is upstream in transformers, not in the main serving engine. arXiv ID and title verified by fetching https://arxiv.org/abs/2401.06118 and reading the citation_title meta tag on 2026-10-03. Venue and year from the Semantic Scholar graph API batch endpoint (paper/arXiv:2401.06118) unless the record's notes say otherwise. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: icml has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.