LQER: Low-Rank Quantization Error Reconstruction for LLMs

Low-bit quantization error in LLMs is highly structured, but recovering it needs either expensive gradient-based optimization or irregular memory access patterns.

After quantizing W to 4 bits as W_hat, model the residual W - W_hat with a rank-r correction driven by an ACTIVATION-INDUCED SCALE MATRIX: rather than reconstructing the weight error directly, LQER constructs a scale matrix from calibration activations and shapes the singular value spectrum of the quantization error so a rank-r approximation captures it. The computation pattern uses dense GEMMs only - it explicitly avoids the Scatter/Gather instructions that irregular outlier-buffer schemes need - and needs no knowledge distillation, grid search or iterative gradient optimization.

Paper's own claim (abstract): 'nearly-lossless W4A8 quantization' on various LLMs and six downstream tasks, using 1.36x FEWER HARDWARE RESOURCES than the leading state-of-the-art method (that 1.36x is a resource-count comparison, not a latency measurement). COMPRESSION RATIO: 4-bit weights with 8-bit activations, i.e. ~4x on weights. QUALITY COST: 'nearly lossless' on the six tasks named, unquantized in the abstract. The dense-GEMM-only claim is the practically important part - it is what makes the method kernel-friendly.

SLUG RENAMED 2026-10-04 (parent agent): was 'lqer', colliding with the quantization record of the same id, which made every bare-slug cross-reference ambiguous. Inbound refs (['scheme']) all mean the quantization record, so none needed repointing; papers are cited via `sources`, never by bare slug. arXiv ID 2402.02446 verified via /abs/; v3. Venue ICML 2024 confirmed by the arXiv comment field ('Accepted at ICML2024') and Semantic Scholar. Year = venue year 2024. The repo already has source qlab-lqer-paper. The '1.36x fewer hardware resources' figure is a resource-ratio claim and should NOT be reported as a speedup without re-deriving it. arXiv ID and title verified by fetching https://arxiv.org/abs/2402.02446 and reading the citation_title meta tag on 2026-10-03. Venue and year from the Semantic Scholar graph API batch endpoint (paper/arXiv:2402.02446) unless the record's notes say otherwise. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: icml has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.