OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models

PTQ methods hand-craft their quantization parameters (clipping thresholds, equivalent transformations), which is why they degrade at INT3/INT4-W4A4.

Block-wise error-minimization framework that learns the quantization parameters rather than hand-designing them, via two components: Learnable Weight Clipping (LWC) optimizes the weight clipping threshold to reshape extreme values; Learnable Equivalent Transformation (LET) shifts activation outliers into the weights by learning a per-channel scale folded into the preceding LayerNorm, so the activation-side quantization difficulty is moved to the weight side. Trained per block on 128 calibration samples, then the block is frozen.

Paper's own claim (abstract): LLaMA-2 7B-70B processed on a SINGLE A100-40G within 1-16 h per model (this is a calibration-cost claim, not a speedup); 'good performance in diverse quantization settings while maintaining the computational efficiency of PTQ'. COMPRESSION RATIO: covers weight-only and W4A4/W4A8 settings, i.e. up to ~4x on weights and up to 8x combined for W4A8. QUALITY COST: the abstract gives no paired perplexity delta - it points at per-setting tables. speedup_reported here is therefore the calibration-cost figure, which is what the abstract actually quantifies.

SLUG RENAMED 2026-10-04 (parent agent): was 'omniquant', colliding with the quantization record of the same id, which made every bare-slug cross-reference ambiguous. Inbound refs (['scheme']) all mean the quantization record, so none needed repointing; papers are cited via `sources`, never by bare slug. arXiv ID 2308.13137 verified via /abs/; v3. Venue ICLR 2024 confirmed by the arXiv comment field ('ICLR 2024 Camera Ready') and Semantic Scholar. Year = venue year 2024 (v1 was 2023-08-25). Note the repo already has source qlab-omniquant-paper for this paper; the 128-samples-on-one-A100-40G calibration cost is from that source record. arXiv ID and title verified by fetching https://arxiv.org/abs/2308.13137 and reading the citation_title meta tag on 2026-10-03. Venue and year from the Semantic Scholar graph API batch endpoint (paper/arXiv:2308.13137) unless the record's notes say otherwise. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: iclr has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.