ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs

Gradient-based rotation learning (SpinQuant) is too expensive at frontier scale, and even gradient-free rotation learning (DartQuant) needs too much calibration data and wall-clock.

Two efficiency ideas for rotation learning. (1) Data selection using the geometric structure of the CONVEX HULL of the activations, which picks a calibration set orders of magnitude smaller than prior rotation methods. (2) Because that reduced set spans an r-dimensional subspace with r < d, optimizing the full d x d rotation reduces exactly to optimizing a d x r matrix on the Stiefel manifold, solved with an efficient ADMM algorithm using thin matrix updates at every step. Same mechanism class as QuaRot/SpinQuant, made scalable.

Paper's own claim (abstract): with a calibration set orders of magnitude smaller than state-of-the-art rotation methods, matching their low-bit quantization performance. CONCRETE: Llama-3-70B at W4A4KV4 completes rotation calibration in UNDER 12 MINUTES with WikiText-2 perplexity 5.63, versus 7.55 for DartQuant which requires 111 minutes. Llama-3.1-405B at W4A4: calibration in just over 2 HOURS on a SINGLE H200, WikiText-2 perplexity 2.97 versus 3.48 for GPTAQ+QuaRot. COMPRESSION RATIO: 4-bit weights, activations and KV cache. QUALITY COST: 5.63 and 2.97 perplexity against full-precision baselines that the abstract does not print - so these are relative comparisons, not quality costs.

arXiv ID 2609.36120 verified via /abs/ (title matches exactly). Venue: Semantic Scholar was rate-limiting and returned no record, and the arXiv comment field is empty - recorded as arxiv-preprint with year = v1 year 2026 and confidence lowered to reflect the missing venue check. The single-H200 405B calibration run is why hardware_relevance names the H200. arXiv ID and title verified by fetching https://arxiv.org/abs/2609.36120 and reading the citation_title meta tag on 2026-10-03. Venue and year from the Semantic Scholar graph API batch endpoint (paper/arXiv:2609.36120) unless the record's notes say otherwise. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'unknown'. `venue` is 'arxiv-preprint', which means no peer-reviewed venue has been established for this work as of the verification date, so the track genuinely is not known. 'unknown' is used here rather than leaving the field null or guessing 'main': the schema separates 'unknown' from 'main' precisely so that an unprereviewed preprint is not silently counted as main-track work. This annotation is what lets a downstream filter distinguish 'peer reviewed, main track' from 'not peer reviewed' without inferring it from the venue string.