QuantMoE-Bench: Examining Post-Training Quantization for Mixture-of-Experts

MoE quantization is done at one uniform precision for the whole model, which is wrong because MoE's routing gives its experts structurally different activation statistics.

A benchmark plus a structural observation rather than a new algorithm: because shared experts are accessed by EVERY token while token-conditioned experts are selectively activated, the two groups have very different quantization sensitivity and should not share a bit width. The paper measures a fine-grained, structure-aware precision assignment for MoE and shows uniform-precision PTQ is leaving quality on the table.

Paper's own claim (abstract): existing methods 'adopt a fixed quantization precision for the entire MoE model', which 'can lead to suboptimal performance, without considering the inherent sparse structure'; the shared-expert vs token-conditioned-expert asymmetry implies different quantization requirements. NO compression ratio paired with a quality delta appears in the abstract - this is an evaluation/diagnostic contribution, so speedup_reported stays null rather than inventing a compression claim.

arXiv ID 2406.08155 verified via /abs/. Venue: Semantic Scholar has no venue record and the arXiv comment field gives none - arxiv-preprint, year = v1 year 2024. Repo already has source qlab2-arxiv-quantmoebench, whose notes record the same shared-vs-token-conditioned precision finding. arXiv ID and title verified by fetching https://arxiv.org/abs/2406.08155 and reading the citation_title meta tag on 2026-10-03. Venue and year from the Semantic Scholar graph API batch endpoint (paper/arXiv:2406.08155) unless the record's notes say otherwise. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'unknown'. `venue` is 'arxiv-preprint', which means no peer-reviewed venue has been established for this work as of the verification date, so the track genuinely is not known. 'unknown' is used here rather than leaving the field null or guessing 'main': the schema separates 'unknown' from 'main' precisely so that an unprereviewed preprint is not silently counted as main-track work. This annotation is what lets a downstream filter distinguish 'peer reviewed, main track' from 'not peer reviewed' without inferring it from the venue string.