EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization

Dense PTQ methods ignore that MoE routing is a DISCRETE decision: small quantization perturbations can change which experts are selected and thereby change the computation path.

Three expert-aware components, each aimed at a named failure mode: (1) smoothing aggregation to suppress activation outliers using the MoE aggregation weights; (2) ROUTING CONSISTENCY ALIGNMENT - a penalty term that pushes the post-quantization top-k expert selection toward matching the pre-quantization selection, so the computation path is preserved; (3) calibration data balance so sparsely-activated experts still get calibrated. The routing-consistency term is the distinguishing idea and is shared in spirit with VSRAQ.

Paper's own claim (abstract): across W4A4 / W3A4 / W3A3 / W2A4 settings on three MoE architectures, average accuracy improvements of 1.15-13.81% over existing methods, with largest gains on reasoning tasks. COMPRESSION RATIO: down to 2-bit weights with 4-bit activations (W2A4) is the most aggressive setting reported, i.e. up to ~8x on weights. QUALITY COST: 1.15-13.81% average accuracy gain over baselines - a wide band that shows how much the baselines vary by architecture.

arXiv ID 2506.13329 verified via /abs/. Venue: Semantic Scholar returns arXiv.org only - arxiv-preprint, year = v1 year 2025. Repo already has source qlab2-arxiv-eaquant, whose notes independently record the same routing-instability framing. arXiv ID and title verified by fetching https://arxiv.org/abs/2506.13329 and reading the citation_title meta tag on 2026-10-03. Venue and year from the Semantic Scholar graph API batch endpoint (paper/arXiv:2506.13329) unless the record's notes say otherwise. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'unknown'. `venue` is 'arxiv-preprint', which means no peer-reviewed venue has been established for this work as of the verification date, so the track genuinely is not known. 'unknown' is used here rather than leaving the field null or guessing 'main': the schema separates 'unknown' from 'main' precisely so that an unprereviewed preprint is not silently counted as main-track work. This annotation is what lets a downstream filter distinguish 'peer reviewed, main track' from 'not peer reviewed' without inferring it from the venue string.