Mixture of Quantized Experts (MoQE): Complementary Effect of Low-bit Quantization and Robustness

MoE deployment is memory-bound because ALL expert weights must be resident, even though only a few experts run per token.

Simple weight-only quantization applied ONLY to the expert FFN weights (down to 2 bits per weight), deliberately leaving attention and the router at higher precision. No training and no special kernel work - the finding is that expert layers are structurally more robust to quantization than dense FFNs, so the standard assumption that you must protect experts does not hold.

Paper's own claim (abstract): low-bit quantization applied only to experts 'delivers a reliable model performance while reducing the memory size significantly even without any additional training in most cases'; expert layers are 'much more robust to the quantization than conventional feedforward networks'; and MoE models with 2-bit expert weights can beat the dense model trained on the same amount of data. COMPRESSION RATIO: 2-bit expert weights (~8x on the expert tensors, which dominate MoE parameter count). QUALITY COST: 'reliable performance' / better than the data-matched dense baseline - the claim is qualitative in the abstract.

arXiv ID 2310.02410 verified via /abs/. Venue: Semantic Scholar returns arXiv.org only, arXiv comment empty - arxiv-preprint, year = v1 year 2023. Part of the repo's qlab2 MoE quantization cluster alongside MoEQuant / EAQuant / QuantMoE-Bench / AlphaQ / VSRAQ. arXiv ID and title verified by fetching https://arxiv.org/abs/2310.02410 and reading the citation_title meta tag on 2026-10-03. Venue and year from the Semantic Scholar graph API batch endpoint (paper/arXiv:2310.02410) unless the record's notes say otherwise. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'unknown'. `venue` is 'arxiv-preprint', which means no peer-reviewed venue has been established for this work as of the verification date, so the track genuinely is not known. 'unknown' is used here rather than leaving the field null or guessing 'main': the schema separates 'unknown' from 'main' precisely so that an unprereviewed preprint is not silently counted as main-track work. This annotation is what lets a downstream filter distinguish 'peer reviewed, main track' from 'not peer reviewed' without inferring it from the venue string.