arXiv:2505.03804 MoEQuant: Enhancing Quantization for Mixture-of-Experts LLMs via Expert-Balanced Sampling and Affinity Guidance

PTQ on MoE degrades badly for two structural reasons the paper names: inter-expert imbalance (uneven sample distribution across experts -> insufficient and biased calibration for rarely-used experts) and intra-expert imbalance (MoE's aggregation mechanism correlates samples differently with their assigned experts).