Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert LLM Inference

Standard top-k MoE routing runs the router AFTER all experts finish, so expert communication and memory traffic are serialised behind the slowest expert and small-batch MoE inference is dominated by that serialisation.

Move gating to the very beginning of the forward pass and pre-gate the expert outputs, so expert selection overlaps the layer computation instead of trailing it. The system side then co-designs the runtime around early gating with a dedicated expert-communication schedule.

NO reliable multiple available: the arXiv abstract states the approach generally and I did not verify a specific figure with its baseline and settings, so per the repo's own rule this field records the absence rather than a number. The paper is ISCA '24 — read its evaluation section before citing a speedup for MoE inference.

Venue verified via Crossref: ISCA 2024 (DOI 10.1109/isca59077.2024.00078, published 2024-06-29). Adoption research-only: pre-gating has not been adopted as a scheduling primitive by production MoE-serving engines. Worth flagging that 'Pre-gated MoE' is frequently cited as if it shipped in a major MoE runtime — it did not. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: isca has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.