MiniMax-01: Scaling Foundation Models with Lightning Attention

Linear attention could not be scaled to hundreds of billions of parameters in practice, because long-context training of linear attention requires computation-communication overlap techniques nobody had made efficient.

Scales lightning attention (Lightning Attention-2's causal tiling algorithm) into an MoE model: 32 experts and 456B TOTAL parameters, of which 45.9B are ACTIVATED per token (about 10.1 percent active). Two supporting systems contributions are what made the scale reachable: an optimised parallel strategy and highly efficient computation-communication OVERLAP techniques for MoE dispatch and for lightning attention itself, so the all-to-all and the attention kernel are hidden behind other work. Because lightning attention holds a fixed-size state, the long-context claim (4M tokens for MiniMax-Text-01) does not come with a proportionally growing KV cache, which is the entire serving argument for the architecture at this scale.

arXiv preprint; no journal-ref. The 45.9B-active-of-456B-total figure is quoted directly from the abstract and is the number to carry into serving capacity planning. speedup_reported is null: the paper claims efficient training and inference on hundreds-of-billions scale and superior long-context capability versus other models, but states no tok/s ratio with baseline and settings. Engine support verified in llama.cpp: LLM_ARCH_MINIMAX_01, LLM_ARCH_MINIMAX_M2 and LLM_ARCH_MINIMAX_M3 exist, MINIMAX_01 is classified hybrid by llm_arch_is_hybrid(), and conversion/minimax.py is present. Successor MiniMax-M1 is recorded separately. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'unknown'. `venue` is 'arxiv-preprint', which means no peer-reviewed venue has been established for this work as of the verification date, so the track genuinely is not known. 'unknown' is used here rather than leaving the field null or guessing 'main': the schema separates 'unknown' from 'main' precisely so that an unprereviewed preprint is not silently counted as main-track work. This annotation is what lets a downstream filter distinguish 'peer reviewed, main track' from 'not peer reviewed' without inferring it from the venue string.