Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models

Linear attention is theoretically linear-time, but the CUMULATIVE SUMMATION needed to compute it destroys the theoretical advantage in the causal (autoregressive) setting - so in practice linear attention is not faster than attention.

Tiling. Split the causal linear-attention computation into intra-block and inter-block parts: compute the intra-block portion with CONVENTIONAL attention machinery (dense, small, well-shaped) and the inter-block portion with linear-attention kernel tricks. A tiling technique is applied in BOTH forward and backward so the IO pattern suits the GPU. Implemented in Triton, explicitly IO-aware.

Paper's own claim (abstract), qualitative-plus-conditional: Lightning Attention-2 'retains consistent training and inference speed REGARDLESS of input sequence length' and is 'significantly faster than other attention mechanisms'. No multiple with a named baseline appears in the abstract. The load-bearing quantitative property is the CONSTANT-INDEPENDENCE from sequence length, which is a different kind of claim from a speedup.

arXiv ID 2401.04658 verified via abs-page citation_title/citation_date on 2026-10-03; title matches. venue=arxiv-preprint: comment says only 'Technical Report', and OpenReview search for this title returns only LATER papers (a NeurIPS 2024 ICML.workshop submission and 'Can Mamba Always Enjoy the Free Lunch?', ICLR 2025) - no venue for this paper itself. adoption=abandoned - the sharpest call in this batch and the basis is checked, not assumed: the official repo OpenNLPLab/lightning-attention was last pushed 2025-02-23 (over a year before this verification date) with 344 stars; its sibling repo OpenNLPLab/TransnormerLLM was last pushed 2024-01-23; and NEITHER Lightning Attention nor TransNormer appears in vLLM's registry, vLLM's 30-file v1 attention backend list, SGLang's 248-file models directory, SGLang's attention backend list, or transformers cache_utils.py. The technique survives only indirectly - TransNormerLLM's later versions shipped a substantially rewritten 'Lightning Attention' that is NOT this algorithm, so citing these as the same thing would be wrong. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'unknown'. `venue` is 'arxiv-preprint', which means no peer-reviewed venue has been established for this work as of the verification date, so the track genuinely is not known. 'unknown' is used here rather than leaving the field null or guessing 'main': the schema separates 'unknown' from 'main' precisely so that an unprereviewed preprint is not silently counted as main-track work. This annotation is what lets a downstream filter distinguish 'peer reviewed, main track' from 'not peer reviewed' without inferring it from the venue string. DUPLICATE-SLUG-CONSOLIDATION-2026-10-05 (agent w6-papers). data/papers/lightning-attention-2.json and data/papers/lightning-attention-2-tiling.json were BOTH live records for arXiv 2401.04658 (Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models), so one paper appeared twice under two slugs and - where the records disagreed - under two different venues. lightning-attention-2-tiling.json has been merged into this record and removed. IDENTIFICATION IS BY ARXIV ID, RE-FETCHED NOT ASSUMED: the arXiv API returns one entry for 2401.04658 with this title and the author list [Qin, Zhen, Sun, Weigao, Li, Dong, Shen, Xuyang], which both records were matching. Source w6p-arxiv-api-duplicate-id-confirmation. Venue evidence per record is in w6p-openreview-api2-duplicate-venue-records. Every arXiv id in this consolidation was taken from an API response rather than from memory or from the records, per the rule that this repo has shipped wrong ids twice. WHAT WAS CHECKED BEFORE REMOVING ANYTHING: every field of every data/**/*.json record was scanned for the string "lightning-attention-2-tiling". Zero inbound references were found - no model, gotcha, source, engine or doc record pointed at it - so the merge breaks no reference. Papers are cited by SOURCE ID, never by slug, so no `sources` entry needed repointing; the retired record's source ids were folded into this record's `sources` so the evidence survives the merge. MERGED FIELD BY FIELD - nothing was silently dropped: mechanism: the retired record's text was materially fuller (750 chars vs 392) and is RETAINED here rather than discarded. Its text: A training-and-inference algorithm, not a new layer definition: it applies a TILING decomposition that separates the INTRA-block component (computed with ordinary attention) from the INTER-block component (computed with linear-attention kernel tricks), and applies the same tiling in forward and backward so the GPU stays occupied. Implemented in Triton to be IO-aware. The result is that training and inference speed stays roughly CONSTANT as input sequence length grows, which is the property no prior causal linear attention had achieved. The linear-attention state is fixed-size and constant in sequence length, so decode memory does not grow with context. This is the algorithmic contribution MiniMax then built the 456B-parameter MiniMax-01 on. speedup_reported: the two records carried DIFFERENT text for the same paper. This record's text is retained and the retired record's variant is quoted so the difference is visible: "Reports that Lightning Attention-2 retains consistent training and inference speed regardless of input sequence length and is significantly faster than other attention mechanisms, with experiments across different model sizes and sequence lengths. No single numeric speedup ratio is given in the abstract with baseline and settings, so the recorded claim is the length-invariance of throughput rather than a bare number." hardware_relevance UNION taken: this record had A100 40GB/80GB SXM4 and H100 SXM; the retired twin had A100 80GB SXM4 and H100 SXM. open_weights: open_weights left null; the retired twin's true is not adopted because its own stated grounds for adoption - MiniMax-01 and MiniMax-M1 building on 'lightning attention' - are a DIFFERENT paper's lineage, not a weight release for this one. adoption stays ABANDONED against the retired twin's in-production, and the disagreement is recorded rather than averaged. The retired twin argued in-production from MiniMax-01/MiniMax-M1 using lightning attention plus LLM_ARCH_MINIMAX_01 in llama.cpp; this record's checked negative is that the OpenNLPLab repository was last pushed 2025-02-23 and that NEITHER Lightning Attention nor TransNormerLLM appears in vLLM's registry, vLLM's 30-file v1 attention backend list, SGLang's 248-file models directory, SGLang's attention backend list, or transformers cache_utils.py. The specific trap: TransNormerLLM's later versions ship a substantially REWRITTEN 'Lightning Attention' that is NOT this algorithm, so a same-named mechanism in a production model is not evidence that this paper's method shipped. AFFILIATIONS CORRECTED - AFFILIATION ERROR FOUND AND CORRECTED, and it is the starkest in the sweep. arXiv HTML byline for 2401.04658v2 gives all six authors the single affiliation 'OpenNLPLab' and names NO university at all. This record listed 'Nanjing University' + 'University of Copenhagen'; the retired twin listed 'Tsinghua University' + 'ModelBest'. That is four different institutions across two records for one paper, and not one of them appears on it. The authors' public affiliations have moved on since 2024 - Yiran Zhong's ModelBest work is real - but an affiliation field is a claim about THIS paper's author block, not about where its authors work now, and that distinction is exactly how the wrong values got in. VENUE AND THE CLAIMS THAT COMPETED FOR IT: No peer-reviewed venue, agreed by both records, and one near-miss is worth naming because it would have been an easy false promotion. Searching the title surfaces 'Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention' (arXiv 2405.17381) with venue 'ICML 2024 Poster', venueid 'ICML.cc/2024/Conference', the SAME first author Zhen Qin, and overlapping coauthors Weigao Sun and Dong Li. That is a DIFFERENT paper - different arXiv ID, different title, different venue - and inheriting ICML 2024 from it would be a title-adjacency error of exactly the class this repo has shipped twice. 2401.04658 has no conference record. Two further identification notes: the arXiv comment reads only 'Technical Report', and the arXiv title ends 'in Large Language Models' where this record carries the shorter 'in Linear Attention' - so the record's name is not the arXiv title, and a title-equality check against the arXiv API would fail on it. STATUS: the contradiction this pair represented is now resolved, so the record is no longer marked contested on account of the duplicate. Fields were changed as stated above; arxiv_id is unchanged.