Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models

RNNs have fast inference and scale efficiently on long sequences but are hard to train and hard to scale, while Transformers train well but decode slowly with cost that grows with context.

Two models. HAWK is an RNN with GATED LINEAR RECURRENCES, which exceeds Mamba's reported downstream-task performance. GRIFFIN is a HYBRID that MIXES those gated linear recurrences with LOCAL ATTENTION - the linear-recurrence path gives constant-cost long-range state and the local attention window restores exact local computation. Because the recurrent state is fixed-size, context length does not inflate the state, unlike a KV cache.

Hawk exceeds the REPORTED performance of Mamba on downstream tasks. Griffin MATCHES THE PERFORMANCE OF LLAMA-2 despite being trained on over 6 TIMES FEWER TOKENS. Models match hardware efficiency of Transformers during TRAINING, and during INFERENCE they have LOWER LATENCY and SIGNIFICANTLY HIGHER THROUGHPUT. Scaled to 14B parameters, with a described sharding scheme for distributed training. No numeric latency or throughput multiplier is given in the abstract - the claims are qualitative plus the 6x-tokens-trained and 14B-scale facts, so no multiplier is recorded.

ADOPTION: research-only - weights are released (open_weights true) but no mainstream inference engine ships a Griffin mode, and the broader hybrid-RNN generation has not displaced attention in production engines. VENUE NOTE: no peer-reviewed venue verifiable via Semantic Scholar; arXiv comment carries only '25 pages, 11 figures', so venue = arxiv-preprint, year = ArXiv v1 (Feb 2024). INCLUDED AS THE ARCHITECTURAL ALTERNATIVE TO KV CACHES, and the distinction matters for anyone reading the long-context records: Infini-attention (pap-eff-infini-attention-infinite-context) bounds a compressive memory inside attention, Griffin replaces part of attention with a gated linear recurrence whose state is fixed-size. The reason it has NOT won in production despite better decode properties is recorded as an observation rather than a paper claim: the decode advantage is against Transformers' baseline latency, and production comparisons run against highly-optimised attention kernels, not paper baselines - so 'lower latency and significantly higher throughput' should not be read as a standing advantage over shipped engines. DANGLING REF STILL OPEN (2026-10-04 paper-read pass; supersedes the earlier 'LEFT UNRESOLVED' note, which reached the same conclusion but only from this record's own text - this time the PAPER itself was read). No A100 and no NVIDIA GPU anywhere in the paper; it ran on TPU throughout: "Since we run all our experiments on TPU-v3, we focus on developing an efficient implementation tailored to this device", and Appendix D tabulates "Specification / TPU-v3 Pod / HBM capacity 32 GB / HBM bandwidth 900 GB/s". Source: https://arxiv.org/pdf/2402.19427 . Its Nvidia mentions are generic ("Nvidia GPUs' TensorCores") and its Discussion explicitly scopes out other accelerators ("The conclusions drawn here do not necessarily apply to other accelerators"). Left dangling. AFFILIATIONS FILLED 2026-10-04 (paper-verification pass): from the Semantic Scholar graph API (paper/arXiv:2402.19427), which is the only machine-readable source that carries author affiliations — the arXiv Atom feed returns author names only, and no ORCID is exposed by either API. One known upstream typo corrected: "Tel Aviv Univesity" -> "Tel Aviv University". S2 affiliation coverage is partial (120 of 249 corpus records carry any), so these lists may be incomplete rather than exhaustive. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'unknown'. `venue` is 'arxiv-preprint', which means no peer-reviewed venue has been established for this work as of the verification date, so the track genuinely is not known. 'unknown' is used here rather than leaving the field null or guessing 'main': the schema separates 'unknown' from 'main' precisely so that an unprereviewed preprint is not silently counted as main-track work. This annotation is what lets a downstream filter distinguish 'peer reviewed, main track' from 'not peer reviewed' without inferring it from the venue string.