RecurrentGemma: Moving Past Transformers for Efficient Open Language Models

Attention-based decoding needs a KV cache that grows with context, which bounds long-sequence inference memory even for small models.

Google's Griffin architecture applied to language models at 2B and 9B parameters. Griffin combines LINEAR RECURRENCES with LOCAL ATTENTION in an alternating pattern: most layers are a gated linear recurrence (a diagonal linear recurrence in the RetNet/RWKV family rather than a selective Mamba scan) and one layer in four is local sliding-window attention, which supplies exact local recall. The defining property is the FIXED-SIZED STATE: the recurrence state does not grow with sequence length and the attention window is bounded, so total inference memory is constant in context length. That is the same serving property as Mamba, reached without a sequential selective scan and therefore without the same tensor-core-hostile scan. Trained on fewer tokens than Gemma baselines and reported to reach comparable performance to similarly sized Gemma models.

arXiv preprint (v2 2024-08-28) from Google DeepMind; no journal-ref on the abs page. The paper reports comparable quality to similarly sized Gemma baselines despite fewer training tokens, plus memory and throughput comparisons at long sequence length, but no single speedup ratio with baseline and settings, so speedup_reported is null. Engine support is the weak point: there is NO recurrentgemma entry in llama.cpp's LLM_ARCH registry (checked 2026-10-03) and no native RecurrentGemma row in the vLLM supported-models list consulted. So despite being a Google model with a served API lineage, the open-weight recurrent variant is not first-class in open engines. Good example of a constant-state architecture that failed the ecosystem-support test that Mamba and Jamba passed. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'unknown'. `venue` is 'arxiv-preprint', which means no peer-reviewed venue has been established for this work as of the verification date, so the track genuinely is not known. 'unknown' is used here rather than leaving the field null or guessing 'main': the schema separates 'unknown' from 'main' precisely so that an unprereviewed preprint is not silently counted as main-track work. This annotation is what lets a downstream filter distinguish 'peer reviewed, main track' from 'not peer reviewed' without inferring it from the venue string.