Titans: Learning to Memorize at Test Time

A model's memory is either its finite context window (short-term) or its weights (fixed at training time); there is no mechanism that ACQUIRES and retains new information at inference the way a cache does.

Neural memory: a long-term persistent memory module with three variants differing in how it is written and read - persistent memory as a straightforward long-term memory; momentum update; and surprise-based (gradient-like) update. This memory is combined with short-term memory as attention and, in the data-dependence variant, a context-based gating decides per input whether to query the long-term memory.

NOT A THROUGHPUT CLAIM. The abstract reports quality: more effective than Transformers and recent modern linear recurrent models on language modelling, common-sense reasoning, genomics and time series; and scaling to LARGER THAN 2M context window with higher needle-in-a-haystack accuracy than baselines. The 2M context figure is a capability result with no baseline engine or memory footprint attached.

arXiv ID 2501.00663 verified via abs-page citation_title/citation_date on 2026-10-03; title matches. CORRECTION TO THE SOURCE RECORD: it says 'arXiv preprint; no journal_ref' with no venue evidence, but OpenReview returns venue='NeurIPS 2025 poster', venueid='NeurIPS.cc/2025/Conference' for the exact title and the Behrouz/Zhong/Mirrokni author list. venue=neurips, year=2025. adoption=research-only: no titans module in vLLM's registry or v1 backend list, SGLang's models directory, or transformers cache_utils.py. Note the OpenReview record also exists as an OpenReview.net/Archive direct upload with venue=None - the NeurIPS.cc/2024->2025 Conference record is the citable one. One forward-looking caution recorded because it will matter to anyone following this line: the same first author has since published ATLAS ('Learning to Optimally Memorize the Context at Test Time', ICML 2026) on OpenReview, which is a successor to Titans - if this record is revisited, ATLAS is where the line went. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: neurips has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings. DUPLICATE-SLUG-CONSOLIDATION-2026-10-05 (agent w6-papers). data/papers/titans.json and data/papers/titans-test-time-memory.json were BOTH live records for arXiv 2501.00663 (Titans: Learning to Memorize at Test Time), so one paper appeared twice under two slugs and - where the records disagreed - under two different venues. titans-test-time-memory.json has been merged into this record and removed. IDENTIFICATION IS BY ARXIV ID, RE-FETCHED NOT ASSUMED: the arXiv API returns one entry for 2501.00663 with this title and the author list [Behrouz, Ali, Zhong, Peilin, Mirrokni, Vahab], which both records were matching. Source w6p-arxiv-api-duplicate-id-confirmation. Venue evidence per record is in w6p-openreview-api2-duplicate-venue-records. Every arXiv id in this consolidation was taken from an API response rather than from memory or from the records, per the rule that this repo has shipped wrong ids twice. WHAT WAS CHECKED BEFORE REMOVING ANYTHING: every field of every data/**/*.json record was scanned for the string "titans-test-time-memory". Zero inbound references were found - no model, gotcha, source, engine or doc record pointed at it - so the merge breaks no reference. Papers are cited by SOURCE ID, never by slug, so no `sources` entry needed repointing; the retired record's source ids were folded into this record's `sources` so the evidence survives the merge. MERGED FIELD BY FIELD - nothing was silently dropped: mechanism: the retired record's text was materially fuller (844 chars vs 407) and is RETAINED here rather than discarded. Its text: Proposes a neural long-term memory module, framed explicitly as learning to memorise at TEST TIME: a surprising-as-written design where memory is a persistent, weights-derived long-term memory rather than something computed by a forward pass over the sequence. As a recurrent/persistent module it maintains its own memory matrices across a very long window, using momentum-style updates to decide what to write and a surprise measure (a test-time loss) to decide what to retrieve, and it is combined with attention so attention handles the current local context while the module supplies long past information. The memory is a set of persistent matrices with a fixed, budgeted size (the paper explores different memory sizes as a scaling axis), so memory capacity is a design parameter independent of context length rather than growing with it. hardware_relevance was empty here while the retired twin listed A100 80GB SXM4 and H100 SXM; populated from the twin. open_weights: open_weights left null rather than taking the retired twin's false. This is a Google Research architecture paper with no released checkpoint, and a boolean here is an assertion about Google's release decision that no cited source supports - null is the honest value. AFFILIATIONS CORRECTED - arXiv HTML byline for 2501.00663v1 carries a single affiliation string, 'Google Research', with a google.com contact address. This record listed Google Research AND Google DeepMind; DeepMind is not named on this paper. The retired twin's Google Research alone matched the byline. VENUE AND THE CLAIMS THAT COMPETED FOR IT: NeurIPS 2025 on both records - a clean duplicate on venue. One source was DROPPED rather than merged: the retired twin cited sco-neurips-2024-proceedings for a NeurIPS 2025 paper, and citing a 2024 proceedings volume for a 2025 acceptance is exactly the volume-number trap this repo already logs for PMLR. The venue itself rests on the OpenReview note with venue 'NeurIPS 2025 poster' / venueid 'NeurIPS.cc/2025/Conference', forum 8GjSf9Rh7Z, corroborated on the Behrouz/Zhong/Mirrokni author list. The retired twin's `mechanism` was materially fuller and is retained; its `speedup_reported` was null where this record carries the paper's actual quality claims including the 2M-context needle-in-a-haystack result. STATUS: the contradiction this pair represented is now resolved, so the record is no longer marked contested on account of the duplicate. Fields were changed as stated above; arxiv_id is unchanged.