A research team affiliated with Google Research, Cornell University, and the University of Southern California has introduced a promising new method for improving how artificial intelligence models remember long sequences of information. The paper, “Memory Caching: RNNs with Growing Memory,” was written by Ali Behrouz, Zeman Li, Yuan Deng, Peilin Zhong, Meisam Razaviyayn, and Vahab Mirrokni. It was accepted for presentation at the 2026 International Conference on Machine Learning. The paper and its PDF are available at https://arxiv.org/abs/2602.24281.

The research addresses a basic tradeoff between Transformers and recurrent neural networks. Transformers—the foundation of most leading large language models—can refer directly to information throughout their context. However, conventional full attention becomes increasingly expensive as the context grows. Recurrent neural networks process information more efficiently by continually updating a fixed-size internal state, but that state eventually becomes overloaded and loses important details.

Memory Caching creates a middle ground. The model divides a sequence into segments and saves checkpoints of its internal memory after processing each segment. When it later needs information, it can consult its current memory and relevant cached checkpoints instead of depending on one continuously overwritten state.

The researchers developed four approaches for combining or selecting these memories. The most important include Gated Residual Memory, which weights checkpoints according to their relevance to the current question, and Sparse Selective Caching, which uses a routing mechanism to select only a limited number of relevant memories. The model’s memory capacity can therefore grow with the sequence while its computational cost remains adjustable.

The experiments show substantial improvements over recurrent models without Memory Caching. On long-context retrieval, language modeling, and recall-intensive tests, the enhanced recurrent models narrowed the performance gap with Transformers. However, Transformers still produced the best overall results on some of the most demanding recall evaluations. The experiments also used models of no more than 1.3 billion parameters, far below the scale of current frontier systems.

Consequently, this paper does not demonstrate that Memory Caching is ready to replace the Transformer architecture underlying frontier systems such as ChatGPT, Claude, or Gemini. Its near-term influence is more likely to appear in hybrid architectures that combine Transformer attention with recurrent processing, compressed memory, sparse routing, and external retrieval.

The technique may be particularly valuable for specialized models that continuously process large amounts of sequential information but only occasionally need to recover particular details. Possible applications include analyzing scientific literature, searching software repositories, monitoring cybersecurity telemetry, interpreting patient histories, processing industrial sensor streams, examining lengthy legal records, analyzing video, and maintaining the working history of persistent AI agents.

For these specialized purposes, processing every part of the history with full attention may be unnecessary and uneconomical. Memory Caching could allow a model to preserve compressed representations of earlier material and selectively retrieve them when relevant. This could reduce computing and memory requirements while supporting much longer operational histories.

The paper’s most important contribution is therefore not the end of the Transformer. It challenges the assumption that capable long-context models must rely exclusively on full Transformer attention. If the results scale successfully, Memory Caching could help move the industry toward specialized and hybrid architectures in which different kinds of memory are selected according to the work a model is expected to perform.