Tabular Foundation Models (TFMs) logopeterindia.net
Knowledge Hub

The Implications of Shared Memory in Looped Transformers

Shared memory in looped transformers: the paper's findings, related looped-transformer research and KV cache background.

31 topics•3 categories•Updated November 2026

What Does Shared Memory Mean for Looped Transformers?

16 entries

Looped transformers apply the same layers several times per token, adding compute without adding parameters. The catch is memory: each recursion normally writes its own key-value (KV) cache, so memory still grows with compute. A paper by Giovanni Monea and co-authors from Cornell and IBM Research, "The Surprising Effectiveness of Shared Memory in Looped Transformers" (arXiv 2610.02383, 1 October 2026), trains looped models so that only the first recursion writes a full cache while later recursions read it and keep a short window of their own. The authors report that this does not cost quality and instead improves it: at 150M to 1B parameters their Looped Prediction Transformer (LPT) and hybrid variant set a new quality-memory frontier, and with five recursions the hybrid lowers FineWeb-Edu validation perplexity by 1.12 to 1.82 against a same-size standard Transformer while using 76 to 79 percent less context memory. They attribute the gain to shared and local memory learning different representations and to the shared memory acting as a gradient path to the first recursion. This page summarizes the paper, lists the work it builds on, and notes what the results do and do not show. Descriptions are paraphrased from each paper's abstract and are not endorsements.

01The Surprising Effectiveness of Shared Memory in Looped Transformers (arXiv)Abstract page for the paper by Monea, Ramji, El-Kurdi, Lastras, Artzi, Godey and Astudillo, submitted 1 October 2026. 02Paper PDF on arXivFull text of the paper, including the Looped Prediction Transformer recipe, the hybrid variant and the analysis of why sharing helps. 03Paper on alphaXivalphaXiv's page for the paper, with discussion tools. 04LLM HubHub for large language model directories, on this site. 05LLM Training and Inferencing Optimization TechniquesKV cache, batching and other techniques, on this site. 06LLM Inference EnginesvLLM, SGLang, llama.cpp and more, on this site. 07Edge-Native Inference EnginesRunning models on consumer and edge hardware, on this site. 08AI Model Optimization TechniquesPruning, quantization and distillation, on this site. 09AI Model Quantization TechniquesGPTQ, AWQ and related methods, on this site. 10Large Reasoning ModelsReasoning models that spend more compute at inference, on this site. 11Small Language Models (SLMs)Compact models for constrained hardware, on this site. 12Reasoning Models for Edge DevicesSmall reasoning models and runtimes, on this site. 13State Space ModelsState space models and hybrids, on this site. 14LLM Building StepsThe pipeline for building an LLM, on this site. 15Fine-Tuning TechniquesLoRA, QLoRA and other fine-tuning methods, on this site. 16Linear Algebra for AIMatrices, attention and low-rank methods, on this site.

Looped and Recursive Transformer Research

8 entries

Papers on repeating layers to add compute without adding parameters.

17Universal Transformers (arXiv)Dehghani, Gouws and co-authors apply a recurrent transformer block repeatedly to refine token representations. 18Looped Transformers as Programmable Computers (arXiv)Giannou, Rajput and co-authors show that a looped transformer can be programmed to run algorithms. 19Looped Transformers are Better at Learning Learning Algorithms (arXiv)Yang, Lee and co-authors study looped transformers for learning algorithms in context. 20Reasoning with Latent Thoughts: On the Power of Looped Transformers (arXiv)Saunshi, Dikkala and co-authors relate looping to reasoning, treating each loop as a step of latent thought. 21Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach (arXiv)Geiping, McLeish and co-authors propose a recurrent-depth language model that scales test-time compute in latent space. 22Scaling Latent Reasoning via Looped Language Models (arXiv)Zhu, Wang and co-authors, the Ouro looped language models whose full-stack formulation the shared-memory paper builds on. 23Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA (arXiv)Bae, Fisch and co-authors relax strict layer tying in recursive transformers using layer-wise LoRA. 24Mixture-of-Recursions (arXiv)Bae, Kim and co-authors learn dynamic recursion depths for adaptive token-level computation.

KV Cache and Memory Efficiency Background

7 entries

Earlier work on shrinking or sharing attention memory.

25Attention Is All You Need (arXiv)Vaswani, Shazeer and co-authors introduce the transformer architecture. 26Fast Transformer Decoding: One Write-Head is All You Need (arXiv)Shazeer's multi-query attention, which shares key and value heads to reduce decoding memory traffic. 27GQA: Grouped-Query Attention (arXiv)Ainslie, Lee-Thorp and co-authors train generalized multi-query models from multi-head checkpoints. 28Reducing Transformer Key-Value Cache Size with Cross-Layer Attention (arXiv)Brandon, Mishra and co-authors share KV activations across layers to shrink the cache. 29You Only Cache Once: Decoder-Decoder Architectures (arXiv)Sun, Dong and co-authors cache key-value pairs once for the whole model. 30Efficient Memory Management for LLM Serving with PagedAttention (arXiv)Kwon, Li and co-authors manage the KV cache in pages, the basis of vLLM. 31Mamba: Linear-Time Sequence Modeling with Selective State Spaces (arXiv)Gu and Dao's selective state space model, relevant to the hybrid variants that mix recurrence with attention.

Frequently Asked Questions

What is a looped transformer?

A transformer that applies the same layers several times before predicting each token, adding computation without adding parameters. In principle it trades time for memory, matching a larger model's compute with fewer weights.

What problem does shared memory solve?

In standard looped models each recursion keeps its own key-value cache, so context memory grows with the number of recursions and, at long contexts, can exceed the memory used by the weights. The paper shares one cache across recursions to avoid this.

How does the shared-memory recipe work?

At each full-attention layer, later recursions attend both to their own entries and to the cache written by the same layer in the first recursion, in one attention operation, and later recursions keep only a short sliding window of their own cache. The result is the Looped Prediction Transformer (LPT), and the same rule applied to a hybrid looped model gives H-LPT.

What did the paper find?

The authors report that sharing memory improves quality instead of costing it. They state that LPT lowers validation perplexity by up to 1.09 nats and raises downstream accuracy by up to 4.8 percentage points over a matched-depth looped transformer with up to 4.7 times less context memory, and that the hybrid at five recursions beats a same-size standard Transformer while using 76 to 79 percent less context memory. These are the authors' results at 150M to 1B parameters.

Why does sharing memory help, according to the authors?

They report that later recursions rely heavily on the shared memory, that shared and local memory develop different key representations, and that reading shared memory sends gradients directly to the first recursion, which they describe as a gradient highway. Blocking that path reduced quality and stability.

What are the implications?

If the results hold more widely, they suggest that looped models can add depth-like compute without the usual memory growth, which matters for long contexts and memory-limited deployment such as edge devices. This is an interpretation, not a claim made as such by the paper; the evidence is limited to pretrained models of 150M to 1B parameters evaluated on perplexity and downstream accuracy, so larger scales and other tasks remain to be tested.

Is this the same as agent memory?

No. This page is about the KV memory inside a transformer's attention, not the long-term memory layers used by AI agents. See the Agent Memory page for that topic.

How current is this page?

The paper was checked on 1 November 2026 on arXiv, where it is listed as a preprint submitted on 1 October 2026 and not yet peer reviewed as far as that page shows. All other papers are linked to their arXiv abstracts. Descriptions are paraphrased from abstracts and are not rankings.