--- title: Google DeepMind's "Recirculation": Feeding Activations... description: A new Google DeepMind paper called Recirculation proposes feeding a fraction of a deep transformer layer's activations back into a shallow layer during prefill... canonical: https://daily.dev/posts/google-deepmind-s-recirculation-feeding-activations-back-through-a-frozen-transformer-znkldqvwv twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Google DeepMind's "Recirculation": Feeding Activations Back Through a Frozen Transformer | daily.dev og:description: A new Google DeepMind paper called Recirculation proposes feeding a fraction of a deep transformer layer's activations back into a shallow layer during prefill... og:url: https://daily.dev/posts/google-deepmind-s-recirculation-feeding-activations-back-through-a-frozen-transformer-znkldqvwv og:image: https://api.daily.dev/og/posts/ZnKLdQVwv.png og:image:alt: Google DeepMind's "Recirculation": Feeding Activations Back Through a Frozen Transformer og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Google DeepMind's "Recirculation": Feeding Activations Back Through a Frozen Transformer **[Collections](https://daily.dev/sources/collections)** · 3 min read · 0 upvotes · 0 comments ## Summary A new Google DeepMind paper called Recirculation proposes feeding a fraction of a deep transformer layer's activations back into a shallow layer during prefill and rerunning that portion of the network, without changing any weights. This lets a frozen model loop over partial conclusions before generating output, addressing the fixed-depth ceiling on state updates and the one-way information flow between shallow and deep layers. On Gemma3, the adaptive variant cuts perplexity by 23% and improves GSM8K accuracy by 21%, with perplexity dropping on 9 of 10 language-modeling datasets and gains up to 35% on the 12B model; multiple-choice benchmarks improved only modestly. The trade-off is that prefill becomes serial instead of parallel, though generation cost stays flat, and the technique's benefits don't necessarily transfer at the same magnitude to other model families. ## Content Google DeepMind has a new paper out, and it's got a rare one-word title: "Recirculation." That alone made me want to read it. The idea inside is worth the attention too. Here's the core problem it's tackling. In a standard transformer, shallow layers never get to see what the deeper layers have already figured out about the context. Information flows one way, layer by layer, and once it's processed deep in the network, it doesn't loop back. A feedforward transformer can only update its internal state as many times as it has layers, and long generations often need more updates than that. Chain-of-thought prompting has become a workaround, but really it's just doing basic state tracking in plain text because the architecture itself can't. Recirculation attacks this directly by adding recurrence at inference time, without touching the model's weights. It mixes a small fraction of a deep layer's activations back into a shallow layer at the next input step. The model effectively feeds its own activations back through itself during prefill, letting it behave more like a dynamical system that can track belief states over a long context, rather than a rigid one-pass pipeline. The practical tradeoff: if your model keeps losing track of state over a long input, you probably don't need to retrain it. You need to let it talk to itself a bit more. But that benefit isn't free. The extra cost lands on prefill, and prefill can no longer run in parallel the way it normally does, since the recirculated activations depend on a prior pass. Generation itself stays flat, though, since all the serial work happens upfront. On the Gemma3 family, the results are hard to ignore. Perplexity dropped on 9 of 10 language-modeling datasets, with a 35% reduction for the 12B model in the paper's core setup, and the adaptive variant cut perplexity by 23% while lifting GSM8k accuracy by 21%, with the original weights frozen and only light hyperparameter tuning applied. Multiple-choice gains were more modest across the board, which is worth keeping in mind before assuming this fixes everything. One caveat that stood out to me: the paper tests this on frozen weights, and the size of the benefit doesn't transfer cleanly across model families. So this isn't a universal free lunch you bolt onto any transformer and expect Gemma3-level gains. It's promising, but family-specific tuning seems to matter a lot. What I find most interesting here isn't just the perplexity numbers. It's the framing: using the model itself to inform architectural modifications, at inference time, without retraining. That's a training-free way to evolve how a model's architecture actually behaves. It's not hard to imagine this kind of self-referential tweak feeding into more ambitious recursive self-improvement approaches down the line. Whether that pans out or not, it's a clever bit of engineering hiding behind a one-word title. ## Questions this post answers ### What is Google DeepMind's Recirculation technique for transformers? Recirculation is a training-free technique that feeds a fraction of a deep transformer layer's activations back into a shallow layer during prefill, then reruns that portion of the network without changing any weights. This lets a frozen model loop over its own partial conclusions before committing to output, addressing the fixed number of state updates imposed by a model's layer count. _Developers tracking inference-time techniques that boost LLM quality without retraining can follow updates like this on daily.dev._ ### How much does Recirculation improve Gemma3 performance on GSM8K and perplexity benchmarks? On Gemma3, the adaptive variant of Recirculation cuts perplexity by 23% and improves GSM8K accuracy by 21%, achieved with frozen weights and only light hyperparameter tuning. Perplexity dropped on 9 of 10 language-modeling datasets tested, with the 12B model showing improvements as large as 35% on some datasets, though multiple-choice benchmarks saw only modest gains. _Anyone evaluating new LLM optimization techniques can compare benchmark claims like these on daily.dev._ ### What is the performance trade-off of using Recirculation in transformer inference? Recirculation keeps generation cost flat but makes prefill serial instead of parallel, since the recirculation step requires rerunning part of the network before producing output. This means the technique trades upfront prefill speed for improved state tracking on long inputs, without requiring any retraining of the model. _Engineers weighing inference-time trade-offs for long-context LLM workloads can dig into details like this on daily.dev._ ## Similar posts on daily.dev - [One Layer Deeper: What I Learned Trying to Compose One Step 64 Times](https://daily.dev/posts/one-layer-deeper-what-i-learned-trying-to-compose-one-step-64-times-a5o80g63g) · Medium · 2 upvotes · 0 comments - [The Machine Learning Practitioner’s Guide to Speculative Decoding](https://daily.dev/posts/the-machine-learning-practitioner-s-guide-to-speculative-decoding-3z4bou5te) · Machine Learning Mastery · 2 upvotes · 0 comments --- Tags: [#ai](https://daily.dev/tags/ai), [#machine-learning](https://daily.dev/tags/machine-learning), [#deep-learning](https://daily.dev/tags/deep-learning), [#ai-inference](https://daily.dev/tags/ai-inference), [#gemma](https://daily.dev/tags/gemma) [View this post on daily.dev](https://daily.dev/posts/google-deepmind-s-recirculation-feeding-activations-back-through-a-frozen-transformer-znkldqvwv) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Google DeepMind's \"Recirculation\": Feeding Activations Back Through a Frozen Transformer","url":"https://daily.dev/posts/google-deepmind-s-recirculation-feeding-activations-back-through-a-frozen-transformer-znkldqvwv","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/google-deepmind-s-recirculation-feeding-activations-back-through-a-frozen-transformer-znkldqvwv"},"datePublished":"2026-08-25T02:07:29.581Z","dateModified":"2026-08-25T21:58:48.904Z","description":"A new Google DeepMind paper called Recirculation proposes feeding a fraction of a deep transformer layer's activations back into a shallow layer during prefill...","image":"https://pbs.twimg.com/media/HQh5xRAaQAA2BBR.jpg","thumbnailUrl":"https://pbs.twimg.com/media/HQh5xRAaQAA2BBR.jpg","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/google-deepmind-s-recirculation-feeding-activations-back-through-a-frozen-transformer-znkldqvwv","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai,machine-learning,deep-learning,ai-inference,gemma","timeRequired":"PT3M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"Google DeepMind's \"Recirculation\": Feeding Activations Back Through a Frozen Transformer"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/google-deepmind-s-recirculation-feeding-activations-back-through-a-frozen-transformer-znkldqvwv#faq","mainEntity":[{"@type":"Question","name":"What is Google DeepMind's Recirculation technique for transformers?","acceptedAnswer":{"@type":"Answer","text":"Recirculation is a training-free technique that feeds a fraction of a deep transformer layer's activations back into a shallow layer during prefill, then reruns that portion of the network without changing any weights. This lets a frozen model loop over its own partial conclusions before committing to output, addressing the fixed number of state updates imposed by a model's layer count. Developers tracking inference-time techniques that boost LLM quality without retraining can follow updates like this on daily.dev."}},{"@type":"Question","name":"How much does Recirculation improve Gemma3 performance on GSM8K and perplexity benchmarks?","acceptedAnswer":{"@type":"Answer","text":"On Gemma3, the adaptive variant of Recirculation cuts perplexity by 23% and improves GSM8K accuracy by 21%, achieved with frozen weights and only light hyperparameter tuning. Perplexity dropped on 9 of 10 language-modeling datasets tested, with the 12B model showing improvements as large as 35% on some datasets, though multiple-choice benchmarks saw only modest gains. Anyone evaluating new LLM optimization techniques can compare benchmark claims like these on daily.dev."}},{"@type":"Question","name":"What is the performance trade-off of using Recirculation in transformer inference?","acceptedAnswer":{"@type":"Answer","text":"Recirculation keeps generation cost flat but makes prefill serial instead of parallel, since the recirculation step requires rerunning part of the network before producing output. This means the technique trades upfront prefill speed for improved state tracking on long inputs, without requiring any retraining of the model. Engineers weighing inference-time trade-offs for long-context LLM workloads can dig into details like this on daily.dev."}}]} ```