--- title: OpenAI Astra and Looped Transformers | daily.dev description: Debunks hype claiming OpenAI's Astra uses a novel 'looped transformer' architecture that hides chain-of-thought reasoning. The looped transformer idea simply... canonical: https://daily.dev/posts/openai-astra-and-looped-transformers-req9aufr1 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: OpenAI Astra and Looped Transformers | daily.dev og:description: Debunks hype claiming OpenAI's Astra uses a novel 'looped transformer' architecture that hides chain-of-thought reasoning. The looped transformer idea simply... og:url: https://daily.dev/posts/openai-astra-and-looped-transformers-req9aufr1 og:image: https://api.daily.dev/og/posts/Req9AUFr1.png og:image:alt: OpenAI Astra and Looped Transformers og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # OpenAI Astra and Looped Transformers **[Sebastian Raschka](https://daily.dev/sources/sebastianraschka)** · 3 min read · 0 upvotes · 0 comments ## Summary Debunks hype claiming OpenAI's Astra uses a novel 'looped transformer' architecture that hides chain-of-thought reasoning. The looped transformer idea simply reuses a layer stack multiple times (e.g., Nanbeige 4.2-3B reuses a 22-layer stack twice to reach 44 effective layers without duplicating weights), roughly doubling compute cost while keeping parameter count the same. This concept traces back to the Mixture-of-Recursions NeurIPS paper, which adds a learned router for per-token adaptive depth. The claim that this technique obscures chain-of-thought reasoning is argued to be likely a misunderstanding, since reusing layers just adds computation in hidden states, similar to how any added model depth would. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is a looped transformer architecture in models like Nanbeige 4.2? A looped transformer reuses the same layer stack multiple times instead of stacking unique layers, increasing effective depth without adding parameters. Nanbeige 4.2-3B reuses a 22-layer stack twice, extending it to an effective 44 layers while keeping storage and RAM the same, though compute cost nearly doubles since text passes through roughly twice as many layers. _Anyone comparing efficient LLM architectures can follow deep dives like this on daily.dev._ ### Does the looped transformer technique hide a model's chain-of-thought reasoning? Not inherently. Reusing layers only adds extra computation within hidden states before a token is generated, the same way ordinary transformer layers do, so it does not by itself suppress visible chain-of-thought text. A plausible effect is that more recurrent passes could let a model need fewer intermediate reasoning tokens, shifting some computation into latent activations, but this is not unique to looped transformers versus simply scaling model size. _Developers evaluating reasoning-model transparency claims can track architecture debates like this on daily.dev._ ### Where did the idea of reusing transformer layers to increase model depth originate? It traces to the Mixture-of-Recursions paper presented at NeurIPS, which introduced dynamic recursive depths for adaptive token-level computation. That approach adds a learned router deciding whether each token receives one, two, or more passes through the layer stack, letting easy tokens exit early while harder tokens get extra computation, a more sophisticated version of the fixed-loop approach used in Nanbeige 4.2. _Researchers tracking the lineage of efficient LLM techniques can find papers like this surfaced on daily.dev._ --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#llm](https://daily.dev/tags/llm), [#deep-learning](https://daily.dev/tags/deep-learning), [#openai](https://daily.dev/tags/openai), [#prompt-engineering](https://daily.dev/tags/prompt-engineering) [View this post on daily.dev](https://daily.dev/posts/openai-astra-and-looped-transformers-req9aufr1) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"OpenAI Astra and Looped Transformers","url":"https://daily.dev/posts/openai-astra-and-looped-transformers-req9aufr1","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/openai-astra-and-looped-transformers-req9aufr1"},"datePublished":"2026-09-02T15:09:30.886Z","dateModified":"2026-09-03T19:37:02.748Z","description":"Debunks hype claiming OpenAI's Astra uses a novel 'looped transformer' architecture that hides chain-of-thought reasoning. The looped transformer idea simply...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/ce2592cf5303d1c32553d5ebbd45deaa?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/ce2592cf5303d1c32553d5ebbd45deaa?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Sebastian Raschka","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Sebastian Raschka","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/eb22d2e73d074c5598d31588ad16a51c","url":"https://daily.dev/sources/sebastianraschka"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/openai-astra-and-looped-transformers-req9aufr1","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,llm,deep-learning,openai,prompt-engineering","timeRequired":"PT3M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Sebastian Raschka","item":"https://daily.dev/sources/sebastianraschka"},{"@type":"ListItem","position":3,"name":"OpenAI Astra and Looped Transformers"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/openai-astra-and-looped-transformers-req9aufr1#faq","mainEntity":[{"@type":"Question","name":"What is a looped transformer architecture in models like Nanbeige 4.2?","acceptedAnswer":{"@type":"Answer","text":"A looped transformer reuses the same layer stack multiple times instead of stacking unique layers, increasing effective depth without adding parameters. Nanbeige 4.2-3B reuses a 22-layer stack twice, extending it to an effective 44 layers while keeping storage and RAM the same, though compute cost nearly doubles since text passes through roughly twice as many layers. Anyone comparing efficient LLM architectures can follow deep dives like this on daily.dev."}},{"@type":"Question","name":"Does the looped transformer technique hide a model's chain-of-thought reasoning?","acceptedAnswer":{"@type":"Answer","text":"Not inherently. Reusing layers only adds extra computation within hidden states before a token is generated, the same way ordinary transformer layers do, so it does not by itself suppress visible chain-of-thought text. A plausible effect is that more recurrent passes could let a model need fewer intermediate reasoning tokens, shifting some computation into latent activations, but this is not unique to looped transformers versus simply scaling model size. Developers evaluating reasoning-model transparency claims can track architecture debates like this on daily.dev."}},{"@type":"Question","name":"Where did the idea of reusing transformer layers to increase model depth originate?","acceptedAnswer":{"@type":"Answer","text":"It traces to the Mixture-of-Recursions paper presented at NeurIPS, which introduced dynamic recursive depths for adaptive token-level computation. That approach adds a learned router deciding whether each token receives one, two, or more passes through the layer stack, letting easy tokens exit early while harder tokens get extra computation, a more sophisticated version of the fixed-loop approach used in Nanbeige 4.2. Researchers tracking the lineage of efficient LLM techniques can find papers like this surfaced on daily.dev."}}]} ```