--- title: Google Just Dropped Transformer 2.0: Meet "Nested Learning" description: Google researchers introduced "Nested Learning," a new machine learning paradigm that treats model architecture and optimization as unified, nested processes... canonical: https://daily.dev/posts/ttqlfvb3q twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Google Just Dropped Transformer 2.0: Meet "Nested Learning" | daily.dev og:description: Google researchers introduced "Nested Learning," a new machine learning paradigm that treats model architecture and optimization as unified, nested processes... og:url: https://daily.dev/posts/ttqlfvb3q og:image: https://api.daily.dev/og/posts/ttQlFVb3q.png og:image:alt: Post cover image og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Google Just Dropped Transformer 2.0: Meet "Nested Learning" **[ML & AI](https://daily.dev/sources/ml_ai)** · [@fotiecodes](https://daily.dev/fotiecodes) · 0 upvotes · 0 comments ## Summary Google researchers introduced "Nested Learning," a new machine learning paradigm that treats model architecture and optimization as unified, nested processes rather than separate components. The approach addresses catastrophic forgetting by enabling multi-timescale updates inspired by human neuroplasticity. Their proof-of-concept architecture, Hope, uses a Continuum Memory System that manages memory as a spectrum with modules updating at different rates. Hope outperforms standard transformers on language modeling and excels at needle-in-haystack tasks. The framework also introduces "Deep Optimizers" that use L2 regression loss instead of dot-product similarity, creating more robust momentum variants. This represents a shift toward continual learning systems that can acquire new skills without erasing existing knowledge. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Introducing Nested Learning: A new ML paradigm for continual learning](https://daily.dev/posts/introducing-nested-learning-a-new-ml-paradigm-for-continual-learning-6hcdyxq2q) · Hacker News · 2 upvotes · 0 comments - [These startups are chasing the next big thing in LLMs](https://daily.dev/posts/these-startups-are-chasing-the-next-big-thing-in-llms-g8wfdvibc) · MIT Technology Review · 0 upvotes · 0 comments - [CNNs, RNNs, and Transformers Explained: A Mental Model for Key Deep Learning Concepts](https://daily.dev/posts/cnns-rnns-and-transformers-explained-a-mental-model-for-key-deep-learning-concepts-pxbxlxvct) · freeCodeCamp · 3 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#google](https://daily.dev/tags/google), [#deep-learning](https://daily.dev/tags/deep-learning), [#neural-networks](https://daily.dev/tags/neural-networks), [#transformers](https://daily.dev/tags/transformers) [View this post on daily.dev](https://daily.dev/posts/ttqlfvb3q) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"DiscussionForumPosting","mainEntityOfPage":"https://daily.dev/posts/ttqlfvb3q","headline":"Google Just Dropped Transformer 2.0: Meet \"Nested Learning\"","text":"Shared: Google Just Dropped Transformer 2.0: Meet \"Nested Learning\"","url":"https://daily.dev/posts/ttqlfvb3q","datePublished":"2026-01-24T15:17:46.184Z","dateModified":"2026-01-24T15:17:46.187Z","author":{"@type":"Person","name":"fotiecodes","url":"https://daily.dev/fotiecodes","image":"https://avatars.githubusercontent.com/u/42372656?v=4","interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"EndorseAction"},"userInteractionCount":1810}},"image":"https://media.daily.dev/image/upload/s--2-1xRawN--/f_auto/v1722860399/public/Placeholder%2011","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"sharedContent":{"@type":"WebPage","url":"https://api.daily.dev/r/XH7Vjui7c"},"isPartOf":{"@type":"WebPage","url":"https://daily.dev/squads/ml_ai","name":"ML & AI"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"ML & AI","item":"https://daily.dev/squads/ml_ai"},{"@type":"ListItem","position":3,"name":"Google Just Dropped Transformer 2.0: Meet \"Nested Learning\""}]} ```