--- title: The Practitioner's Guide to the Maximal Update... description: Maximal Update Parameterization (μP) enables stable hyperparameter transfer across model scales, reducing costly tuning and improving training stability for... canonical: https://daily.dev/posts/the-practitioner-s-guide-to-the-maximal-update-parameterization-pysbctcde twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: The Practitioner's Guide to the Maximal Update Parameterization | daily.dev og:description: Maximal Update Parameterization (μP) enables stable hyperparameter transfer across model scales, reducing costly tuning and improving training stability for... og:url: https://daily.dev/posts/the-practitioner-s-guide-to-the-maximal-update-parameterization-pysbctcde og:image: https://api.daily.dev/og/posts/pySBcTCde.png og:image:alt: The Practitioner's Guide to the Maximal Update Parameterization og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # The Practitioner's Guide to the Maximal Update Parameterization **[EleutherAI](https://daily.dev/sources/eleutherai)** · 15 min read · 0 upvotes · 0 comments ## Summary Maximal Update Parameterization (μP) enables stable hyperparameter transfer across model scales, reducing costly tuning and improving training stability for large language models. The guide covers the core math behind μP — controlling activation magnitudes in forward pass, backward pass, and weight updates — and provides a practical implementation guide for transformer models. Key benefits include transferring optimal HPs from small proxy models to large-scale runs (μTransfer), improved loss at scale, and tighter scaling law fits. A reference nanoGPT implementation with coordinate check and μTransfer verification tests is provided. Implementation requires targeted changes: adjusting weight initialization variance, learning rate scaling per layer type, output logit scaling, and attention logit scaling. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [From SGD to Muon: A Field Guide to Neural Network Optimizers](https://daily.dev/posts/from-sgd-to-muon-a-field-guide-to-neural-network-optimizers-mtzsf6vyp) · Medium · 1 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#deep-learning](https://daily.dev/tags/deep-learning) [View this post on daily.dev](https://daily.dev/posts/the-practitioner-s-guide-to-the-maximal-update-parameterization-pysbctcde) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"The Practitioner's Guide to the Maximal Update Parameterization","url":"https://daily.dev/posts/the-practitioner-s-guide-to-the-maximal-update-parameterization-pysbctcde","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/the-practitioner-s-guide-to-the-maximal-update-parameterization-pysbctcde"},"datePublished":"2026-08-02T07:04:30.225Z","dateModified":"2026-08-02T07:31:52.871Z","description":"Maximal Update Parameterization (μP) enables stable hyperparameter transfer across model scales, reducing costly tuning and improving training stability for...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7fa3fbd6e99da7e05ab42115e7e68fde?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7fa3fbd6e99da7e05ab42115e7e68fde?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"EleutherAI","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"EleutherAI","logo":"https://media.daily.dev/image/upload/s--aAcRyWGF--/f_auto,q_auto/v1785654263/logos/eleutherai","url":"https://daily.dev/sources/eleutherai"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/the-practitioner-s-guide-to-the-maximal-update-parameterization-pysbctcde","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,deep-learning","timeRequired":"PT15M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"EleutherAI","item":"https://daily.dev/sources/eleutherai"},{"@type":"ListItem","position":3,"name":"The Practitioner's Guide to the Maximal Update Parameterization"}]} ```