--- title: How NVIDIA Builds Open Models for the Age of AI | daily.dev description: NVIDIA is the world's largest publisher of open AI models, with families spanning reasoning (Nemotron), physical AI and robotics (Cosmos, Isaac GR00T),... canonical: https://daily.dev/posts/how-nvidia-builds-open-models-for-the-age-of-ai-bjuhqj0tg twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How NVIDIA Builds Open Models for the Age of AI | daily.dev og:description: NVIDIA is the world's largest publisher of open AI models, with families spanning reasoning (Nemotron), physical AI and robotics (Cosmos, Isaac GR00T),... og:url: https://daily.dev/posts/how-nvidia-builds-open-models-for-the-age-of-ai-bjuhqj0tg og:image: https://api.daily.dev/og/posts/BjUhQJ0TG.png og:image:alt: How NVIDIA Builds Open Models for the Age of AI og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How NVIDIA Builds Open Models for the Age of AI **[ByteByteGo](https://daily.dev/sources/bytebytego)** · 16 min read · 17 upvotes · 0 comments ## Summary NVIDIA is the world's largest publisher of open AI models, with families spanning reasoning (Nemotron), physical AI and robotics (Cosmos, Isaac GR00T), autonomous vehicles (Alpamayo), drug discovery (BioNeMo), quantum computing (Ising), and climate forecasting (Earth-2). Bryan Catanzaro, VP of Applied Deep Learning Research, explains the key architectural choices: a hybrid Mamba+Transformer design that handles million-token contexts efficiently, mixture-of-experts layers, and 4-bit (NVFP4) pretraining co-designed with Blackwell GPUs. Post-training uses supervised fine-tuning followed by large-scale reinforcement learning across diverse environments. A unified foundation strategy—reusing backbones like Cosmos Reason across robotics and AV teams—lets a small team ship many models quickly. NVIDIA's open approach goes beyond releasing weights: it publishes training datasets, RL environments, and recipes. The business rationale is that open models grow the AI ecosystem, which in turn drives GPU demand, while also keeping NVIDIA's researchers honest about where AI is heading. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What architecture does NVIDIA use to make its Nemotron models fast and capable at the same time? NVIDIA uses a hybrid architecture that swaps most Transformer attention layers for Mamba (state space model) layers, keeping only a few attention layers for exact recall. Mamba layers scale linearly with sequence length and keep memory constant, making long context windows practical, while the remaining attention layers restore pinpoint recall. Mixture-of-experts layers are added on top so only a subset of parameters activate per token. _Developers comparing model architectures for long-context or efficient inference can track these design tradeoffs on daily.dev._ ### Why did NVIDIA pretrain its large models in 4-bit precision instead of higher precision? NVIDIA pretrained its larger models in a 4-bit number format called NVFP4 from the very first training step, rather than training at higher precision and shrinking afterward as most teams do. This was possible because NVIDIA co-designed the approach with its Blackwell GPU generation, which was built with fast 4-bit hardware, reducing memory use and compute cost while preserving accuracy. _Teams weighing training precision and hardware tradeoffs can follow this kind of model-hardware co-design work on daily.dev._ ### Why does NVIDIA release its AI models as open source when it also sells the GPUs they run on? NVIDIA open-sources models like Nemotron and Cosmos because building and releasing them publicly keeps its researchers connected to real users and honest about where AI is heading, which informs future chip design. Commercially, NVIDIA benefits when any company adopts open models and grows its own AI usage, since that demand ultimately drives GPU purchases, so NVIDIA supports the ecosystem rather than competing with it. _Developers deciding whether to build on open versus closed models can weigh this reasoning via daily.dev._ ## Similar posts on daily.dev - [How Open Models Are Driving AI Research](https://daily.dev/posts/how-open-models-are-driving-ai-research-y1h7ugicm) · NVIDIA · 1 upvotes · 0 comments - [Nvidia brings together AI labs to build the next generation of open base models](https://daily.dev/posts/nvidia-brings-together-ai-labs-to-build-the-next-generation-of-open-base-models-a6skrnnsz) · The New Stack · 0 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#open-source](https://daily.dev/tags/open-source), [#llm](https://daily.dev/tags/llm), [#nvidia](https://daily.dev/tags/nvidia) [View this post on daily.dev](https://daily.dev/posts/how-nvidia-builds-open-models-for-the-age-of-ai-bjuhqj0tg) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How NVIDIA Builds Open Models for the Age of AI","url":"https://daily.dev/posts/how-nvidia-builds-open-models-for-the-age-of-ai-bjuhqj0tg","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-nvidia-builds-open-models-for-the-age-of-ai-bjuhqj0tg"},"datePublished":"2026-07-27T15:08:02.685Z","dateModified":"2026-09-13T21:06:58.938Z","description":"NVIDIA is the world's largest publisher of open AI models, with families spanning reasoning (Nemotron), physical AI and robotics (Cosmos, Isaac GR00T),...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7363ec67ae60870488fb6df86fd9c39c?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7363ec67ae60870488fb6df86fd9c39c?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"ByteByteGo","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"ByteByteGo","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/35be29234ee14d01a9cd049c52e12753","url":"https://daily.dev/sources/bytebytego"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-nvidia-builds-open-models-for-the-age-of-ai-bjuhqj0tg","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":17},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,open-source,llm,nvidia","timeRequired":"PT16M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"ByteByteGo","item":"https://daily.dev/sources/bytebytego"},{"@type":"ListItem","position":3,"name":"How NVIDIA Builds Open Models for the Age of AI"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-nvidia-builds-open-models-for-the-age-of-ai-bjuhqj0tg#faq","mainEntity":[{"@type":"Question","name":"What architecture does NVIDIA use to make its Nemotron models fast and capable at the same time?","acceptedAnswer":{"@type":"Answer","text":"NVIDIA uses a hybrid architecture that swaps most Transformer attention layers for Mamba (state space model) layers, keeping only a few attention layers for exact recall. Mamba layers scale linearly with sequence length and keep memory constant, making long context windows practical, while the remaining attention layers restore pinpoint recall. Mixture-of-experts layers are added on top so only a subset of parameters activate per token. Developers comparing model architectures for long-context or efficient inference can track these design tradeoffs on daily.dev."}},{"@type":"Question","name":"Why did NVIDIA pretrain its large models in 4-bit precision instead of higher precision?","acceptedAnswer":{"@type":"Answer","text":"NVIDIA pretrained its larger models in a 4-bit number format called NVFP4 from the very first training step, rather than training at higher precision and shrinking afterward as most teams do. This was possible because NVIDIA co-designed the approach with its Blackwell GPU generation, which was built with fast 4-bit hardware, reducing memory use and compute cost while preserving accuracy. Teams weighing training precision and hardware tradeoffs can follow this kind of model-hardware co-design work on daily.dev."}},{"@type":"Question","name":"Why does NVIDIA release its AI models as open source when it also sells the GPUs they run on?","acceptedAnswer":{"@type":"Answer","text":"NVIDIA open-sources models like Nemotron and Cosmos because building and releasing them publicly keeps its researchers connected to real users and honest about where AI is heading, which informs future chip design. Commercially, NVIDIA benefits when any company adopts open models and grows its own AI usage, since that demand ultimately drives GPU purchases, so NVIDIA supports the ecosystem rather than competing with it. Developers deciding whether to build on open versus closed models can weigh this reasoning via daily.dev."}}]} ```