--- title: NVIDIA's Nemotron Ultra is a 550B-parameter open-weight... description: NVIDIA released Nemotron 3 Ultra, a 550B open-weight model using sparse activation (only ~55B parameters active per token) with a hybrid Mamba-Transformer... canonical: https://daily.dev/posts/nvidia-s-nemotron-ultra-is-a-550b-parameter-open-weight-model-topping-us-benchmarks-x9de5pmja twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: NVIDIA's Nemotron Ultra is a 550B-parameter open-weight model topping US benchmarks | daily.dev og:description: NVIDIA released Nemotron 3 Ultra, a 550B open-weight model using sparse activation (only ~55B parameters active per token) with a hybrid Mamba-Transformer... og:url: https://daily.dev/posts/nvidia-s-nemotron-ultra-is-a-550b-parameter-open-weight-model-topping-us-benchmarks-x9de5pmja og:image: https://api.daily.dev/og/posts/X9DE5PMJa.png og:image:alt: NVIDIA's Nemotron Ultra is a 550B-parameter open-weight model topping US benchmarks og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # NVIDIA's Nemotron Ultra is a 550B-parameter open-weight model topping US benchmarks **[Collections](https://daily.dev/sources/collections)** · 4 min read · 6 upvotes · 0 comments ## Summary NVIDIA released Nemotron 3 Ultra, a 550B open-weight model using sparse activation (only ~55B parameters active per token) with a hybrid Mamba-Transformer architecture. Key features include a 1M token context window, LatentMoE routing, Multi-head Latent Attention for KV cache compression, and NVFP4 quantization delivering ~5x throughput on Blackwell GPUs. Training used Multi-Teacher On-Policy Distillation (MOPD) across 10+ specialized teacher models, plus RL on agentic task trajectories. It scores 65–70.4% on SWEBench Verified and delivers 300–350 tokens/second on supported hardware. Weights are available on Hugging Face under OpenMDW-1.1, with NVIDIA also open-sourcing training recipes, datasets, and RL environments. Two companion models shipped alongside: a 600M streaming ASR model and a 4B content safety guardrail model. No multimodal support; local deployment requires hundreds of GBs of GPU memory. ## Content ## What it is NVIDIA released Nemotron 3 Ultra, a 550 billion parameter open-weight model that only activates about 55 billion parameters per token — roughly 10% of the total. That sparse activation is what makes it fast enough to be practical. It's already sitting at the top of open-weight benchmarks according to Artificial Analysis, beating out several larger models from Chinese labs on agentic tasks despite having fewer active parameters. The weights are available on Hugging Face under the OpenMDW-1.1 license, which NVIDIA describes as comparable to Apache 2.0 for ML weights. You can also access it through NVIDIA's cloud, OpenRouter, Perplexity (Pro and Max subscribers), Vercel AI Gateway, and major cloud providers including AWS, Google Cloud, and Azure. ## Architecture The model uses a hybrid Mamba-Transformer design rather than a pure transformer stack. Mamba layers handle long-context sequences more efficiently than standard attention, which matters when you're working with the 1 million token context window the model supports. Other architectural details: - **LatentMoE** for efficient expert routing - **Multi-head Latent Attention (MLA)** to compress the KV cache - **Multi-token prediction** to speed up autoregressive generation - **NVFP4 quantization** on Blackwell GPUs, delivering roughly 5x throughput over BF16 On Hopper and Blackwell hardware, NVFP4 support is built in. The vLLM team shipped day-0 support for the model, and vLLM was also used during training for high-throughput rollouts and evaluation. ## How it was trained The most interesting training detail is what NVIDIA calls Multi-Teacher On-Policy Distillation (MOPD). Rather than training a single large model from scratch on everything, they trained more than 10 specialized teacher models — each focused on a domain like code, tool use, or instruction following — then distilled all of them into Nemotron 3 Ultra. The idea is that each teacher is better at its specialty than a generalist model would be, and the student inherits that cross-domain strength. Post-training used reinforcement learning on agent harness trajectories, meaning the model was trained on actual agentic task completions rather than just supervised examples. NVIDIA is also open-sourcing the training recipes, datasets, and RL environments, so organizations can fine-tune the model for specific use cases. ### Synthetic pretraining data A separate pipeline called task-seeded synthetic Q&A generation contributed to pretraining. The workflow pulls training splits from roughly 70 public task families via lm-eval-harness, generates new similar questions, enriches answers with reasoning and task-relevant context, then filters the results. In a 100B-token continuation experiment on Nemotron-3 Nano, this approach added +1.8 on MMLU-Pro, +1.9 on average code, +1.6 on commonsense, and +11.1 on GPQA, while keeping math scores stable. ## Performance On SWEBench Verified (a coding benchmark), the model scores between 65% and 70.4%. It delivers over 300 tokens per second on supported hardware, and Vercel reports up to 350 tokens/second. NVIDIA claims up to 30% lower cost on agentic tasks compared to other leading open models. The model is specifically designed for long-running autonomous agent workflows: planning, tool calling, sub-agent delegation, error recovery, deep research, and enterprise automation. It supports OpenAI-compatible API endpoints. ## What it can't do No vision or multimodal capabilities. The full model requires hundreds of gigabytes of GPU memory, so local deployment is out of reach for most people. Cloud GPU providers like Lambda are the practical option if you want to self-host. In hands-on testing, it handles quick scripting and terminal tasks well but struggles with complex graphics simulations and game development code. ## The rest of the release NVIDIA shipped two companion models alongside Ultra: - **Nemotron 3.5 ASR** — a 600M parameter multilingual streaming speech-to-text model with sub-100ms latency - **Nemotron 3.5 Content Safety** — a 4B guardrail model covering 23 safety categories All weights, data, and training recipes are open under OpenMDW-1.1. ## Context This release lands in a busy week for open models. Microsoft announced seven new MAI models at BUILD including MAI-Thinking-1. Ideogram 4.0 became the top open-weights text-to-image model. Xiaomi's Mimo V2.5 Pro, a 1T+ parameter MoE, is leading open-weight benchmarks in long-context reasoning. The open-weight space is moving fast, and Nemotron 3 Ultra is a meaningful entry — particularly for teams building agentic systems who want something they can actually run and modify. ## Similar posts on daily.dev - [NVIDIA Nemotron 3 Ultra: 10x cheaper than GPT-4o](https://daily.dev/posts/nvidia-nemotron-3-ultra-10x-cheaper-than-gpt-4o-7r1ve6mgp) · FireUp · 18 upvotes · 4 comments - [Nvidia’s best model is now live](https://daily.dev/posts/nvidia-s-best-model-is-now-live-tsagrzqp8) · The New Stack · 0 upvotes · 0 comments --- Tags: [#ai](https://daily.dev/tags/ai), [#llm](https://daily.dev/tags/llm), [#nvidia](https://daily.dev/tags/nvidia) [View this post on daily.dev](https://daily.dev/posts/nvidia-s-nemotron-ultra-is-a-550b-parameter-open-weight-model-topping-us-benchmarks-x9de5pmja) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"NVIDIA's Nemotron Ultra is a 550B-parameter open-weight model topping US benchmarks","url":"https://daily.dev/posts/nvidia-s-nemotron-ultra-is-a-550b-parameter-open-weight-model-topping-us-benchmarks-x9de5pmja","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/nvidia-s-nemotron-ultra-is-a-550b-parameter-open-weight-model-topping-us-benchmarks-x9de5pmja"},"datePublished":"2026-06-01T18:05:05.202Z","dateModified":"2026-06-18T15:47:36.886Z","description":"NVIDIA released Nemotron 3 Ultra, a 550B open-weight model using sparse activation (only ~55B parameters active per token) with a hybrid Mamba-Transformer...","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/nvidia-s-nemotron-ultra-is-a-550b-parameter-open-weight-model-topping-us-benchmarks-x9de5pmja","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":6},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai,llm,nvidia","timeRequired":"PT4M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"NVIDIA's Nemotron Ultra is a 550B-parameter open-weight model topping US benchmarks"}]} ```