--- title: Making Softmax More Efficient with NVIDIA Blackwell Ultra description: NVIDIA Blackwell Ultra (GB300) doubles the throughput of Special Function Units (SFUs) responsible for computing exponential functions (MUFU.EX2), directly... canonical: https://daily.dev/posts/making-softmax-more-efficient-with-nvidia-blackwell-ultra-c04kqgrxq twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Making Softmax More Efficient with NVIDIA Blackwell Ultra | daily.dev og:description: NVIDIA Blackwell Ultra (GB300) doubles the throughput of Special Function Units (SFUs) responsible for computing exponential functions (MUFU.EX2), directly... og:url: https://daily.dev/posts/making-softmax-more-efficient-with-nvidia-blackwell-ultra-c04kqgrxq og:image: https://api.daily.dev/og/posts/c04kqgrXq.png og:image:alt: Making Softmax More Efficient with NVIDIA Blackwell Ultra og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Making Softmax More Efficient with NVIDIA Blackwell Ultra **[NVIDIA Developer](https://daily.dev/sources/nvidiadev)** · 9 min read · 0 upvotes · 0 comments ## Summary NVIDIA Blackwell Ultra (GB300) doubles the throughput of Special Function Units (SFUs) responsible for computing exponential functions (MUFU.EX2), directly targeting the softmax bottleneck in LLM attention mechanisms. As context lengths grow and architectures like MLA and GQA increase the density of softmax operations relative to matrix multiplications, SFUs become the limiting factor that forces Tensor Cores to idle. The post explains the attention loop pipeline (BMM1 → softmax → BMM2), shows how doubled SFU throughput compresses the softmax phase by ~50%, and provides a CUDA micro-benchmark to measure raw MUFU.EX2 throughput. Benchmark results confirm ~2x SFU throughput on GB300 vs GB200 and a ~35% gain in FP8 forward propagation throughput for models like DeepSeek-V3. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Scaling Token Factory Revenue and AI Efficiency by Maximizing Performance per Watt](https://daily.dev/posts/scaling-token-factory-revenue-and-ai-efficiency-by-maximizing-performance-per-watt-2qzgn7uql) · NVIDIA Developer · 0 upvotes · 0 comments - [Delivering Massive Performance Leaps for Mixture of Experts Inference on NVIDIA Blackwell](https://daily.dev/posts/delivering-massive-performance-leaps-for-mixture-of-experts-inference-on-nvidia-blackwell-psxmr99dv) · NVIDIA Developer · 1 upvotes · 0 comments - [NVIDIA Extreme Co-Design Delivers New MLPerf Inference Records](https://daily.dev/posts/nvidia-extreme-co-design-delivers-new-mlperf-inference-records-jwjjtv0yx) · NVIDIA Developer · 1 upvotes · 0 comments - [New SemiAnalysis InferenceX Data Shows NVIDIA Blackwell Ultra Delivers up to 50x Better Performance and 35x Lower Costs for Agentic AI](https://daily.dev/posts/new-semianalysis-inferencex-data-shows-nvidia-blackwell-ultra-delivers-up-to-50x-better-performance--xlkxksrjj) · NVIDIA · 0 upvotes · 0 comments --- Tags: [#cuda](https://daily.dev/tags/cuda) [View this post on daily.dev](https://daily.dev/posts/making-softmax-more-efficient-with-nvidia-blackwell-ultra-c04kqgrxq) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Making Softmax More Efficient with NVIDIA Blackwell Ultra","url":"https://daily.dev/posts/making-softmax-more-efficient-with-nvidia-blackwell-ultra-c04kqgrxq","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/making-softmax-more-efficient-with-nvidia-blackwell-ultra-c04kqgrxq"},"datePublished":"2026-02-25T17:00:57.891Z","dateModified":"2026-02-25T17:01:22.995Z","description":"NVIDIA Blackwell Ultra (GB300) doubles the throughput of Special Function Units (SFUs) responsible for computing exponential functions (MUFU.EX2), directly...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/6359a0ae40a66c87bf8a4e5341f6e619?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/6359a0ae40a66c87bf8a4e5341f6e619?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"NVIDIA Developer","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"NVIDIA Developer","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/86e45aab42ba48ce83103d01b1119910","url":"https://daily.dev/sources/nvidiadev"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/making-softmax-more-efficient-with-nvidia-blackwell-ultra-c04kqgrxq","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"cuda","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"NVIDIA Developer","item":"https://daily.dev/sources/nvidiadev"},{"@type":"ListItem","position":3,"name":"Making Softmax More Efficient with NVIDIA Blackwell Ultra"}]} ```