Up to 30x More Work Per Watt: NVIDIA Vera Rubin NVL72 Sets a New Efficiency Standard for AI Agents
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
NVIDIA reports new measured performance data showing its Vera Rubin NVL72 systems deliver up to 30x higher throughput per megawatt and 35x lower cost per million tokens than the previous-generation GB300 NVL72, based on real-world agentic coding workloads from SemiAnalysis's AgentX benchmark. The gains stem from techniques like disaggregated serving, expert parallelism, distributed KV-caching, NVFP4 quantization, and sixth-generation NVLink interconnects. Vera Rubin is described as in full production and scaling across NVIDIA's partner ecosystem, though these early results are still pending SemiAnalysis review and don't yet reflect Vera CPU tool-calling performance.
Table of contents
Vera Rubin NVL72: 30x Higher Throughput per Megawatt and 35x Lower Token CostExtreme Codesign for Agentic ScaleQuestions this post answers
How much more efficient is NVIDIA Vera Rubin NVL72 compared to GB300 NVL72 for agentic AI workloads?
Vera Rubin NVL72 delivers up to 30 times higher throughput per megawatt and up to 35 times lower cost per million tokens than GB300 NVL72, measured on real-world agentic coding trajectories using the SemiAnalysis AgentX workload. These early results, measured on the DeepSeek V4 Pro model, are currently pending SemiAnalysis review and do not yet reflect Vera CPU performance for tool calling. Teams sizing GPU infrastructure for agentic workloads can track hardware benchmarks like this on daily.dev.
What inference optimization techniques does NVIDIA use to improve agentic AI performance on Vera Rubin?
Vera Rubin NVL72 combines disaggregated serving (separating prefill and decode), rate matching between prefill and decode GPUs, large-scale expert parallelism for mixture-of-experts models, distributed and offloaded KV-caching, KV-aware request routing, and fused CUDA kernels like MegaMoE. NVFP4 4-bit quantization and sixth-generation NVLink interconnects, offering 10x higher packet rates and 3x lower latency than off-the-shelf Ethernet, further boost throughput. Engineers designing inference pipelines can follow codesign techniques like these on daily.dev.
Why do agentic AI workflows require more tokens than a simple chat request?
Agentic AI workloads consume roughly 15 times more tokens than a simple chat request because agents and sub-agents keep reasoning until a task completes, with accumulated context from each step feeding into the next. Context in agentic sessions can reach hundreds of thousands of input tokens, compared to the 1K-8K token range typical of chat or document summarization, making long-context handling central to performance. Developers building token-heavy agent pipelines can follow this kind of infrastructure analysis on daily.dev.