--- title: Route AI Agents Across Models with NVIDIA NeMo Switchyard description: NVIDIA NeMo Switchyard is an open-source library for routing AI agent workloads across multiple models based on task requirements, cost, and latency. It offers... canonical: https://daily.dev/posts/route-ai-agents-across-models-with-nvidia-nemo-switchyard-umybpp8ek twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Route AI Agents Across Models with NVIDIA NeMo Switchyard | daily.dev og:description: NVIDIA NeMo Switchyard is an open-source library for routing AI agent workloads across multiple models based on task requirements, cost, and latency. It offers... og:url: https://daily.dev/posts/route-ai-agents-across-models-with-nvidia-nemo-switchyard-umybpp8ek og:image: https://api.daily.dev/og/posts/uMYBpP8eK.png og:image:alt: Route AI Agents Across Models with NVIDIA NeMo Switchyard og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Route AI Agents Across Models with NVIDIA NeMo Switchyard **[NVIDIA Developer](https://daily.dev/sources/nvidiadev)** · 10 min read · 0 upvotes · 0 comments ## Summary NVIDIA NeMo Switchyard is an open-source library for routing AI agent workloads across multiple models based on task requirements, cost, and latency. It offers tuning-free routers (LLM classifier, stage router, escalation router) and tunable routers (prefill router) that evaluate signals from model capabilities, cost profiles, and infrastructure. Real-world benchmarks show significant results: LangChain testing with the escalation router achieved a 74% cost reduction at a ~6-point accuracy tradeoff by sending only 7% of calls to the frontier model. Cognition's staged-routing in Devin Desktop achieved 50.6% accuracy at $3.11 mean cost—within 2.8 percentage points of frontier accuracy at ~28% lower cost. The system integrates with LiteLLM, Kong, LangChain, and other tools, and supports OpenAI, Anthropic, and Responses API formats. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What cost savings did NeMo Switchyard's escalation router achieve in LangChain benchmarks? Using the escalation router to route between NVIDIA Nemotron 3.5 Lightning and Claude Opus 4.8, LangChain's 145-task multi-turn agentic benchmark achieved a 74% cost reduction compared to a frontier-only baseline, sending just 7% of calls to the frontier model, at a measured ~6-point accuracy tradeoff across five runs. _Teams optimizing AI agent costs track routing benchmarks like these on daily.dev._ ### What routing algorithms does NVIDIA NeMo Switchyard support? NeMo Switchyard offers tuning-free routers — LLM classifier (uses an LLM judge to select and maintain session affinity), stage router (examines tool activity to match model capability to agent phase), and escalation router (starts cheap, escalates on sustained difficulty) — plus a tunable prefill router that extracts residual stream signals and trains a shared-trunk MLP to predict per-model accuracy. _Developers choosing between routing strategies for their agent stacks follow NeMo Switchyard updates on daily.dev._ ### How did Cognition's staged routing with NeMo Switchyard perform on FrontierCode Main? Routing between Opus 5 and Kimi K2.7 on FrontierCode Main, Cognition's benchmark for production-grade coding tasks, achieved 50.6% accuracy at a $3.11 mean cost — within 2.8 percentage points of Opus 5 accuracy at approximately 28% lower mean cost, deployed in Devin Desktop for real-world NVIDIA internal testing. _Engineers evaluating cost-accuracy tradeoffs in coding agents find benchmark comparisons like this on daily.dev._ ## Similar posts on daily.dev - [Nvidia launches a smaller, faster Nemotron model and a router to put it to work](https://daily.dev/posts/nvidia-launches-a-smaller-faster-nemotron-model-and-a-router-to-put-it-to-work-adgtw6klw) · The New Stack · 0 upvotes · 0 comments - [MoE Routing for Faster, Cheaper AI Inference](https://daily.dev/posts/moe-routing-for-faster-cheaper-ai-inference-9qjnbnbah) · DigitalOcean Community · 0 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#ai-gateway](https://daily.dev/tags/ai-gateway), [#llm](https://daily.dev/tags/llm), [#nvidia](https://daily.dev/tags/nvidia) [View this post on daily.dev](https://daily.dev/posts/route-ai-agents-across-models-with-nvidia-nemo-switchyard-umybpp8ek) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Route AI Agents Across Models with NVIDIA NeMo Switchyard","url":"https://daily.dev/posts/route-ai-agents-across-models-with-nvidia-nemo-switchyard-umybpp8ek","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/route-ai-agents-across-models-with-nvidia-nemo-switchyard-umybpp8ek"},"datePublished":"2026-08-11T13:04:59.606Z","dateModified":"2026-10-15T02:22:18.930Z","description":"NVIDIA NeMo Switchyard is an open-source library for routing AI agent workloads across multiple models based on task requirements, cost, and latency. It offers...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/3071b09da352939d9b3d7163f274e7ad?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/3071b09da352939d9b3d7163f274e7ad?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"NVIDIA Developer","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"NVIDIA Developer","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/86e45aab42ba48ce83103d01b1119910","url":"https://daily.dev/sources/nvidiadev"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/route-ai-agents-across-models-with-nvidia-nemo-switchyard-umybpp8ek","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents,ai-gateway,llm,nvidia","timeRequired":"PT10M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"NVIDIA Developer","item":"https://daily.dev/sources/nvidiadev"},{"@type":"ListItem","position":3,"name":"Route AI Agents Across Models with NVIDIA NeMo Switchyard"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/route-ai-agents-across-models-with-nvidia-nemo-switchyard-umybpp8ek#faq","mainEntity":[{"@type":"Question","name":"What cost savings did NeMo Switchyard's escalation router achieve in LangChain benchmarks?","acceptedAnswer":{"@type":"Answer","text":"Using the escalation router to route between NVIDIA Nemotron 3.5 Lightning and Claude Opus 4.8, LangChain's 145-task multi-turn agentic benchmark achieved a 74% cost reduction compared to a frontier-only baseline, sending just 7% of calls to the frontier model, at a measured ~6-point accuracy tradeoff across five runs. Teams optimizing AI agent costs track routing benchmarks like these on daily.dev."}},{"@type":"Question","name":"What routing algorithms does NVIDIA NeMo Switchyard support?","acceptedAnswer":{"@type":"Answer","text":"NeMo Switchyard offers tuning-free routers — LLM classifier (uses an LLM judge to select and maintain session affinity), stage router (examines tool activity to match model capability to agent phase), and escalation router (starts cheap, escalates on sustained difficulty) — plus a tunable prefill router that extracts residual stream signals and trains a shared-trunk MLP to predict per-model accuracy. Developers choosing between routing strategies for their agent stacks follow NeMo Switchyard updates on daily.dev."}},{"@type":"Question","name":"How did Cognition's staged routing with NeMo Switchyard perform on FrontierCode Main?","acceptedAnswer":{"@type":"Answer","text":"Routing between Opus 5 and Kimi K2.7 on FrontierCode Main, Cognition's benchmark for production-grade coding tasks, achieved 50.6% accuracy at a $3.11 mean cost — within 2.8 percentage points of Opus 5 accuracy at approximately 28% lower mean cost, deployed in Devin Desktop for real-world NVIDIA internal testing. Engineers evaluating cost-accuracy tradeoffs in coding agents find benchmark comparisons like this on daily.dev."}}]} ```