--- title: The worst part of using multiple LLM providers is not... description: When integrating multiple LLM providers, the real complexity isn't API format differences — it's the operational logic that follows: fallbacks, retries, load... canonical: https://daily.dev/posts/the-worst-part-of-using-multiple-llm-providers-is-not-the-api-differences-2pncww8kq twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: The worst part of using multiple LLM providers is not the API differences | daily.dev og:description: When integrating multiple LLM providers, the real complexity isn't API format differences — it's the operational logic that follows: fallbacks, retries, load... og:url: https://daily.dev/posts/the-worst-part-of-using-multiple-llm-providers-is-not-the-api-differences-2pncww8kq og:image: https://api.daily.dev/og/posts/2PncwW8KQ.png og:image:alt: The worst part of using multiple LLM providers is not the API differences og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # The worst part of using multiple LLM providers is not the API differences **[Swapnoneel Saha](https://daily.dev/sources/ldi7vqbpkryv936wnukjd)** · [@swapnoneel](https://daily.dev/swapnoneel) · 2 min read · 4 upvotes · 0 comments ## Summary When integrating multiple LLM providers, the real complexity isn't API format differences — it's the operational logic that follows: fallbacks, retries, load balancing, cost tracking, and observability. This logic tends to get duplicated across services. Bifrost, an open-source LLM gateway, addresses this by centralizing provider selection, fallback rules, caching, budgets, and observability into a single layer. The analogy is drawn to traditional infrastructure patterns where networking, auth, and traffic policies live in shared infrastructure rather than individual applications. The post argues that as LLM usage scales, direct SDK calls become insufficient and a gateway layer starts to make sense. ## Content When we started using more than one LLM provider, I expected the main problem to be API compatibility. It was not. Changing request formats is annoying, but it is usually easy to solve. The harder problem starts after the first request fails. What should happen if a model is unavailable? Should the request go to another provider? What if that provider uses a different model? Should we retry? How many times? What happens to latency? How do we know which provider handled the request? How do we measure cost across all of them? This logic grows very quickly. And if you have several applications, the same logic starts to appear in several codebases. That is the problem Bifrost is trying to solve. [Bifrost ](https://github.com/maximhq/bifrost)is an open-source LLM gateway. Instead of putting provider logic inside every application, you put much of it in one common layer. The application sends a request to Bifrost. Bifrost can then handle provider selection, fallbacks, retries, load balancing, caching, budgets, and observability. The interesting part for me is not that it gives you one API. There are many ways to normalize APIs. The interesting part is that it gives you one place to define what should happen when things go wrong. That becomes more useful as the system grows. For example, imagine that three internal services use the same LLM provider. Without a gateway, all three services may have their own retry logic and fallback rules. Now imagine that you want to change the fallback provider. You have to change three services. With a gateway, that policy can live in one place. This is a common infrastructure pattern outside AI. We usually do not want every application to implement its own networking, authentication, logging, or traffic policy. LLM access seems to be moving in the same direction. At first, direct API calls are enough. Later, the model call becomes infrastructure. That is when a gateway starts to make more sense. I am curious where other teams draw this line. At what point did direct provider SDKs stop being enough for you? ## Similar posts on daily.dev - [Top 5 LLM Gateways in 2026: A Deep-Dive Comparison for Production Teams](https://daily.dev/posts/top-5-llm-gateways-in-2026-a-deep-dive-comparison-for-production-teams-izylo6on6) · DEV · 3 upvotes · 0 comments - [I Compared 5 LLM Gateway Tools for Real-World Production Use](https://daily.dev/posts/i-compared-5-llm-gateway-tools-for-real-world-production-use-m17rwdojp) · DEV · 2 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#observability](https://daily.dev/tags/observability) [View this post on daily.dev](https://daily.dev/posts/the-worst-part-of-using-multiple-llm-providers-is-not-the-api-differences-2pncww8kq) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"DiscussionForumPosting","mainEntityOfPage":"https://daily.dev/posts/the-worst-part-of-using-multiple-llm-providers-is-not-the-api-differences-2pncww8kq","headline":"The worst part of using multiple LLM providers is not the API differences","text":"When integrating multiple LLM providers, the real complexity isn't API format differences — it's the operational logic that follows: fallbacks, retries, load balancing, cost tracking, and observability. This logic tends to get duplicated across services. Bifrost, an open-source LLM gateway, addresses this by centralizing provider selection, fallback rules, caching, budgets, and observability into a single layer. The analogy is drawn to traditional infrastructure patterns where networking, auth, and traffic policies live in shared infrastructure rather than individual applications. The post argues that as LLM usage scales, direct SDK calls become insufficient and a gateway layer starts to make sense.","url":"https://daily.dev/posts/the-worst-part-of-using-multiple-llm-providers-is-not-the-api-differences-2pncww8kq","datePublished":"2026-08-11T18:10:39.955Z","dateModified":"2026-09-13T20:49:07.509Z","author":{"@type":"Person","name":"Swapnoneel Saha","url":"https://daily.dev/swapnoneel","image":"https://media.daily.dev/image/upload/s--7WuNOXsT--/f_auto/v1783760285/avatars/avatar_Ldi7VQbPKrYV936wNUKJD?_a=BAMAMicg0","description":"Technical Growth Operator","interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"EndorseAction"},"userInteractionCount":2450}},"image":"https://media.daily.dev/image/upload/s--_yQZQ9LC--/f_auto/v1786471841/posts/2PncwW8KQ?_a=BAMAMicg0","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":4},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"isPartOf":{"@type":"WebPage","url":"https://daily.dev/sources/ldi7vqbpkryv936wnukjd","name":"Swapnoneel Saha"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Swapnoneel Saha","item":"https://daily.dev/sources/ldi7vqbpkryv936wnukjd"},{"@type":"ListItem","position":3,"name":"The worst part of using multiple LLM providers is not the API differences"}]} ```