--- title: "How Spotify Deployed Kong's AI Gateway to Power Generative AI at Scale" url: https://daily.dev/posts/how-spotify-deployed-kong-s-ai-gateway-to-power-generative-ai-at-scale-faonlx9mr source_url: https://engineering.atspotify.com/2026/7/how-spotify-deployed-kongs-ai-gateway-to-power-generative-ai-at-scale type: article source: "Spotify Labs" published: 2026-07-14T17:04:54.240Z updated: 2026-07-14T18:22:27.824Z tags: ["observability", "api-gateway", "ai-gateway"] reading_time: 9 upvotes: 5 comments: 1 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How Spotify Deployed Kong's AI Gateway to Power Generative AI at Scale **[Spotify Labs](https://daily.dev/sources/spotify)** · 9 min read · 5 upvotes · 1 comments ## Summary Spotify faced fragmented LLM integrations across hundreds of internal teams, each solving the same authentication, rate limiting, cost tracking, and failover problems independently. To unify this, they deployed Kong's AI Gateway in May 2025 on GKE in hybrid mode, with a control plane backed by PostgreSQL and stateless data plane proxies. The gateway uses a plugin chain covering multi-provider routing (OpenAI, Vertex AI, first-party Hendrix models), Prometheus-based token-level telemetry, OpenTelemetry distributed tracing, and AI content guardrails. Over 1,000 internal services have onboarded, reducing new team integration from weeks to an afternoon config change. Future plans include A/B model testing via Spotify's Confidence platform and expanded support for agentic workflows. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community discussion Top comments from developers on daily.dev. **@pdfopsdev** · 0 upvotes > A gateway hop costing ~40ms is a dealbreaker for normal API traffic but rounding error against multi-second LLM completions — that latency asymmetry is why 'put everything behind the gateway' actually works for AI traffic when it never quite did for microservices. The per-consumer token attribution is the sleeper feature here; most orgs still can't answer 'which team spent what on LLMs last month'. ## Similar posts on daily.dev - [KubeCon NA 2025 - Erica Hughberg and Alexa Griffith on Tools for the Age of GenAI](https://daily.dev/posts/kubecon-na-2025---erica-hughberg-and-alexa-griffith-on-tools-for-the-age-of-genai-pavyctafs) · InfoQ · 0 upvotes · 0 comments --- Tags: [#observability](https://daily.dev/tags/observability), [#api-gateway](https://daily.dev/tags/api-gateway), [#ai-gateway](https://daily.dev/tags/ai-gateway) [View this post on daily.dev](https://daily.dev/posts/how-spotify-deployed-kong-s-ai-gateway-to-power-generative-ai-at-scale-faonlx9mr)