--- title: LiteLLM: AI Gateway on Kubernetes and Metrics in... description: Step-by-step walkthrough of deploying LiteLLM AI Gateway on AWS EKS using a custom Helm wrapper chart. Covers connecting AWS RDS PostgreSQL and Redis, managing... canonical: https://daily.dev/posts/litellm-ai-gateway-on-kubernetes-and-metrics-in-victoriametrics-hswmfwm8s twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: LiteLLM: AI Gateway on Kubernetes and Metrics in VictoriaMetrics | daily.dev og:description: Step-by-step walkthrough of deploying LiteLLM AI Gateway on AWS EKS using a custom Helm wrapper chart. Covers connecting AWS RDS PostgreSQL and Redis, managing... og:url: https://daily.dev/posts/litellm-ai-gateway-on-kubernetes-and-metrics-in-victoriametrics-hswmfwm8s og:image: https://api.daily.dev/og/posts/hswmFWM8S.png og:image:alt: LiteLLM: AI Gateway on Kubernetes and Metrics in VictoriaMetrics og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # LiteLLM: AI Gateway on Kubernetes and Metrics in VictoriaMetrics **[ITNEXT](https://daily.dev/sources/itnext)** · 17 min read · 1 upvotes · 0 comments ## Summary Step-by-step walkthrough of deploying LiteLLM AI Gateway on AWS EKS using a custom Helm wrapper chart. Covers connecting AWS RDS PostgreSQL and Redis, managing secrets via External Secrets Operator and AWS Secrets Manager, configuring Ingress on AWS ALB with PodDisruptionBudget for HA, and setting up Prometheus metrics collection into VictoriaMetrics via vmagent and VMPodScrape. Also demonstrates proxying Claude Code requests through LiteLLM and creating virtual API keys with role-based access. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do I fix an unauthorized error when scraping the LiteLLM /metrics endpoint even though the docs say it's unauthenticated by default? Set require_auth_for_metrics_endpoint to false explicitly in the litellm_settings section of the proxy config, since the Helm-deployed chart enforces authentication on /metrics by default despite documentation stating otherwise. Alternatively, create a proxy_admin_viewer role user via the LiteLLM API and pass its key as a bearer token when scraping the endpoint. _daily.dev surfaces practical fixes like this for teams wiring LiteLLM metrics into their monitoring stack._ ### How do I connect LiteLLM's proxy to an existing external PostgreSQL database instead of deploying a new one via the Helm chart? Set db.useExisting to true and db.deployStandalone to false in the LiteLLM Helm values, then provide the endpoint, database name, and a secret reference containing usernameKey and passwordKey pointing to credentials stored in a Kubernetes Secret (for example one synced from AWS Secrets Manager via the External Secrets Operator). _daily.dev helps engineers wiring LiteLLM into existing Postgres and Redis infrastructure compare configuration approaches._ ### How do I scrape metrics from multiple LiteLLM pod replicas into VictoriaMetrics? Create a VMPodScrape resource selecting pods by label (app.kubernetes.io/name: litellm) and specifying the http port and /metrics/ path, since a static_configs entry in vmagent's inlineScrapeConfig only targets a single service address and won't cover all pod replicas individually. This lets vmagent discover and scrape every LiteLLM pod rather than one endpoint. _daily.dev tracks setups like this for teams instrumenting multi-replica services with VictoriaMetrics._ ## Similar posts on daily.dev - [LiteLLM: Metrics, Traces, and VictoriaMetrics Stack Integration](https://daily.dev/posts/litellm-metrics-traces-and-victoriametrics-stack-integration-ybfi3rnfv) · ITNEXT · 0 upvotes · 0 comments - [The LLM Gateway Pattern: Why Every Kubernetes-Based AI App Needs One](https://daily.dev/posts/the-llm-gateway-pattern-why-every-kubernetes-based-ai-app-needs-one-isxnljenb) · freeCodeCamp · 4 upvotes · 0 comments - [LiteLLM: AI Gateway for LLMs — Features Overview](https://daily.dev/posts/litellm-ai-gateway-for-llms-features-overview-93hxfkgaq) · ITNEXT · 0 upvotes · 0 comments - [Open-weight AI on Kubernetes: A Developer’s Guide to vLLM vs. KubeAI](https://daily.dev/posts/open-weight-ai-on-kubernetes-a-developer-s-guide-to-vllm-vs-kubeai-hdxk8ddoh) · SitePoint · 0 upvotes · 0 comments - [How to route external and local LLMs with Models-as-a-Service](https://daily.dev/posts/how-to-route-external-and-local-llms-with-models-as-a-service-ufwzx55fl) · Red Hat Developer · 0 upvotes · 0 comments --- Tags: [#kubernetes](https://daily.dev/tags/kubernetes), [#helm](https://daily.dev/tags/helm), [#litellm](https://daily.dev/tags/litellm) [View this post on daily.dev](https://daily.dev/posts/litellm-ai-gateway-on-kubernetes-and-metrics-in-victoriametrics-hswmfwm8s) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"LiteLLM: AI Gateway on Kubernetes and Metrics in VictoriaMetrics","url":"https://daily.dev/posts/litellm-ai-gateway-on-kubernetes-and-metrics-in-victoriametrics-hswmfwm8s","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/litellm-ai-gateway-on-kubernetes-and-metrics-in-victoriametrics-hswmfwm8s"},"datePublished":"2026-07-20T08:30:44.344Z","dateModified":"2026-09-14T07:27:12.397Z","description":"Step-by-step walkthrough of deploying LiteLLM AI Gateway on AWS EKS using a custom Helm wrapper chart. Covers connecting AWS RDS PostgreSQL and Redis, managing...","image":"https://media.daily.dev/image/upload/s--qPvKM23u--/f_auto/v1722860399/public/Placeholder%2009","thumbnailUrl":"https://media.daily.dev/image/upload/s--qPvKM23u--/f_auto/v1722860399/public/Placeholder%2009","isAccessibleForFree":true,"articleSection":"ITNEXT","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"ITNEXT","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/itnext2","url":"https://daily.dev/sources/itnext"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/litellm-ai-gateway-on-kubernetes-and-metrics-in-victoriametrics-hswmfwm8s","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"kubernetes,helm,litellm","timeRequired":"PT17M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"ITNEXT","item":"https://daily.dev/sources/itnext"},{"@type":"ListItem","position":3,"name":"LiteLLM: AI Gateway on Kubernetes and Metrics in VictoriaMetrics"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/litellm-ai-gateway-on-kubernetes-and-metrics-in-victoriametrics-hswmfwm8s#faq","mainEntity":[{"@type":"Question","name":"How do I fix an unauthorized error when scraping the LiteLLM /metrics endpoint even though the docs say it's unauthenticated by default?","acceptedAnswer":{"@type":"Answer","text":"Set require_auth_for_metrics_endpoint to false explicitly in the litellm_settings section of the proxy config, since the Helm-deployed chart enforces authentication on /metrics by default despite documentation stating otherwise. Alternatively, create a proxy_admin_viewer role user via the LiteLLM API and pass its key as a bearer token when scraping the endpoint. daily.dev surfaces practical fixes like this for teams wiring LiteLLM metrics into their monitoring stack."}},{"@type":"Question","name":"How do I connect LiteLLM's proxy to an existing external PostgreSQL database instead of deploying a new one via the Helm chart?","acceptedAnswer":{"@type":"Answer","text":"Set db.useExisting to true and db.deployStandalone to false in the LiteLLM Helm values, then provide the endpoint, database name, and a secret reference containing usernameKey and passwordKey pointing to credentials stored in a Kubernetes Secret (for example one synced from AWS Secrets Manager via the External Secrets Operator). daily.dev helps engineers wiring LiteLLM into existing Postgres and Redis infrastructure compare configuration approaches."}},{"@type":"Question","name":"How do I scrape metrics from multiple LiteLLM pod replicas into VictoriaMetrics?","acceptedAnswer":{"@type":"Answer","text":"Create a VMPodScrape resource selecting pods by label (app.kubernetes.io/name: litellm) and specifying the http port and /metrics/ path, since a static_configs entry in vmagent's inlineScrapeConfig only targets a single service address and won't cover all pod replicas individually. This lets vmagent discover and scrape every LiteLLM pod rather than one endpoint. daily.dev tracks setups like this for teams instrumenting multi-replica services with VictoriaMetrics."}}]} ```