--- title: Load balancing and scaling long-lived connections in... description: Kubernetes Services don't actually load balance long-lived connections such as HTTP keep-alive, gRPC, WebSockets, AMQP, or database connections, because... canonical: https://daily.dev/posts/load-balancing-and-scaling-long-lived-connections-in-kubernetes-cx4gzpn0d twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Load balancing and scaling long-lived connections in Kubernetes | daily.dev og:description: Kubernetes Services don't actually load balance long-lived connections such as HTTP keep-alive, gRPC, WebSockets, AMQP, or database connections, because... og:url: https://daily.dev/posts/load-balancing-and-scaling-long-lived-connections-in-kubernetes-cx4gzpn0d og:image: https://api.daily.dev/og/posts/Cx4GzPN0D.png og:image:alt: Load balancing and scaling long-lived connections in Kubernetes og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Load balancing and scaling long-lived connections in Kubernetes **[LearnKube](https://daily.dev/sources/learnkube)** · 14 min read · 0 upvotes · 0 comments ## Summary Kubernetes Services don't actually load balance long-lived connections such as HTTP keep-alive, gRPC, WebSockets, AMQP, or database connections, because iptables (or ipvs/eBPF) rules only get invoked once per TCP connection setup, not per request. This means one Pod can end up handling disproportionately more traffic than its siblings. The piece walks through how kube-proxy implements Services via iptables probability rules, why persistent connections bypass that mechanism, and how to fix it: client-side load balancing (retrieving endpoints from a headless Service and rotating connections), database-specific solutions like JDBC's loadBalanceHosts or pgpool, or offloading the problem to a proxy/service mesh like Envoy, HAProxy, Istio, or Linkerd. It closes with guidance on when the imbalance actually matters based on client-to-server ratio. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why does Kubernetes send all my gRPC or WebSocket traffic to only one pod instead of spreading it across replicas? Kubernetes Services load balance by rewriting the destination IP via iptables rules only when a new TCP connection is established. Protocols like gRPC, HTTP/2, WebSockets, and AMQP reuse a single long-lived TCP connection for many requests, so the iptables rule fires once and all subsequent traffic sticks to that same pod, leaving other replicas underutilized. _For persistent-connection workloads on kubernetes, daily.dev surfaces client-side load balancing and service mesh approaches worth comparing._ ### How can I load balance queries across multiple PostgreSQL replicas from a Java or Python app? In JDBC, appending loadBalanceHosts=true to a multi-host connection string like jdbc:postgresql://node1,node2,node3/database distributes queries across the listed Postgres replicas automatically. SQLAlchemy, by contrast, only tries multiple IPs in sequence until one connects and then keeps that single connection stable, without balancing. An external proxy like pgpool is another option since it accepts a single endpoint and load balances queries across replicas itself. _Developers weighing driver-level versus proxy-based load balancing for postgres track these tradeoffs on daily.dev._ ### Does HTTP keep-alive break load balancing in Kubernetes? Yes, once a TCP connection is opened with HTTP keep-alive enabled, the Kubernetes Service's iptables rule is only invoked on that first connection, and every subsequent HTTP request over the same connection goes to the same pod. This improves latency and resource usage but sacrifices even distribution across backend replicas, especially with few clients and many server pods. _Teams enabling keep-alive on kubernetes backends can follow these load-balancing tradeoffs via daily.dev._ ## Similar posts on daily.dev - [How Databricks Implemented Intelligent Kubernetes Load Balancing](https://daily.dev/posts/how-databricks-implemented-intelligent-kubernetes-load-balancing-0bcfej757) · ByteByteGo · 3 upvotes · 0 comments - [Kubernetes Networking Explained Without the Hand-Waving](https://daily.dev/posts/kubernetes-networking-explained-without-the-hand-waving-b9yscwngi) · Medium · 5 upvotes · 1 comments --- Tags: [#kubernetes](https://daily.dev/tags/kubernetes), [#postgresql](https://daily.dev/tags/postgresql), [#grpc](https://daily.dev/tags/grpc), [#service-mesh](https://daily.dev/tags/service-mesh) [View this post on daily.dev](https://daily.dev/posts/load-balancing-and-scaling-long-lived-connections-in-kubernetes-cx4gzpn0d) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Load balancing and scaling long-lived connections in Kubernetes","url":"https://daily.dev/posts/load-balancing-and-scaling-long-lived-connections-in-kubernetes-cx4gzpn0d","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/load-balancing-and-scaling-long-lived-connections-in-kubernetes-cx4gzpn0d"},"datePublished":"2026-08-31T11:20:03.434Z","dateModified":"2026-08-31T11:20:30.816Z","description":"Kubernetes Services don't actually load balance long-lived connections such as HTTP keep-alive, gRPC, WebSockets, AMQP, or database connections, because...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/5a88832962da1f376b0cd275e641467e?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/5a88832962da1f376b0cd275e641467e?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"LearnKube","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"LearnKube","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/90786df1b5d44abb91a090a0b6c81517","url":"https://daily.dev/sources/learnkube"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/load-balancing-and-scaling-long-lived-connections-in-kubernetes-cx4gzpn0d","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"kubernetes,postgresql,grpc,service-mesh","timeRequired":"PT14M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"LearnKube","item":"https://daily.dev/sources/learnkube"},{"@type":"ListItem","position":3,"name":"Load balancing and scaling long-lived connections in Kubernetes"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/load-balancing-and-scaling-long-lived-connections-in-kubernetes-cx4gzpn0d#faq","mainEntity":[{"@type":"Question","name":"Why does Kubernetes send all my gRPC or WebSocket traffic to only one pod instead of spreading it across replicas?","acceptedAnswer":{"@type":"Answer","text":"Kubernetes Services load balance by rewriting the destination IP via iptables rules only when a new TCP connection is established. Protocols like gRPC, HTTP/2, WebSockets, and AMQP reuse a single long-lived TCP connection for many requests, so the iptables rule fires once and all subsequent traffic sticks to that same pod, leaving other replicas underutilized. For persistent-connection workloads on kubernetes, daily.dev surfaces client-side load balancing and service mesh approaches worth comparing."}},{"@type":"Question","name":"How can I load balance queries across multiple PostgreSQL replicas from a Java or Python app?","acceptedAnswer":{"@type":"Answer","text":"In JDBC, appending loadBalanceHosts=true to a multi-host connection string like jdbc:postgresql://node1,node2,node3/database distributes queries across the listed Postgres replicas automatically. SQLAlchemy, by contrast, only tries multiple IPs in sequence until one connects and then keeps that single connection stable, without balancing. An external proxy like pgpool is another option since it accepts a single endpoint and load balances queries across replicas itself. Developers weighing driver-level versus proxy-based load balancing for postgres track these tradeoffs on daily.dev."}},{"@type":"Question","name":"Does HTTP keep-alive break load balancing in Kubernetes?","acceptedAnswer":{"@type":"Answer","text":"Yes, once a TCP connection is opened with HTTP keep-alive enabled, the Kubernetes Service's iptables rule is only invoked on that first connection, and every subsequent HTTP request over the same connection goes to the same pod. This improves latency and resource usage but sacrifices even distribution across backend replicas, especially with few clients and many server pods. Teams enabling keep-alive on kubernetes backends can follow these load-balancing tradeoffs via daily.dev."}}]} ```