--- title: How Zalando Built an In-Process Client-Side Load... description: Zalando's engineering team built an in-process client-side load balancer for their Product Read API, which handles ~1 million requests per second. The system... canonical: https://daily.dev/posts/how-zalando-built-an-in-process-client-side-load-balancer-for-one-million-requests-per-second-gwprc272c twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How Zalando Built an In-Process Client-Side Load Balancer for One Million Requests per Second | daily.dev og:description: Zalando's engineering team built an in-process client-side load balancer for their Product Read API, which handles ~1 million requests per second. The system... og:url: https://daily.dev/posts/how-zalando-built-an-in-process-client-side-load-balancer-for-one-million-requests-per-second-gwprc272c og:image: https://api.daily.dev/og/posts/gWPrc272c.png og:image:alt: How Zalando Built an In-Process Client-Side Load Balancer for One Million Requests per Second og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How Zalando Built an In-Process Client-Side Load Balancer for One Million Requests per Second **[InfoQ](https://daily.dev/sources/infoq)** · 4 min read · 0 upvotes · 0 comments ## Summary Zalando's engineering team built an in-process client-side load balancer for their Product Read API, which handles ~1 million requests per second. The system fans a single request into up to 100 parallel calls to individual product pods. By moving routing in-process rather than through Skipper (their shared edge load balancer), they achieved more predictable latency, reduced Skipper's fleet from 50+ pods to 8, and cut daily infrastructure costs from $450 to $110. Key engineering decisions included reimplementing Skipper's consistent hashing algorithm (xxHash64, 100 virtual nodes) to avoid cache splits during migration, using a watch-based Kubernetes informer instead of polling, and implementing N-ring fade-in over 30 seconds for new pod warm-up. AZ-aware routing was attempted but abandoned after it fragmented caches and caused DynamoDB read explosions. The team cautions that this approach is only justified at genuine extremes — for most teams, a mature proxy like Skipper or Envoy is the right answer. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why did Zalando build its own client-side load balancer instead of just using Skipper for internal traffic? Zalando's Product Read API batch endpoint fans one request into up to 100 parallel calls to product pods, each passing through Skipper, the shared cluster edge load balancer. Because a batch waits on the slowest of 100 hops through infrastructure the team didn't own, they couldn't separate Skipper's latency spikes from their own, so they moved routing for that high fan-out internal traffic in-process while keeping Skipper for edge and single-GET traffic. _Anyone weighing in-process routing versus a shared proxy can follow architecture writeups like this on daily.dev._ ### How did Zalando avoid cache splits when migrating from Skipper to a client-side load balancer? Zalando reimplemented Skipper's exact consistent-hashing algorithm, using xxHash64 with 100 virtual nodes per endpoint, so both the old and new routing paths mapped identical keys to identical pods during migration. Because both used the same hash function and virtual node count, they produced identical hash rings for the same set of pods, avoiding cache splits, and adding or removing an endpoint remaps only about 1/N of keys. _Teams planning a similar dual-path migration can track consistent-hashing techniques and case studies on daily.dev._ ### What went wrong when Zalando tried availability-zone-aware routing for its load balancer? Zalando's attempt at AZ-aware routing, intended to cut inter-AZ network costs, instead fragmented caches and triggered DynamoDB read explosions. The team abandoned that approach and instead hardened their client-side load balancer with a single jittered retry, FIFO overload shedding, and richer logging that surfaces brief node-level freezes so the balancer can route around them automatically. _Engineers evaluating AZ-aware routing tradeoffs can find real-world failure cases like this on daily.dev._ ## Similar posts on daily.dev - [Client-Side Load Balancing at a Million Requests Per Second](https://daily.dev/posts/client-side-load-balancing-at-a-million-requests-per-second-bvi2j48ww) · Zalando · 20 upvotes · 1 comments - [How Databricks Implemented Intelligent Kubernetes Load Balancing](https://daily.dev/posts/how-databricks-implemented-intelligent-kubernetes-load-balancing-0bcfej757) · ByteByteGo · 3 upvotes · 0 comments - [The Hidden Cost of “Just Works” Load Balancing in a Service Mesh](https://daily.dev/posts/the-hidden-cost-of-just-works-load-balancing-in-a-service-mesh-yohlgudu2) · Container Journal · 1 upvotes · 0 comments --- Tags: [#kubernetes](https://daily.dev/tags/kubernetes) [View this post on daily.dev](https://daily.dev/posts/how-zalando-built-an-in-process-client-side-load-balancer-for-one-million-requests-per-second-gwprc272c) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How Zalando Built an In-Process Client-Side Load Balancer for One Million Requests per Second","url":"https://daily.dev/posts/how-zalando-built-an-in-process-client-side-load-balancer-for-one-million-requests-per-second-gwprc272c","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-zalando-built-an-in-process-client-side-load-balancer-for-one-million-requests-per-second-gwprc272c"},"datePublished":"2026-07-25T06:49:13.010Z","dateModified":"2026-09-14T08:17:29.008Z","description":"Zalando's engineering team built an in-process client-side load balancer for their Product Read API, which handles ~1 million requests per second. The system...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d0663415b9c3e5031bcfe5c7dfd119f5?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d0663415b9c3e5031bcfe5c7dfd119f5?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"InfoQ","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"InfoQ","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/afc3bced3e1e4b188dd9127017a60e0c","url":"https://daily.dev/sources/infoq"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-zalando-built-an-in-process-client-side-load-balancer-for-one-million-requests-per-second-gwprc272c","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"kubernetes","timeRequired":"PT4M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"InfoQ","item":"https://daily.dev/sources/infoq"},{"@type":"ListItem","position":3,"name":"How Zalando Built an In-Process Client-Side Load Balancer for One Million Requests per Second"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-zalando-built-an-in-process-client-side-load-balancer-for-one-million-requests-per-second-gwprc272c#faq","mainEntity":[{"@type":"Question","name":"Why did Zalando build its own client-side load balancer instead of just using Skipper for internal traffic?","acceptedAnswer":{"@type":"Answer","text":"Zalando's Product Read API batch endpoint fans one request into up to 100 parallel calls to product pods, each passing through Skipper, the shared cluster edge load balancer. Because a batch waits on the slowest of 100 hops through infrastructure the team didn't own, they couldn't separate Skipper's latency spikes from their own, so they moved routing for that high fan-out internal traffic in-process while keeping Skipper for edge and single-GET traffic. Anyone weighing in-process routing versus a shared proxy can follow architecture writeups like this on daily.dev."}},{"@type":"Question","name":"How did Zalando avoid cache splits when migrating from Skipper to a client-side load balancer?","acceptedAnswer":{"@type":"Answer","text":"Zalando reimplemented Skipper's exact consistent-hashing algorithm, using xxHash64 with 100 virtual nodes per endpoint, so both the old and new routing paths mapped identical keys to identical pods during migration. Because both used the same hash function and virtual node count, they produced identical hash rings for the same set of pods, avoiding cache splits, and adding or removing an endpoint remaps only about 1/N of keys. Teams planning a similar dual-path migration can track consistent-hashing techniques and case studies on daily.dev."}},{"@type":"Question","name":"What went wrong when Zalando tried availability-zone-aware routing for its load balancer?","acceptedAnswer":{"@type":"Answer","text":"Zalando's attempt at AZ-aware routing, intended to cut inter-AZ network costs, instead fragmented caches and triggered DynamoDB read explosions. The team abandoned that approach and instead hardened their client-side load balancer with a single jittered retry, FIFO overload shedding, and richer logging that surfaces brief node-level freezes so the balancer can route around them automatically. Engineers evaluating AZ-aware routing tradeoffs can find real-world failure cases like this on daily.dev."}}]} ```