---
title: "Client-Side Load Balancing at a Million Requests Per Second"
url: https://daily.dev/posts/client-side-load-balancing-at-a-million-requests-per-second-bvi2j48ww
source_url: https://engineering.zalando.com/posts/2026/06/client-side-load-balancing.html
type: article
source: "Zalando"
published: 2026-06-23T07:39:46.015Z
updated: 2026-06-24T14:20:45.913Z
tags: ["kubernetes", "java", "distributed-systems"]
reading_time: 31
upvotes: 20
comments: 1
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Client-Side Load Balancing at a Million Requests Per Second

**[Zalando](https://daily.dev/sources/zalando)** · 31 min read · 20 upvotes · 1 comments

## Summary

Zalando's engineering team built an in-process client-side load balancer (CSLB) to handle over a million requests per second of internal fan-out traffic for their Product Read API, replacing shared Skipper ingress hops. The implementation replicates Skipper's xxHash64 consistent-hash ring for cache locality, uses a Kubernetes watch-based informer for pod discovery, and adds N-ring fade-in to prevent cold-cache spikes on scale-up. A key innovation is occupancy-based bounded load using Little's Law (seconds of work per second) rather than in-flight counts or throughput, combined with a latency multiplier borrowed from Finagle. Results include eliminating Skipper's fleet from 50+ pods to 8, reducing their own pod fleet by 25%, and saving over $1,000/day. AZ-aware routing was prototyped but paused due to edge cases around bounded-load threshold miscalculation during dual fade-in. The post also covers pipeline improvements, retry hardening, FIFO buffering, and how detailed logging revealed mysterious node-level network freezes that had previously been invisible.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://engineering.zalando.com/posts/2026/06/client-side-load-balancing.html>

## Similar posts on daily.dev

- [How Zalando Built an In-Process Client-Side Load Balancer for One Million Requests per Second](https://daily.dev/posts/how-zalando-built-an-in-process-client-side-load-balancer-for-one-million-requests-per-second-gwprc272c) · InfoQ · 0 upvotes · 0 comments
- [How Databricks Implemented Intelligent Kubernetes Load Balancing](https://daily.dev/posts/how-databricks-implemented-intelligent-kubernetes-load-balancing-0bcfej757) · ByteByteGo · 3 upvotes · 0 comments

---

Tags: [#kubernetes](https://daily.dev/tags/kubernetes), [#java](https://daily.dev/tags/java), [#distributed-systems](https://daily.dev/tags/distributed-systems)

[View this post on daily.dev](https://daily.dev/posts/client-side-load-balancing-at-a-million-requests-per-second-bvi2j48ww)