---
title: "The Hidden Cost of “Just Works” Load Balancing in a Service Mesh"
url: https://daily.dev/posts/the-hidden-cost-of-just-works-load-balancing-in-a-service-mesh-yohlgudu2
source_url: https://cloudnativenow.com/contributed-content/the-hidden-cost-of-just-works-load-balancing-in-a-service-mesh
type: article
source: "Container Journal"
published: 2026-08-14T22:48:41.219Z
updated: 2026-08-14T22:51:44.712Z
tags: ["aws", "kubernetes", "istio", "service-mesh"]
reading_time: 5
upvotes: 1
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# The Hidden Cost of “Just Works” Load Balancing in a Service Mesh

**[Container Journal](https://daily.dev/sources/container_journal)** · 5 min read · 1 upvotes · 0 comments

## Summary

Multi-AZ Kubernetes clusters running Istio default to random load balancing across availability zones, which quietly adds latency and AWS cross-AZ data transfer costs. Tracing data from a mid-sized cluster running ~3,500 RPS showed same-AZ p50 latency of 15-18ms versus 25-30ms cross-AZ, and cross-AZ traffic charges around $600/month before optimization. The fix is Istio's locality-aware load balancing via DestinationRule, using an 80/10/10 weighted split rather than 100% local-only routing, combined with even pod distribution across AZs, spread ingress gateways, and disabling NLB cross-zone load balancing. The tradeoff is added operational complexity in reasoning about traffic distribution.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://cloudnativenow.com/contributed-content/the-hidden-cost-of-just-works-load-balancing-in-a-service-mesh>

## Questions this post answers

### How do I configure Istio DestinationRule for locality-aware load balancing across availability zones?

Set localityLbSetting.enabled to true in the DestinationRule's trafficPolicy.loadBalancer, then use the distribute field to specify weighted percentages per zone, such as 80% to the local AZ and 10% each to the other two. Pair this with outlierDetection settings like consecutiveErrors: 5, interval: 30s, and baseEjectionTime: 30s so unhealthy endpoints are automatically ejected.

_daily.dev surfaces practical Istio traffic policy configs for engineers tuning multi-AZ mesh setups._

### Why is my cross-AZ traffic in a multi-AZ Kubernetes cluster with Istio so expensive and slow?

Kubernetes Services and Istio's Envoy sidecars distribute traffic randomly across all healthy endpoints with no zone awareness, so a pod has roughly a two-in-three chance of calling a downstream pod in a different AZ across three zones. In one mid-sized cluster running about 3,500 RPS, this pushed weighted p50 latency to around 24ms instead of 17ms and generated roughly $600 per month in AWS cross-AZ data transfer charges at $0.01/GB.

_Engineers debugging mesh latency and AWS bandwidth bills can track fixes like this on daily.dev._

### Should I route 100% of Istio traffic to the local availability zone for cost savings?

No, routing all traffic locally removes the ability to absorb a bad deploy or a hot pod in that zone and makes recovery from a full AZ failure much harder. A better pattern is a weighted split, such as 80% same-AZ traffic with 10% each to the other two zones, combined with outlier detection so unhealthy endpoints get ejected automatically.

_daily.dev helps infrastructure teams weigh tradeoffs like locality routing before rolling changes to production._

## Similar posts on daily.dev

- [Cost-Effective Reliability: Making Sense of Multi-Zone Kubernetes](https://daily.dev/posts/cost-effective-reliability-making-sense-of-multi-zone-kubernetes-rjalhbgjg) · Cloud Native Now · 0 upvotes · 0 comments
- [Client-Side Load Balancing at a Million Requests Per Second](https://daily.dev/posts/client-side-load-balancing-at-a-million-requests-per-second-bvi2j48ww) · Zalando · 20 upvotes · 1 comments
- [How Zalando Built an In-Process Client-Side Load Balancer for One Million Requests per Second](https://daily.dev/posts/how-zalando-built-an-in-process-client-side-load-balancer-for-one-million-requests-per-second-gwprc272c) · InfoQ · 0 upvotes · 0 comments

---

Tags: [#aws](https://daily.dev/tags/aws), [#kubernetes](https://daily.dev/tags/kubernetes), [#istio](https://daily.dev/tags/istio), [#service-mesh](https://daily.dev/tags/service-mesh)

[View this post on daily.dev](https://daily.dev/posts/the-hidden-cost-of-just-works-load-balancing-in-a-service-mesh-yohlgudu2)