---
title: "uber-sites"
url: https://daily.dev/posts/uber-sites-cteihjinc
source_url: https://www.uber.com/en-JO/blog/balancing-hdfs-datanodes-in-the-uber-datalake/
type: article
source: "Uber Engineering"
published: 2024-03-14T15:45:37.060Z
updated: 2024-05-09T09:28:05.588Z
tags: ["apache-hadoop"]
reading_time: 9
upvotes: 0
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# uber-sites

**[Uber Engineering](https://daily.dev/sources/uber)** · 9 min read · 0 upvotes · 0 comments

## Summary

Uber has developed a new algorithm to optimize the HDFS balancer, increasing data balance and throughput in their large-scale data storage clusters. The algorithm focuses on increasing the number of DataNode pairs, prioritizing smaller occupied DataNodes, and improving observability. As a result, the throughput increased by more than 5x, highly utilized DataNodes were brought down to 0%, overall usage of DataNodes was reduced, and cluster utilization improved.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://www.uber.com/en-JO/blog/balancing-hdfs-datanodes-in-the-uber-datalake/>

## Similar posts on daily.dev

- [Don’t just attend KubeCon \+ CloudNativeCon, Merge Forward your experience\!](https://daily.dev/posts/don-t-just-attend-kubecon-cloudnativecon-merge-forward-your-experience--l0rpp73x8) · CNCF · 0 upvotes · 0 comments
- [Announcing H2 2026 KCDs](https://daily.dev/posts/announcing-h2-2026-kcds-m96goajm1) · CNCF · 1 upvotes · 0 comments
- [Two months of Open Community Groups](https://daily.dev/posts/two-months-of-open-community-groups-asf52zhbs) · CNCF · 0 upvotes · 0 comments

---

Tags: [#apache-hadoop](https://daily.dev/tags/apache-hadoop)

[View this post on daily.dev](https://daily.dev/posts/uber-sites-cteihjinc)