--- title: "How Uber Migrated from Hive to Spark SQL for ETL Workloads" url: https://daily.dev/posts/how-uber-migrated-from-hive-to-spark-sql-for-etl-workloads-vurlnv6pa source_url: https://www.uber.com/blog/how-uber-migrated-from-hive-to-spark-sql-for-etl-workloads/ type: article source: "Uber Engineering" published: 2025-06-12T13:24:58.556Z updated: 2025-06-12T13:25:24.293Z tags: ["uber", "apache-spark", "etl"] reading_time: 15 upvotes: 3 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How Uber Migrated from Hive to Spark SQL for ETL Workloads **[Uber Engineering](https://daily.dev/sources/uber)** · 15 min read · 3 upvotes · 0 comments ## Summary Uber successfully migrated 18,000 Hive ETL workflows generating 5 million monthly queries to Spark SQL, achieving 50% reduction in runtime and resource usage. The migration involved building three core services: Query Translation Service for converting HiveQL to Spark SQL, Data Validation Service for ensuring output consistency, and Automated Migration Service for orchestrating shadow testing. Key challenges included handling syntax differences, floating-point precision issues, non-deterministic functions, and performance gaps like file creation patterns. The team developed custom solutions including Hive-compatible behaviors in Spark, expression translations, and partition rebalancing to maintain data consistency while leveraging Spark's performance benefits. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Don’t just attend KubeCon \+ CloudNativeCon, Merge Forward your experience\!](https://daily.dev/posts/don-t-just-attend-kubecon-cloudnativecon-merge-forward-your-experience--l0rpp73x8) · CNCF · 0 upvotes · 0 comments - [Announcing H2 2026 KCDs](https://daily.dev/posts/announcing-h2-2026-kcds-m96goajm1) · CNCF · 1 upvotes · 0 comments - [Two months of Open Community Groups](https://daily.dev/posts/two-months-of-open-community-groups-asf52zhbs) · CNCF · 0 upvotes · 0 comments - [CNCF Unveils Schedule for KubeCon \+ CloudNativeCon Europe 2026](https://daily.dev/posts/cncf-unveils-schedule-for-kubecon-cloudnativecon-europe-2026-ikhcoa5cb) · CNCF · 2 upvotes · 0 comments - [CNCF Debuts KubeCon \+ CloudNativeCon Japan 2026 Schedule](https://daily.dev/posts/cncf-debuts-kubecon-cloudnativecon-japan-2026-schedule-xp5pyudub) · CNCF · 1 upvotes · 0 comments --- Tags: [#uber](https://daily.dev/tags/uber), [#apache-spark](https://daily.dev/tags/apache-spark), [#etl](https://daily.dev/tags/etl) [View this post on daily.dev](https://daily.dev/posts/how-uber-migrated-from-hive-to-spark-sql-for-etl-workloads-vurlnv6pa)