--- title: "LyftLearn Evolution: Rethinking ML Platform Architecture" url: https://daily.dev/posts/lyftlearn-evolution-rethinking-ml-platform-architecture-tpztmqapa source_url: https://eng.lyft.com/lyftlearn-evolution-rethinking-ml-platform-architecture-547de6c950e1 type: article source: "Lyft Engineering" published: 2025-11-18T19:18:45.469Z updated: 2025-11-18T19:19:09.695Z tags: ["machine-learning", "aws", "kubernetes", "platform-engineering", "mlops"] reading_time: 18 upvotes: 0 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # LyftLearn Evolution: Rethinking ML Platform Architecture **[Lyft Engineering](https://daily.dev/sources/lyfteng)** · 18 min read · 0 upvotes · 0 comments ## Summary Lyft migrated their ML platform from a fully Kubernetes-based architecture to a hybrid approach, using AWS SageMaker for offline training and batch workloads while keeping Kubernetes for online model serving. The transition reduced operational complexity by eliminating custom orchestration logic, background watchers, and cluster management overhead. Key technical challenges included replicating the Kubernetes runtime environment, building cross-platform Docker images, optimizing startup times with SOCI indexes and warm pools, and solving cross-cluster networking for Spark. The migration was designed to be invisible to users, requiring zero changes to ML code while significantly improving system reliability and reducing compute costs. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Lyft Rearchitects ML Platform with Hybrid AWS SageMaker-Kubernetes Approach](https://daily.dev/posts/lyft-rearchitects-ml-platform-with-hybrid-aws-sagemaker-kubernetes-approach-ntubrebjb) · InfoQ · 0 upvotes · 0 comments - [How Lyft Built an ML Platform That Serves Millions of Predictions Per Second](https://daily.dev/posts/how-lyft-built-an-ml-platform-that-serves-millions-of-predictions-per-second-v99yv1odt) · ByteByteGo · 53 upvotes · 1 comments - [How Discord Scaled Its ML Platform from Single-GPU Workflows to a Shared Ray Cluster](https://daily.dev/posts/how-discord-scaled-its-ml-platform-from-single-gpu-workflows-to-a-shared-ray-cluster-hp3xg2b0s) · InfoQ · 1 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#aws](https://daily.dev/tags/aws), [#kubernetes](https://daily.dev/tags/kubernetes), [#platform-engineering](https://daily.dev/tags/platform-engineering), [#mlops](https://daily.dev/tags/mlops) [View this post on daily.dev](https://daily.dev/posts/lyftlearn-evolution-rethinking-ml-platform-architecture-tpztmqapa)