Netflix operates over 30,000 Apache Flink jobs and has been running two autoscalers: an in-house one built around 2019 that watches cluster-level metrics from outside and scales the whole TaskManager count as one knob, and the Apache Flink community's OSS autoscaler that reasons from inside the job using each operator's true processing rate to compute per-vertex parallelism. The homegrown scaler cuts resource usage 25-45% but can't handle multi-operator stateful DAGs or catch metric blind spots. The OSS autoscaler, adopted for custom jobs and reaching general availability last year, required Netflix-specific engineering: a Temporal-based workflow-per-job architecture, JobManager metric-collection fixes for jobs up to 3,000 subtasks, forward-chain-aware scaling, and sink-backpressure detection. One team cut annualized Flink compute costs 58%, saving about $1.1 million a year. Netflix now runs target utilization at 0.45 instead of the community default of 0.7 for stability, and plans to migrate fully onto the OSS-based autoscaler while exploring Flink 2's disaggregated state architecture to reduce state-recovery costs during rescaling.
Table of contents
Why autoscaling is not optional at our scaleThe first autoscaler: watching from outsideThe second autoscaler: reasoning from insideMaking it work at Netflix scaleGet Netflix Technology Blog ’s stories in your inboxThe road to one autoscalerKey TakeawaysQuestions this post answers
What is true processing rate (TPR) in the Apache Flink autoscaler and how is it calculated?
True processing rate is an estimate of the throughput an operator could sustain if it were fully busy, calculated by dividing observed throughput by the fraction of time the operator spent actually working (its busy fraction). For example, an operator handling 700 records per second while busy 70% of the time has a TPR of 1,000 records per second. The Flink Kubernetes Operator autoscaler uses this per-operator estimate, along with input/output ratios and a target utilization, to compute the parallelism each vertex in the job graph needs. Engineers tuning Flink autoscaling can follow architecture writeups like this on daily.dev.
Why does scaling one side of a Flink forward connection independently cause problems?
Two vertices joined by a forward connection must run at the same parallelism because records pass between them in memory over a fixed local channel. Scaling only one of them does not cause Flink to fail outright; instead it silently converts that edge into a network shuffle, changing performance characteristics without warning. Netflix's internal Flink fork detects forward-connected subgraphs and scales each as a single unit to avoid this. Teams debugging unexpected Flink shuffle behavior can track fixes like this via daily.dev.
How much did Netflix save by switching a Flink job to the OSS-based autoscaler?
Netflix's client telemetry and logging team cut its annualized Flink compute spend by 58%, saving approximately $1.1 million a year, after adopting the OSS-based autoscaler for general availability. The savings came from dynamically adapting to daily traffic cycles, continually right-sizing capacity instead of relying on manual tuning, and using uniform container dimensions for better bin-packing. Engineers building a cost case for autoscaling adoption can find case studies like this on daily.dev.
13.9K Impressions1 Comment