--- title: "From Homegrown to Flink: Migrating a Stateful Ad Event Join at Scale" url: https://daily.dev/posts/from-homegrown-to-flink-migrating-a-stateful-ad-event-join-at-scale-6lmjzq9eh source_url: https://engineering.zalando.com/posts/2026/07/migrating-ad-event-processing-to-flink.html type: article source: "Zalando" published: 2026-07-24T07:34:07.147Z updated: 2026-07-25T12:18:20.144Z tags: ["java", "apache-flink"] reading_time: 13 upvotes: 7 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # From Homegrown to Flink: Migrating a Stateful Ad Event Join at Scale **[Zalando](https://daily.dev/sources/zalando)** · 13 min read · 7 upvotes · 0 comments ## Summary Zalando's Ad Platform team replaced a 7-year-old homegrown in-memory stream join with Apache Flink to match ad auction events with user interactions in near-real time within a 15-minute window. The homegrown Java app lacked checkpointing and proper stream partitioning, causing state loss on restarts and requiring overprovisioning. After evaluating alternatives (Nakadi SQL, Spark), they chose Flink with a RocksDB state backend and 3-minute incremental checkpoints. A key optimization was replacing Flink's built-in Interval Join with a custom KeyedCoProcessFunction using direct RocksDB point lookups, eliminating costly seek operations that consumed 15–30% CPU. Extensive RocksDB tuning (write buffers, compaction sizes, LZ4 compression, SSD-optimized settings) was needed to stabilize state growth. Infrastructure challenges included Kinesis connector limitations, Karpenter pod evictions interfering with Flink's autoscaler, and OOM kills requiring careful JVM memory parameter tuning. After a 4-week shadow pipeline validation and 1-week A/B test, they cut average pod count from 20 to 5, reduced memory from 320GB to ~100GB, and halved EC2 costs from ~€80 to ~€30/day, while improving event match rate by 0.5%. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Why We Ditched Flink Table API Joins: Cutting State by 75% with DataStream Unions](https://daily.dev/posts/why-we-ditched-flink-table-api-joins-cutting-state-by-75-with-datastream-unions-ynzazjzo0) · Zalando · 0 upvotes · 0 comments - [How Medidata Cut Compute Costs by 80% Using Snowflake and Apache Iceberg](https://daily.dev/posts/how-medidata-cut-compute-costs-by-80-using-snowflake-and-apache-iceberg-led5ywgov) · Snowflake Community · 1 upvotes · 0 comments --- Tags: [#java](https://daily.dev/tags/java), [#apache-flink](https://daily.dev/tags/apache-flink) [View this post on daily.dev](https://daily.dev/posts/from-homegrown-to-flink-migrating-a-stateful-ad-event-join-at-scale-6lmjzq9eh)