--- title: From Homegrown to Flink: Migrating a Stateful Ad Event... description: Zalando's Ad Platform team replaced a 7-year-old homegrown in-memory stream join with Apache Flink to match ad auction events with user interactions in... canonical: https://daily.dev/posts/from-homegrown-to-flink-migrating-a-stateful-ad-event-join-at-scale-6lmjzq9eh twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: From Homegrown to Flink: Migrating a Stateful Ad Event Join at Scale | daily.dev og:description: Zalando's Ad Platform team replaced a 7-year-old homegrown in-memory stream join with Apache Flink to match ad auction events with user interactions in... og:url: https://daily.dev/posts/from-homegrown-to-flink-migrating-a-stateful-ad-event-join-at-scale-6lmjzq9eh og:image: https://api.daily.dev/og/posts/6lmJZQ9eh.png og:image:alt: From Homegrown to Flink: Migrating a Stateful Ad Event Join at Scale og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # From Homegrown to Flink: Migrating a Stateful Ad Event Join at Scale **[Zalando](https://daily.dev/sources/zalando)** · 13 min read · 7 upvotes · 0 comments ## Summary Zalando's Ad Platform team replaced a 7-year-old homegrown in-memory stream join with Apache Flink to match ad auction events with user interactions in near-real time within a 15-minute window. The homegrown Java app lacked checkpointing and proper stream partitioning, causing state loss on restarts and requiring overprovisioning. After evaluating alternatives (Nakadi SQL, Spark), they chose Flink with a RocksDB state backend and 3-minute incremental checkpoints. A key optimization was replacing Flink's built-in Interval Join with a custom KeyedCoProcessFunction using direct RocksDB point lookups, eliminating costly seek operations that consumed 15–30% CPU. Extensive RocksDB tuning (write buffers, compaction sizes, LZ4 compression, SSD-optimized settings) was needed to stabilize state growth. Infrastructure challenges included Kinesis connector limitations, Karpenter pod evictions interfering with Flink's autoscaler, and OOM kills requiring careful JVM memory parameter tuning. After a 4-week shadow pipeline validation and 1-week A/B test, they cut average pod count from 20 to 5, reduced memory from 320GB to ~100GB, and halved EC2 costs from ~€80 to ~€30/day, while improving event match rate by 0.5%. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why does Flink's interval join perform poorly at high throughput with unique keys? Flink's interval join opens a RocksDB iterator and performs a seek before reading records on every element processed, regardless of partition size. When bid IDs are unique and each partition holds exactly one entry, this still triggers a seek per element, consuming 15-30% of CPU. It also maintains an expiration timer queue backed by RocksDB that can grow to around 70 million entries, causing further disk scans when the in-memory cache drains. _Anyone tuning Flink joins at scale can track hard-won operator internals like this through daily.dev._ ### How can you replace Flink's interval join with a custom low-level join to reduce CPU usage? Implement a KeyedCoProcessFunction with two ValueStates, one for each event stream, partitioned by a unique key such as bid ID. Since the key is unique, each lookup becomes a direct RocksDB point lookup with no iterator or seek. Combine this with RocksDB TTL compaction for state expiry instead of explicit timers, removing the need for a large timer queue and cutting CPU cores used for the join operator from 30-60 down to 10. _Engineers debugging Flink state and CPU overhead can follow deep operator-level writeups like this on daily.dev._ ### What RocksDB settings help control state growth in a Flink job using TTL-based cleanup instead of timers? Increase write buffer count and size (e.g. from 2 buffers at 64MB to 16 at 512MB) and raise compaction file size targets (level target-file-size-base to 256MB, max-size-level-base to 1GB) so compaction runs less often but processes more data, giving the TTL filter more chances to remove expired keys. Also enable async compaction after rescale, range-delete on rescale, LZ4 compression, and FLASH_SSD_OPTIMIZED predefined options to shrink on-disk footprint. _Teams fighting growing RocksDB state in Flink pipelines can find configuration deep-dives like this via daily.dev._ ## Similar posts on daily.dev - [Why We Ditched Flink Table API Joins: Cutting State by 75% with DataStream Unions](https://daily.dev/posts/why-we-ditched-flink-table-api-joins-cutting-state-by-75-with-datastream-unions-ynzazjzo0) · Zalando · 0 upvotes · 0 comments - [From 40 seconds to under 10: rebuilding incident detection on OpenTelemetry, Apache Kafka, and Apache Flink on Kubernetes](https://daily.dev/posts/from-40-seconds-to-under-10-rebuilding-incident-detection-on-opentelemetry-apache-kafka-and-apach-owhrwpfy6) · CNCF · 9 upvotes · 0 comments - [How Medidata Cut Compute Costs by 80% Using Snowflake and Apache Iceberg](https://daily.dev/posts/how-medidata-cut-compute-costs-by-80-using-snowflake-and-apache-iceberg-led5ywgov) · Snowflake Community · 1 upvotes · 0 comments --- Tags: [#java](https://daily.dev/tags/java), [#apache-flink](https://daily.dev/tags/apache-flink) [View this post on daily.dev](https://daily.dev/posts/from-homegrown-to-flink-migrating-a-stateful-ad-event-join-at-scale-6lmjzq9eh) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"From Homegrown to Flink: Migrating a Stateful Ad Event Join at Scale","url":"https://daily.dev/posts/from-homegrown-to-flink-migrating-a-stateful-ad-event-join-at-scale-6lmjzq9eh","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/from-homegrown-to-flink-migrating-a-stateful-ad-event-join-at-scale-6lmjzq9eh"},"datePublished":"2026-07-24T07:34:07.147Z","dateModified":"2026-09-13T21:43:24.498Z","description":"Zalando's Ad Platform team replaced a 7-year-old homegrown in-memory stream join with Apache Flink to match ad auction events with user interactions in...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7e3a6edc29f339e0c1fbce0f02547a1a?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7e3a6edc29f339e0c1fbce0f02547a1a?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Zalando","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Zalando","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/94640378400546638ce8f6311afa73e3","url":"https://daily.dev/sources/zalando"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/from-homegrown-to-flink-migrating-a-stateful-ad-event-join-at-scale-6lmjzq9eh","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":7},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"java,apache-flink","timeRequired":"PT13M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Zalando","item":"https://daily.dev/sources/zalando"},{"@type":"ListItem","position":3,"name":"From Homegrown to Flink: Migrating a Stateful Ad Event Join at Scale"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/from-homegrown-to-flink-migrating-a-stateful-ad-event-join-at-scale-6lmjzq9eh#faq","mainEntity":[{"@type":"Question","name":"Why does Flink's interval join perform poorly at high throughput with unique keys?","acceptedAnswer":{"@type":"Answer","text":"Flink's interval join opens a RocksDB iterator and performs a seek before reading records on every element processed, regardless of partition size. When bid IDs are unique and each partition holds exactly one entry, this still triggers a seek per element, consuming 15-30% of CPU. It also maintains an expiration timer queue backed by RocksDB that can grow to around 70 million entries, causing further disk scans when the in-memory cache drains. Anyone tuning Flink joins at scale can track hard-won operator internals like this through daily.dev."}},{"@type":"Question","name":"How can you replace Flink's interval join with a custom low-level join to reduce CPU usage?","acceptedAnswer":{"@type":"Answer","text":"Implement a KeyedCoProcessFunction with two ValueStates, one for each event stream, partitioned by a unique key such as bid ID. Since the key is unique, each lookup becomes a direct RocksDB point lookup with no iterator or seek. Combine this with RocksDB TTL compaction for state expiry instead of explicit timers, removing the need for a large timer queue and cutting CPU cores used for the join operator from 30-60 down to 10. Engineers debugging Flink state and CPU overhead can follow deep operator-level writeups like this on daily.dev."}},{"@type":"Question","name":"What RocksDB settings help control state growth in a Flink job using TTL-based cleanup instead of timers?","acceptedAnswer":{"@type":"Answer","text":"Increase write buffer count and size (e.g. from 2 buffers at 64MB to 16 at 512MB) and raise compaction file size targets (level target-file-size-base to 256MB, max-size-level-base to 1GB) so compaction runs less often but processes more data, giving the TTL filter more chances to remove expired keys. Also enable async compaction after rescale, range-delete on rescale, LZ4 compression, and FLASH_SSD_OPTIMIZED predefined options to shrink on-disk footprint. Teams fighting growing RocksDB state in Flink pipelines can find configuration deep-dives like this via daily.dev."}}]} ```