--- title: Kafka's log compaction corrupts data. Here's how we fixed it description: Apache Kafka's log compaction has a critical race condition between compaction and replication that can corrupt data in four distinct ways: deleted data... canonical: https://daily.dev/posts/kafka-s-log-compaction-corrupts-data-here-s-how-we-fixed-it-anuxl2uxk twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Kafka's log compaction corrupts data. Here's how we fixed it | daily.dev og:description: Apache Kafka's log compaction has a critical race condition between compaction and replication that can corrupt data in four distinct ways: deleted data... og:url: https://daily.dev/posts/kafka-s-log-compaction-corrupts-data-here-s-how-we-fixed-it-anuxl2uxk og:image: https://api.daily.dev/og/posts/anuxL2UXK.png og:image:alt: Kafka's log compaction corrupts data. Here's how we fixed it og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Kafka's log compaction corrupts data. Here's how we fixed it **[Redpanda](https://daily.dev/sources/redpanda)** · 12 min read · 3 upvotes · 0 comments ## Summary Apache Kafka's log compaction has a critical race condition between compaction and replication that can corrupt data in four distinct ways: deleted data reappearing, aborted transactions being served as committed, committed data being hidden, and read_committed consumers getting frozen partitions. The root cause is that each broker compacts independently, so tombstones and transaction control markers can be removed on some replicas before an offline replica has processed them. Redpanda Streaming solves this with a coordinated compaction protocol using two pairs of offsets (MCCO/MTRO for tombstones, MXFO/MXRO for transaction markers) that ensure no tombstone or marker is removed until every replica has compacted past the associated data records. The bug is reproducible on Kafka 3.9 through 4.2 with a provided Docker Compose reproducer. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [How Redpanda Cloud Topics rethinks Kafka compaction](https://daily.dev/posts/how-redpanda-cloud-topics-rethinks-kafka-compaction-kp4ux5h3r) · Redpanda · 3 upvotes · 0 comments - [Compaction & Retention: Edge Cases That Make Your Kafka Lag Metrics Inaccurate](https://daily.dev/posts/compaction-retention-edge-cases-that-make-your-kafka-lag-metrics-inaccurate-05dpvuqhb) · SoftwareMill · 0 upvotes · 0 comments --- Tags: [#data-engineering](https://daily.dev/tags/data-engineering), [#distributed-systems](https://daily.dev/tags/distributed-systems), [#apache-kafka](https://daily.dev/tags/apache-kafka), [#redpanda](https://daily.dev/tags/redpanda) [View this post on daily.dev](https://daily.dev/posts/kafka-s-log-compaction-corrupts-data-here-s-how-we-fixed-it-anuxl2uxk) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Kafka's log compaction corrupts data. Here's how we fixed it","url":"https://daily.dev/posts/kafka-s-log-compaction-corrupts-data-here-s-how-we-fixed-it-anuxl2uxk","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/kafka-s-log-compaction-corrupts-data-here-s-how-we-fixed-it-anuxl2uxk"},"datePublished":"2026-06-25T16:02:01.581Z","dateModified":"2026-08-24T07:08:45.659Z","description":"Apache Kafka's log compaction has a critical race condition between compaction and replication that can corrupt data in four distinct ways: deleted data...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/a7fb3752e44f1f4dad2651380f5cbd0e?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/a7fb3752e44f1f4dad2651380f5cbd0e?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Redpanda","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Redpanda","logo":"https://media.daily.dev/image/upload/s--1R2EIDcb--/f_auto,q_auto/v1769934743/logos/redpanda","url":"https://daily.dev/sources/redpanda"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/kafka-s-log-compaction-corrupts-data-here-s-how-we-fixed-it-anuxl2uxk","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":3},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-engineering,distributed-systems,apache-kafka,redpanda","timeRequired":"PT12M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Redpanda","item":"https://daily.dev/sources/redpanda"},{"@type":"ListItem","position":3,"name":"Kafka's log compaction corrupts data. Here's how we fixed it"}]} ```