--- title: Kafka Share Groups and Parallelizing Consumption — Part... description: Kafka 4.2 Share Groups introduce a new parallelism model where consumers can scale beyond partition count. However, the default max.poll.records=500 causes a... canonical: https://daily.dev/posts/kafka-share-groups-and-parallelizing-consumption-part-1-tuning-max-poll-records-jack-vanlightly-yvnjyazal twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Kafka Share Groups and Parallelizing Consumption — Part 1: Tuning max.poll.records — Jack Vanlightly | daily.dev og:description: Kafka 4.2 Share Groups introduce a new parallelism model where consumers can scale beyond partition count. However, the default max.poll.records=500 causes a... og:url: https://daily.dev/posts/kafka-share-groups-and-parallelizing-consumption-part-1-tuning-max-poll-records-jack-vanlightly-yvnjyazal og:image: https://api.daily.dev/og/posts/yvNjyazAL.png og:image:alt: Kafka Share Groups and Parallelizing Consumption — Part 1: Tuning max.poll.records — Jack Vanlightly og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Kafka Share Groups and Parallelizing Consumption — Part 1: Tuning max.poll.records — Jack Vanlightly **[Jack Vanlightly](https://daily.dev/sources/jack-vanlightly)** · 13 min read · 0 upvotes · 0 comments ## Summary Kafka 4.2 Share Groups introduce a new parallelism model where consumers can scale beyond partition count. However, the default max.poll.records=500 causes a 'greedy-capture regime' where a few consumers monopolize the inflight record budget (group.share.partition.max.record.locks, default 2000), leaving others idle. With 300 consumers and 5ms processing time, default config yields only 4800 msg/s instead of the theoretical 60K msg/s. The fix is to set max.poll.records = group.share.partition.max.record.locks / consumers_per_partition, then reduce further to account for timing variance. With max.poll.records=30 and 312 consumers across 6 partitions, the benchmark achieves the target 60K msg/s. The post also warns about an 'accidental fair-sharing regime' that can mask the misconfiguration under low load, only to collapse under higher throughput. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Understanding and Avoiding CommitFailedException in Kafka](https://daily.dev/posts/understanding-and-avoiding-commitfailedexception-in-kafka-5g3zp4iwo) · Baeldung · 2 upvotes · 0 comments - [Kafka Queue Semantics Now GA with Share Consumer API](https://daily.dev/posts/kafka-queue-semantics-now-ga-with-share-consumer-api-aqzvx7ajg) · Confluent Blog · 1 upvotes · 0 comments - [Apache Kafka performance \#1](https://daily.dev/posts/apache-kafka-performance-1-dc6zd9py1) · Jack Vanlightly · 5 upvotes · 1 comments - [How Apache Kafka flexed to support queues](https://daily.dev/posts/how-apache-kafka-flexed-to-support-queues-ytdb29stz) · InfoWorld · 1 upvotes · 0 comments - [Apache Kafka 4.2.0 Released: Share Groups, Streams & More](https://daily.dev/posts/apache-kafka-4-2-0-released-share-groups-streams-more-0dctaidm9) · Confluent Blog · 17 upvotes · 0 comments --- Tags: [#distributed-systems](https://daily.dev/tags/distributed-systems), [#apache-kafka](https://daily.dev/tags/apache-kafka) [View this post on daily.dev](https://daily.dev/posts/kafka-share-groups-and-parallelizing-consumption-part-1-tuning-max-poll-records-jack-vanlightly-yvnjyazal) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Kafka Share Groups and Parallelizing Consumption — Part 1: Tuning max.poll.records — Jack Vanlightly","url":"https://daily.dev/posts/kafka-share-groups-and-parallelizing-consumption-part-1-tuning-max-poll-records-jack-vanlightly-yvnjyazal","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/kafka-share-groups-and-parallelizing-consumption-part-1-tuning-max-poll-records-jack-vanlightly-yvnjyazal"},"datePublished":"2026-07-09T07:01:20.833Z","dateModified":"2026-07-09T07:04:22.949Z","description":"Kafka 4.2 Share Groups introduce a new parallelism model where consumers can scale beyond partition count. However, the default max.poll.records=500 causes a...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f1273b01c4fd75587d06ca009ad287f3?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f1273b01c4fd75587d06ca009ad287f3?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Jack Vanlightly","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Jack Vanlightly","logo":"https://media.daily.dev/image/upload/s--lIrxyl_V--/f_auto,q_auto/v1781695531/logos/jack-vanlightly?_a=BAMAMiWQ0","url":"https://daily.dev/sources/jack-vanlightly"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/kafka-share-groups-and-parallelizing-consumption-part-1-tuning-max-poll-records-jack-vanlightly-yvnjyazal","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"distributed-systems,apache-kafka","timeRequired":"PT13M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Jack Vanlightly","item":"https://daily.dev/sources/jack-vanlightly"},{"@type":"ListItem","position":3,"name":"Kafka Share Groups and Parallelizing Consumption — Part 1: Tuning max.poll.records — Jack Vanlightly"}]} ```