Deep dive into Kafka 4.3 Share Groups focusing on how producer batch sizes and the share.acquire.mode configuration affect consumer parallelism. The post explains the difference between batch_optimized (soft cap on max.poll.records, batch-boundary acquisition, concurrent fetches) and record_limit (strict cap, per-record acquisition, single fetch at a time). Empirical benchmarks using Dimster show that doubling batch.size from 16KB to 32KB (64 records/batch) drops consumption from 60K to 37K msg/s with batch_optimized. Three fixes are demonstrated: reducing batch size, increasing group.share.partition.max.record.locks, or switching to record_limit mode. Trade-offs include increased network bandwidth with record_limit due to batches being delivered multiple times when fetch ranges don't align with batch boundaries.

14m read timeFrom jack-vanlightly.com
Post cover image
Table of contents
share.acquire.mode
2 Impressions