--- title: Best Data Streaming posts of 2025 | daily.dev description: The most upvoted Data Streaming posts of 2025, curated by the daily.dev community. canonical: https://daily.dev/tags/data-streaming/best-of/2025 twitter:card: summary_large_image twitter:site: @dailydotdev og:url: https://daily.dev/tags/data-streaming/best-of/2025 og:type: website og:site_name: daily.dev og:title: Best Data Streaming posts of 2025 | daily.dev og:description: The most upvoted Data Streaming posts of 2025, curated by the daily.dev community. og:image: https://media.daily.dev/image/upload/s--VAY5ToZt--/f_auto/v1724209435/public/daily.dev%20-%20open%20graph --- # Best of Data Streaming — 2025 1. 1 [](/posts/the-future-of-ai-agents-is-event-driven-3moznkokd "The Future of AI Agents is Event-Driven") Article Confluent Blog · 1y The Future of AI Agents is Event-Driven AI agents are poised to transform enterprise operations by adopting event-driven architecture. This architectural approach addresses interoperability challenges and enhances scalability. EDA allows agents to operate independently, integrate seamlessly, and adapt workflows dynamically, overcoming the limitations of fixed workflows and tightly coupled systems. It ensures agents can effectively handle complex, interconnected tasks, thereby unlocking their full potential. The article highlights the importance of EDA in creating resilient, scalable AI systems and warns against the risks of outdated architecture in the evolving AI landscape. 89 1 2. 2 [](/posts/why-microservices-need-event-driven-architectures-for-agility-and-scale-wmn10o1o2 "Why Microservices Need Event-Driven Architectures for Agility and Scale") Article Confluent Blog · 1y Why Microservices Need Event-Driven Architectures for Agility and Scale Event-driven architectures solve critical problems in microservices by replacing synchronous API calls with asynchronous event communication. This approach eliminates bottlenecks, prevents cascading failures, and enables real-time responsiveness. Apache Kafka serves as the central platform for event streaming, allowing services to publish and subscribe to events independently. The shift from REST-based to event-driven microservices delivers faster innovation cycles, improved system resilience, and better customer experiences across industries like finance, e-commerce, and telecommunications. 65 3. 3 [](/posts/apache-kafka-4-1-the-3-big-things-developers-need-to-know-eq1jddh8d "Apache Kafka 4.1: The 3 Big Things Developers Need To Know") Article The New Stack · 1y Apache Kafka 4.1: The 3 Big Things Developers Need To Know Apache Kafka 4.1 introduces three major developer-focused features: Queues for Kafka (KIP-932) enabling cooperative message consumption with per-message acknowledgment, native JWT-Bearer authentication support eliminating static credentials, and a new Kafka Streams rebalance protocol for better coordination. The release also includes improvements to consumer group protocols, transaction handling, and unified metrics. 32 1 4. 4 [](/posts/the-new-look-and-feel-of-apache-kafka-4-0-k6etglkb6 "The New Look and Feel of Apache Kafka 4.0") Article The New Stack · 2y The New Look and Feel of Apache Kafka 4.0 Apache Kafka 4.0 introduces significant upgrades, including the replacement of ZooKeeper with KRaft for metadata management, enhancing stability and reducing complexity. The release features Queues for Kafka to allow scaling consumers beyond topic partitions, improved consumer group rebalancing, and new capabilities for code injection and observability. These updates aim to streamline Kafka's operations and improve the developer experience. 32 5. 5 [](/posts/building-real-time-multi-agent-ai-with-confluent-7z7k7t9vl "Building Real-Time Multi-Agent AI With Confluent") Article Confluent Blog · 1y Building Real-Time Multi-Agent AI With Confluent Agent Taskflow is utilizing the Confluent data streaming platform to build real-time, multi-agent AI orchestration systems. These systems enable coordinated efforts between AI agents to execute tasks autonomously and in real-time. Key features include a drag-and-drop builder, observability, and fault tolerance. The platform addresses technical challenges in communication, scalability, and governance, ensuring efficient and secure deployment of multi-agent systems. 30 1 6. 6 [](/posts/can-a-database-be-faster-than-a-formula-1-engine--to1ovvu4r "Can a Database Be Faster Than a Formula 1 Engine?") Article SingleStore · 2y Can a Database Be Faster Than a Formula 1 Engine? Formula 1 cars generate vast amounts of real-time telemetry data, which is crucial for strategic decision-making. Each car is fitted with 300 sensors producing 1.1 million data points per second. Teams use this data for simulations, performance analysis, and strategy adjustments. SingleStore's real-time analytics capabilities are highlighted, showcasing its ability to handle high-throughput data streams and provide millisecond response times. The post includes a practical guide for setting up a data ingestion and visualization simulation using SingleStore, Confluent Kafka, and Grafana. 25 1 7. 7 [](/posts/bufstream-stream-kafka-messages-to-iceberg-tables-in-minutes-rqwhb9pzn "Bufstream: Stream Kafka Messages to Iceberg Tables in Minutes") Article Data Engineer Things · 2y Bufstream: Stream Kafka Messages to Iceberg Tables in Minutes Bufstream offers a cost-effective and cloud-native alternative to Kafka by using object storage, significantly reducing infrastructure costs. It enhances data quality management by integrating schema validation directly into the broker and seamlessly transforms Kafka messages into Iceberg tables, simplifying the data pipeline. Bufstream also addresses challenges with Kafka's cloud inefficiencies and provides built-in support for schema enforcement and granular access control. 25 8. 8 [](/posts/ag2ai-faststream-faststream-is-a-powerful-and-easy-to-use-python-framework-for-building-asynchronou-rngwbfcrd "ag2ai/faststream: FastStream is a powerful and easy-to-use Python framework for building asynchronous services interacting with event streams such as Apache Kafka, RabbitMQ, NATS and Redis.") Article Community Picks · 2y ag2ai/faststream: FastStream is a powerful and easy-to-use Python framework for building asynchronous services interacting with event streams such as Apache Kafka, RabbitMQ, NATS and Redis. FastStream is a Python framework designed for building asynchronous services that interact with event streams such as Apache Kafka, RabbitMQ, NATS, and Redis. It simplifies the creation of producers and consumers for message queues, leveraging Pydantic for data validation and providing automatic documentation generation. Key features include support for multiple message brokers, intuitive development with full-typed editor support, efficient dependency management, and in-memory testing. FastStream integrates easily with HTTP frameworks like FastAPI, making it suitable for developers at various experience levels. 24 1 9. 9 [](/posts/a2a-mcp-kafka-and-flink-the-new-stack-for-ai-agents-ksvzziuqy "A2A, MCP, Kafka and Flink: The New Stack for AI Agents") Article The New Stack · 1y A2A, MCP, Kafka and Flink: The New Stack for AI Agents The post discusses the need for a new infrastructure stack to enable AI agents to collaborate effectively. This stack includes four open components: Google’s Agent2Agent (A2A) protocol for agent communication, Anthropic’s Model Context Protocol (MCP) for tool access, Apache Kafka for event-driven communication, and Apache Flink for real-time data processing. By integrating these technologies, AI agents can operate beyond isolated silos, scaling to complex ecosystems that facilitate collaboration, observability, and resilience. 23 1 10. 10 [](/posts/change-data-capture-tools-kmbwihvjg "Change Data Capture Tools") Article Dev Genius · 2y Change Data Capture Tools Change Data Capture (CDC) tools automatically track and replicate database changes in real time. Different mechanisms like log-based, trigger-based, query-based, timestamp-based, and hybrid CDC tools are used. Debezium is a popular open-source CDC platform with high scalability, integrating well with Kafka. Other tools like DBConvert Streams, Maxwell Daemon, and Sequin offer various features and integrations for efficient data replication. Challenges such as setup complexity and performance overhead are common with these tools. 22 11. 11 [](/posts/how-canva-collects-25-billion-events-a-day-kphh43jxj "How Canva Collects 25 Billion Events a Day") Article ByteByteGo · 1y How Canva Collects 25 Billion Events a Day Canva collects 25 billion events daily by implementing a robust analytics infrastructure. Key elements include strict schema governance using Protobuf, a unified TypeScript client across platforms, and an AWS Kinesis-backed pipeline for event enrichment and distribution. The system's design focuses on decoupling ingestion from delivery and optimizing costs through compression and clever use of AWS services, maintaining reliability while reducing expenses significantly. 12 12. 12 [](/posts/build-an-ai-personalization-engine-with-confluent-databricks-5uuyhfxq6 "Build an AI Personalization Engine with Confluent & Databricks") Article Confluent Blog · 1y Build an AI Personalization Engine with Confluent & Databricks Confluent and Databricks can be combined to build real-time AI applications by bridging operational and analytical data systems. The tutorial demonstrates creating an AI-powered marketing personalization engine using Tableflow to convert Kafka topics into Delta Lake tables, Apache Flink for stream processing, and Oracle CDC connectors for real-time data ingestion. The example implementation helps a fictional hotel brand identify low-booking properties and generate targeted promotional campaigns using AI-generated content. 10 2 13. 13 [](/posts/kafka-4-0-kraft-simplifies-architecture-hckppn5lg "Kafka 4.0: KRaft Simplifies Architecture") Article InfoQ · 2y Kafka 4.0: KRaft Simplifies Architecture Apache Kafka 4.0 introduces KRaft mode, eliminating the need for ZooKeeper, thereby simplifying architecture and enhancing scalability. The release also includes a next-generation consumer group protocol for improved performance and early access to point-to-point messaging. Additionally, it updates minimum Java requirements and removes deprecated APIs. 10 [See all Data Streaming archives](/tags/data-streaming/best-of) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@graph":[{"@type":"CollectionPage","@id":"https://daily.dev/tags/data-streaming/best-of/2025#page","url":"https://daily.dev/tags/data-streaming/best-of/2025","name":"Best Data Streaming Posts of 2025","description":"The most upvoted Data Streaming posts of 2025, curated by the daily.dev community.","isPartOf":{"@type":"WebSite","url":"https://daily.dev"}},{"@type":"ItemList","@id":"https://daily.dev/tags/data-streaming/best-of/2025#items","numberOfItems":13,"itemListElement":[{"@type":"ListItem","position":1,"url":"https://daily.dev/posts/the-future-of-ai-agents-is-event-driven-3moznkokd","name":"The Future of AI Agents is Event-Driven"},{"@type":"ListItem","position":2,"url":"https://daily.dev/posts/why-microservices-need-event-driven-architectures-for-agility-and-scale-wmn10o1o2","name":"Why Microservices Need Event-Driven Architectures for Agility and Scale"},{"@type":"ListItem","position":3,"url":"https://daily.dev/posts/apache-kafka-4-1-the-3-big-things-developers-need-to-know-eq1jddh8d","name":"Apache Kafka 4.1: The 3 Big Things Developers Need To Know"},{"@type":"ListItem","position":4,"url":"https://daily.dev/posts/the-new-look-and-feel-of-apache-kafka-4-0-k6etglkb6","name":"The New Look and Feel of Apache Kafka 4.0"},{"@type":"ListItem","position":5,"url":"https://daily.dev/posts/building-real-time-multi-agent-ai-with-confluent-7z7k7t9vl","name":"Building Real-Time Multi-Agent AI With Confluent"},{"@type":"ListItem","position":6,"url":"https://daily.dev/posts/can-a-database-be-faster-than-a-formula-1-engine--to1ovvu4r","name":"Can a Database Be Faster Than a Formula 1 Engine?"},{"@type":"ListItem","position":7,"url":"https://daily.dev/posts/bufstream-stream-kafka-messages-to-iceberg-tables-in-minutes-rqwhb9pzn","name":"Bufstream: Stream Kafka Messages to Iceberg Tables in Minutes"},{"@type":"ListItem","position":8,"url":"https://daily.dev/posts/ag2ai-faststream-faststream-is-a-powerful-and-easy-to-use-python-framework-for-building-asynchronou-rngwbfcrd","name":"ag2ai/faststream: FastStream is a powerful and easy-to-use Python framework for building asynchronous services interacting with event streams such as Apache Kafka, RabbitMQ, NATS and Redis."},{"@type":"ListItem","position":9,"url":"https://daily.dev/posts/a2a-mcp-kafka-and-flink-the-new-stack-for-ai-agents-ksvzziuqy","name":"A2A, MCP, Kafka and Flink: The New Stack for AI Agents"},{"@type":"ListItem","position":10,"url":"https://daily.dev/posts/change-data-capture-tools-kmbwihvjg","name":"Change Data Capture Tools"},{"@type":"ListItem","position":11,"url":"https://daily.dev/posts/how-canva-collects-25-billion-events-a-day-kphh43jxj","name":"How Canva Collects 25 Billion Events a Day"},{"@type":"ListItem","position":12,"url":"https://daily.dev/posts/build-an-ai-personalization-engine-with-confluent-databricks-5uuyhfxq6","name":"Build an AI Personalization Engine with Confluent & Databricks"},{"@type":"ListItem","position":13,"url":"https://daily.dev/posts/kafka-4-0-kraft-simplifies-architecture-hckppn5lg","name":"Kafka 4.0: KRaft Simplifies Architecture"}]},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Tags","item":"https://daily.dev/tags"},{"@type":"ListItem","position":3,"name":"Data Streaming","item":"https://daily.dev/tags/data-streaming"},{"@type":"ListItem","position":4,"name":"Best of","item":"https://daily.dev/tags/data-streaming/best-of"},{"@type":"ListItem","position":5,"name":"2025"}]}]} ```