--- title: "Kafka and AI walk into a bar..." url: https://daily.dev/posts/kafka-and-ai-walk-into-a-bar--yswehdxe0 source_url: https://softwaremill.com/managing-kafka-with-ai-and-mcp type: article source: "SoftwareMill" published: 2026-08-14T09:09:56.668Z updated: 2026-08-14T09:10:24.309Z tags: ["mcp", "kafka", "claude-code", "confluent-cloud"] reading_time: 7 upvotes: 0 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Kafka and AI walk into a bar... **[SoftwareMill](https://daily.dev/sources/softwaremill)** · 7 min read · 0 upvotes · 0 comments ## Summary A Kafka administrator hands-on tests mcp-confluent, an MCP server from Confluent that lets AI assistants like Claude Code manage Kafka clusters via natural language instead of memorizing inconsistent CLI flags. The setup covers configuring connections for both Confluent Cloud and generic Apache Kafka (including mTLS and SASL/SCRAM auth), and walks through listing clusters and topics, producing/consuming messages, and inspecting consumer groups. Along the way the author hits a genuine unreported bug where Protobuf schemas registered via the standard Schema Registry REST API fail when produced through the use-latest path, unless originally registered by the SDK's own serializer. Local Apache Kafka deployments get a limited toolset compared to Confluent Cloud, and REST-dependent features like detailed topic configs don't work at all locally. The author's verdict: interesting for learning and documentation search, but too slow (tens of seconds per command) to replace CLI tools for daily admin work. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why does producing to a Protobuf topic fail with mcp-confluent even though the schema is registered correctly? Producing fails when a Protobuf subject was registered as plain .proto text through the standard Schema Registry REST API, which is the spec-correct way to register schemas. The use-latest produce path in mcp-confluent v1.5.0 only works if the schema was originally registered as a base64-encoded FileDescriptorProto by the SDK's own serializer, making this a genuine unreported bug for standard registration workflows. _daily.dev surfaces real-world bug reports like this Protobuf Schema Registry mismatch before they cost you a debugging session._ ### Can mcp-confluent manage a local Apache Kafka cluster or only Confluent Cloud? It supports both, but the toolset is limited for local deployments compared to Confluent Cloud. Local Apache Kafka installations only get access to Kafka and Schema Registry tools, missing Flink SQL, Connectors, Tableflow, Metrics, and Billing tools that are available for Confluent Cloud, plus REST-dependent features like detailed topic configuration don't work locally at all. _engineers comparing Kafka tooling options for self-managed versus cloud deployments track these gaps on daily.dev._ ### How do you configure mcp-confluent to connect to a generic Apache Kafka cluster with mTLS or SASL/SCRAM authentication? Configuration requires a YAML file defining connections with a kafka block specifying bootstrap_servers and either an SSL block with ca.location, certificate.location, and key.location for mTLS, or an auth block with api_key type using username and password for SCRAM-SHA-512. Setting read_only true on a connection is recommended for production clusters to prevent accidental writes. _teams wiring AI assistants into production Kafka clusters reference configs like this on daily.dev before going live._ ## Similar posts on daily.dev - [Confluent Makes it Easier to Build and Secure Real-Time AI at Scale](https://daily.dev/posts/confluent-makes-it-easier-to-build-and-secure-real-time-ai-at-scale-cwztvhsas) · SD Times · 0 upvotes · 0 comments - [Introducing Confluent Private Cloud](https://daily.dev/posts/introducing-confluent-private-cloud-p5czbbpel) · Confluent Blog · 0 upvotes · 0 comments - [Migrate From Hosted Kafka to Confluent Cloud in the KRaft Era](https://daily.dev/posts/migrate-from-hosted-kafka-to-confluent-cloud-in-the-kraft-era-fovqvobt4) · Confluent Blog · 0 upvotes · 0 comments - [Confluent \+ Google Cloud: Real-Time Backbone for Agentic AI](https://daily.dev/posts/confluent-google-cloud-real-time-backbone-for-agentic-ai-no3bslqii) · Confluent Blog · 1 upvotes · 0 comments --- Tags: [#mcp](https://daily.dev/tags/mcp), [#kafka](https://daily.dev/tags/kafka), [#claude-code](https://daily.dev/tags/claude-code), [#confluent-cloud](https://daily.dev/tags/confluent-cloud) [View this post on daily.dev](https://daily.dev/posts/kafka-and-ai-walk-into-a-bar--yswehdxe0)