Explores strategies for multi-tenant workload distribution in event-driven systems, using the Emmett framework as a reference. Covers two main approaches: horizontal sharding (spinning up separate containers per tenant) and vertical partitioning (running multiple tenants within one process using worker threads). Discusses trade-offs between the two, including cost, isolation, and the noisy neighbor problem. Explains how consistent hashing can route messages to the right consumer, drawing parallels with Kafka's consumer group protocol. Concludes that implementing full distributed consensus (Raft/Paxos) in Emmett is not worthwhile given mature solutions like Kafka, but partitioned projectors using worker threads per tenant are a planned feature.

8m read timeFrom event-driven.io
Post cover image
Table of contents
Distributing the workload dynamically between processorsWhat options do we have for such a multi-tenant setup?