monday Engineering
Read post

Solving HTAP for a Trillion-Table System

monday.com replaced its MySQL, Cassandra, and Redis fleet with a purpose-built columnar serving layer powered by DuckDB, achieving 5x faster board loads, 20x for large boards, 50x for aggregations, and 40-60% infrastructure cost reduction. The architecture (mondayDB 3) uses CQRS with a batch layer (DuckDB files in S3), an external distributed WAL, and a soft-stateful serving layer of Go processes on NVMe-backed Kubernetes nodes. Key design decisions include per-tenant file isolation (one DuckDB file per board), a sync-then-query read pattern, and a custom routing layer called Ranja using Weighted Rendezvous Hashing for cache affinity. The migration of 1M+ organizations from MySQL took 18 months with zero downtime, using dual-read validation and feature-flag-controlled incremental rollout. The same architecture is now being extended as an AI contextual layer for text search, semantic retrieval, and RAG.

    #duckdb
May 19•22m read time•From engineering.monday.com
Post cover image
Table of contents
TL;DRHow We Got HereWhat Matters First – ResultsmondayDB 3 – CQRS FTWThe WAL: Our External Write-Ahead LogRead Path: Sync-Then-QuerySmart Routing for Cache AffinityEntity Plugin SystemZero Downtime MigrationLessons LearnedWhy We Chose DuckDBWhat’s NextThank You.
42.1K Impressions3 Comments
monday Engineering's image
monday Engineering

Monday is a blog or publication focused on productivity, time management, and personal development. ...

37 Followers

•

416 Upvotes

Would you recommend this post?

Copy link
WhatsApp
Facebook
X
New Squad
  • © 2026 Daily Dev Ltd.
  • Guidelines
  • Explore
  • Tags
  • Sources
  • Squads
  • Leaderboard