<p><strong>TLDR:</strong> DoorDash published a detailed post-mortem on replacing a Redis-plus-relational hybrid feature store with a stateless Kvrocks-on-RocksDB system that now handles 130M HMGETs per second. Sigma Computing closed an $80M Series E at a $3B valuation, doubling in a year on the back of $200M ARR. AWS Glue zero-ETL landed in Mumbai, and ClickHouse had a busy day with multiple case studies and a new SQL charting layer in ClickStack.</p>
<hr>
<h2>DoorDash’s clusterless ML feature store</h2>
<p>DoorDash replaced a Redis-plus-relational-database hybrid that had hit a scaling wall with a stateless system built on Apache Kvrocks (Redis protocol over RocksDB) running on commodity SSD instance-store disks. The key architectural insight: data is batch-loaded from S3 as RocksDB backups, so horizontal scaling is just adding nodes with no cluster state to synchronize. They also discovered that large Redis clusters degrade client performance past 2,000 nodes, which they solved by having their custom Redis Cluster Manager return a per-client subset of nodes rather than the full topology. The system now serves 1.6B features at 130M HMGETs per second within a 50ms P999 latency target.</p>
<h2>Sigma Computing raises $80M at $3B valuation</h2>
<p>Sigma closed an $80M Series E led by Princeville Capital with strategic participation from Databricks Ventures, ServiceNow Ventures, and Workday Ventures. The company hit $200M ARR in April, up from $100M a year prior, with over 2,000 customers. The product sits on top of Snowflake, Databricks, and BigQuery with a spreadsheet interface that doesn’t require SQL. Their newest feature, Sigma Agents, runs no-code AI agents inside the warehouse’s existing security model and became the fastest-adopted feature in company history. The strategic investor lineup is worth noting — Databricks and Workday both have skin in the game now.</p>
<h2>AWS Glue zero-ETL expands to Mumbai</h2>
<p>AWS Glue zero-ETL integrations are now available in the Asia Pacific (Mumbai) region. The service handles schema mapping, change data capture, and incremental replication in near real-time from sources including DynamoDB, Oracle, SQL Server, MySQL, PostgreSQL, and SaaS apps like Salesforce and SAP. No ETL pipelines to build or maintain.</p>
<h2>ClickHouse: case studies and new ClickStack SQL alerting</h2>
<p>ClickHouse had an unusually active day. D. E. Shaw published a case study showing ClickHouse ingesting 3.5M samples per second during evaluation versus a competitor’s 480K, with the production cluster now handling 530K records per second on average across 68TB of compressed metrics. ChatFeatured migrated from PlanetScale Postgres to ClickHouse-managed Postgres and dropped a 30-day analytics query from 2.5 minutes to under a second. On the product side, ClickStack added SQL-based charting and alerting, letting you write arbitrary ClickHouse SQL for dashboards and encode full alerting logic in a query that returns a binary 1/0 to trigger alerts.</p>
<hr>
<h2>Also notable</h2>
<ul>
<li><strong>pg_sorted_heap 0.14.0:</strong> New release adds PostgreSQL 16/17/18 support, partition-aware vector search, composite zone-map metadata, and GraphRAG lifecycle coverage including crash recovery and concurrent maintenance.</li>
<li><strong>Postgres FDW pushdown:</strong> A detailed walkthrough of how pg_clickhouse’s FDW decides what to push down to ClickHouse, with the key point that pushdown is all-or-nothing at the upper-relation level — one unsupported sub-expression can force millions of rows across the wire instead of a small aggregate.</li>
<li><strong>Redis Iris preview:</strong> Redis launched a context and memory engine for AI agents with five components including Context Retriever (semantic model over business data that auto-generates MCP tools) and Agent Memory for cross-session state. Both are in preview.</li>
<li><strong>LangChain SmithDB:</strong> LangChain announced SmithDB, a purpose-built data layer for agent observability and evaluation workloads, targeting the increasingly complex query patterns that agent tracing generates.</li>
<li><strong>Monzo data mesh:</strong> Neobank Monzo published details on a governed data mesh spanning 100 teams and 12,000 dbt models.</li>
<li><strong>TigerFS:</strong> Open source project from the TimescaleDB team that mounts a PostgreSQL database as a local filesystem using NFS (macOS) or FUSE (Linux), with bidirectional sync and full ACID guarantees — pitched as useful for multi-agent workflows.</li>
<li><strong>Swiggy search ranking:</strong> Swiggy’s team published their real-time ML ranking system for autocomplete, running LTR models (XGBoost, RankLib) with inference inside OpenSearch to avoid extra network hops.</li>
<li><strong>ClickHouse Join table engine:</strong> ClickHouse Cloud’s Join table engine now uses SharedJoin backed by ReplacingMergeTree for ANY joins, giving automatic upserts and deduplication without the limitations of the open-source implementation.</li>
<li><strong>Grab multi-agent data investigation:</strong> Grab’s analytics warehouse team built a five-agent supervisor system on LangGraph that handles the majority of standard data investigation inquiries autonomously, cutting resolution time by an order of magnitude and reclaiming two engineering days per week.</li>
<li><strong>Databricks analytics engineer pathway:</strong> Databricks Academy launched a six-course learning pathway for SQL practitioners covering dbt-style ETL with Materialized Views, Streaming Tables, Unity Catalog Metric Views, and declarative Spark pipelines.</li>
<li><strong>SQL Server Pacemaker HA Agent v2:</strong> Tech preview for SQL Server 2025 CU3+ on RHEL 9+ with faster failovers, configurable failure condition levels, improved write-lease integration for split-brain prevention, and TLS 1.3 support.</li>
<li><strong>obsidian-duckdb-motherduck plugin:</strong> Obsidian plugin that runs DuckDB WASM queries inline in notes, supports Parquet/CSV/JSON/Iceberg/Delta, and can freeze query output as plain markdown for git-diffable, publishable notes.</li>
</ul>


Query optimization, schema design, and data store coverage across PostgreSQL, MongoDB, Redis, and more.

Databases Digest

A data engineering roundup covering several major stories: DoorDash replaced a Redis-plus-relational ML feature store with a stateless Apache Kvrocks/RocksDB system that now handles 130M HMGETs/sec at 1.6B features with P999 latency under 50ms. Sigma Computing raised $80M at a $3B valuation with $200M ARR and a new no-code AI agent feature. AWS Glue zero-ETL expanded to the Mumbai region. ClickHouse published multiple case studies (D.E. Shaw ingesting 3.5M samples/sec) and added SQL-based charting and alerting to ClickStack. Additional items include Redis Iris AI memory engine preview, LangChain SmithDB for agent observability, Monzo's 100-team data mesh with 12,000 dbt models, and a Grab multi-agent data investigation system built on LangGraph.

DoorDash's clusterless feature store hits 130M ops/sec, Sigma Computing doubles valuation to $3B