---
title: "How the D. E. Shaw group powers high-cardinality observability at scale with ClickHouse"
url: https://daily.dev/posts/how-the-d-e-shaw-group-powers-high-cardinality-observability-at-scale-with-clickhouse-51pw0hsby
source_url: https://clickhouse.com/blog/deshaw
type: article
source: "ClickHouse"
published: 2026-05-18T12:53:28.470Z
updated: 2026-05-18T12:53:57.384Z
tags: ["data-science", "observability", "opentelemetry", "clickhouse"]
reading_time: 8
upvotes: 0
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# How the D. E. Shaw group powers high-cardinality observability at scale with ClickHouse

**[ClickHouse](https://daily.dev/sources/clickhouse)** · 8 min read · 0 upvotes · 0 comments

## Summary

The D. E. Shaw group, a global investment and technology firm, replaced its previous observability platform with ClickHouse to handle high-cardinality metrics across millions of compute workloads on its internal HPC grid. During evaluation, ClickHouse ingested 3.5 million samples per second versus a competitor's 480k, and completed queries in seconds where competitors timed out. The production cluster now handles 530k records per second on average with spikes over 1 million, storing 68 TB of compressed metrics data. The platform enables capacity planning, compute efficiency analysis, and is expanding into distributed tracing via OpenTelemetry with 12.5x compression ratios. Key engineering decisions included preserving InfluxDB line protocol ingestion, building custom backfill tooling, and using materialized views for long-horizon aggregations.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://clickhouse.com/blog/deshaw>

## Similar posts on daily.dev

- [How We Achieved Almost Limitless Observability](https://daily.dev/posts/how-we-achieved-almost-limitless-observability-a4fwitlto) · monday Engineering · 8 upvotes · 0 comments
- [Shopify powers observability for global-scale commerce with ClickHouse](https://daily.dev/posts/shopify-powers-observability-for-global-scale-commerce-with-clickhouse-r45dmqcus) · ClickHouse · 3 upvotes · 0 comments
- [ClickHouse achieves AWS Cloud Operations Competency](https://daily.dev/posts/clickhouse-achieves-aws-cloud-operations-competency-8uvx6eyrc) · ClickHouse · 1 upvotes · 0 comments

---

Tags: [#data-science](https://daily.dev/tags/data-science), [#observability](https://daily.dev/tags/observability), [#opentelemetry](https://daily.dev/tags/opentelemetry), [#clickhouse](https://daily.dev/tags/clickhouse)

[View this post on daily.dev](https://daily.dev/posts/how-the-d-e-shaw-group-powers-high-cardinality-observability-at-scale-with-clickhouse-51pw0hsby)