--- title: Inside Laravel Nightwatch’s Observability Pipeline:... description: Laravel Nightwatch processes over 1 billion observability events daily using Amazon MSK and ClickHouse Cloud. The platform combines MSK Express brokers for... canonical: https://daily.dev/posts/inside-laravel-nightwatch-s-observability-pipeline-real-time-event-processing-with-amazon-msk-and-c-ra0puxe6m twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Inside Laravel Nightwatch’s Observability Pipeline: Real-Time Event Processing with Amazon MSK and ClickHouse Cloud | daily.dev og:description: Laravel Nightwatch processes over 1 billion observability events daily using Amazon MSK and ClickHouse Cloud. The platform combines MSK Express brokers for... og:url: https://daily.dev/posts/inside-laravel-nightwatch-s-observability-pipeline-real-time-event-processing-with-amazon-msk-and-c-ra0puxe6m og:image: https://api.daily.dev/og/posts/Ra0pUXe6m.png og:image:alt: Inside Laravel Nightwatch’s Observability Pipeline: Real-Time Event Processing with Amazon MSK and ClickHouse Cloud og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Inside Laravel Nightwatch’s Observability Pipeline: Real-Time Event Processing with Amazon MSK and ClickHouse Cloud **[ClickHouse](https://daily.dev/sources/clickhouse)** · 6 min read · 13 upvotes · 1 comments ## Summary Laravel Nightwatch processes over 1 billion observability events daily using Amazon MSK and ClickHouse Cloud. The platform combines MSK Express brokers for event ingestion, ClickPipes for streaming data to ClickHouse, and AWS Lambda for validation. This architecture achieves sub-second query latency while handling millions of events per second. On launch day, the system processed 500 million events with 97ms average dashboard latency for 5,300 users. The dual-database design separates transactional workloads on RDS PostgreSQL from analytical workloads on ClickHouse Cloud, enabling horizontal scaling and cost-effective real-time monitoring at global scale. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community discussion Top comments from developers on daily.dev. **@dmitriylezhnev** · 0 upvotes > I like how elegant this solution looks on paper! I'd say even boring to a degree. Good job! --- Tags: [#aws](https://daily.dev/tags/aws), [#laravel](https://daily.dev/tags/laravel), [#observability](https://daily.dev/tags/observability), [#kafka](https://daily.dev/tags/kafka), [#clickhouse](https://daily.dev/tags/clickhouse) [View this post on daily.dev](https://daily.dev/posts/inside-laravel-nightwatch-s-observability-pipeline-real-time-event-processing-with-amazon-msk-and-c-ra0puxe6m) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Inside Laravel Nightwatch’s Observability Pipeline: Real-Time Event Processing with Amazon MSK and ClickHouse Cloud","url":"https://daily.dev/posts/inside-laravel-nightwatch-s-observability-pipeline-real-time-event-processing-with-amazon-msk-and-c-ra0puxe6m","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/inside-laravel-nightwatch-s-observability-pipeline-real-time-event-processing-with-amazon-msk-and-c-ra0puxe6m"},"datePublished":"2025-10-08T17:10:05.138Z","dateModified":"2025-10-08T17:10:29.936Z","description":"Laravel Nightwatch processes over 1 billion observability events daily using Amazon MSK and ClickHouse Cloud. The platform combines MSK Express brokers for...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/acbff18e8080a20e3f56137bfa18faf3?_a=AQAEulh","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/acbff18e8080a20e3f56137bfa18faf3?_a=AQAEulh","isAccessibleForFree":true,"articleSection":"ClickHouse","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"ClickHouse","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/clickhouse","url":"https://daily.dev/sources/clickhouse"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/inside-laravel-nightwatch-s-observability-pipeline-real-time-event-processing-with-amazon-msk-and-c-ra0puxe6m","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":13},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"aws,laravel,observability,kafka,clickhouse","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"ClickHouse","item":"https://daily.dev/sources/clickhouse"},{"@type":"ListItem","position":3,"name":"Inside Laravel Nightwatch’s Observability Pipeline: Real-Time Event Processing with Amazon MSK and ClickHouse Cloud"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/inside-laravel-nightwatch-s-observability-pipeline-real-time-event-processing-with-amazon-msk-and-c-ra0puxe6m","comment":[{"@type":"Comment","text":"I like how elegant this solution looks on paper! I’d say even boring to a degree. Good job!","datePublished":"2025-10-21T12:34:32.389Z","url":"https://daily.dev/posts/Ra0pUXe6m#c-h1FPfPbH9","author":{"@type":"Person","name":"Dmitriy Lezhnev","url":"https://daily.dev/dmitriylezhnev","image":"https://lh3.googleusercontent.com/a/ACg8ocJVN86Tdnt60xxFSGgjdPe3Un-PqVxs13Xgt_zigukW1JEuudg4=s96-c"}}]} ```