--- title: Building event-driven pipelines with SQS and S3 | daily.dev description: Learn how to build automated, event-driven data pipelines by configuring S3 to send notifications to SQS when new objects are created, then using Redpanda... canonical: https://daily.dev/posts/building-event-driven-pipelines-with-sqs-and-s3-k7tul0gy8 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Building event-driven pipelines with SQS and S3 | daily.dev og:description: Learn how to build automated, event-driven data pipelines by configuring S3 to send notifications to SQS when new objects are created, then using Redpanda... og:url: https://daily.dev/posts/building-event-driven-pipelines-with-sqs-and-s3-k7tul0gy8 og:image: https://api.daily.dev/og/posts/k7tul0GY8.png og:image:alt: Building event-driven pipelines with SQS and S3 og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Building event-driven pipelines with SQS and S3 **[Redpanda](https://daily.dev/sources/redpanda)** · 4 min read · 0 upvotes · 0 comments ## Summary Learn how to build automated, event-driven data pipelines by configuring S3 to send notifications to SQS when new objects are created, then using Redpanda Connect to listen to the queue and automatically process uploaded files. This tutorial walks through setting up S3 event notifications, configuring Redpanda Connect to poll SQS, and parsing event messages to fetch and process S3 objects in real-time without manual intervention. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Unordered mode for S3 ClickPipes is now available](https://daily.dev/posts/unordered-mode-for-s3-clickpipes-is-now-available-awfb1gmox) · ClickHouse · 2 upvotes · 0 comments - [How to Collect Server Events Across Hundreds of Microservices](https://daily.dev/posts/how-to-collect-server-events-across-hundreds-of-microservices-orjlpldo1) · ITNEXT · 0 upvotes · 0 comments - [Event-based Data Ingestion and Transformation with Snowpipe](https://daily.dev/posts/event-based-data-ingestion-and-transformation-with-snowpipe-s5pxec5nv) · Snowflake Community · 0 upvotes · 0 comments - [The Complete Guide to Message Queues and Event Streaming: Building Scalable, Reliable, and Real-Time Distributed Systems](https://daily.dev/posts/the-complete-guide-to-message-queues-and-event-streaming-building-scalable-reliable-and-real-time-72zillsk8) · Medium · 0 upvotes · 0 comments - [Your bucket is already a message queue](https://daily.dev/posts/your-bucket-is-already-a-message-queue-1szqzwfrk) · Tigris · 3 upvotes · 0 comments --- Tags: [#aws](https://daily.dev/tags/aws), [#architecture](https://daily.dev/tags/architecture), [#data-engineering](https://daily.dev/tags/data-engineering) [View this post on daily.dev](https://daily.dev/posts/building-event-driven-pipelines-with-sqs-and-s3-k7tul0gy8) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Building event-driven pipelines with SQS and S3","url":"https://daily.dev/posts/building-event-driven-pipelines-with-sqs-and-s3-k7tul0gy8","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/building-event-driven-pipelines-with-sqs-and-s3-k7tul0gy8"},"datePublished":"2026-02-01T08:36:54.308Z","dateModified":"2026-02-01T08:39:05.621Z","description":"Learn how to build automated, event-driven data pipelines by configuring S3 to send notifications to SQS when new objects are created, then using Redpanda...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/093280334527dba0410f56a1d018523c?_a=AQAEulh","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/093280334527dba0410f56a1d018523c?_a=AQAEulh","isAccessibleForFree":true,"articleSection":"Redpanda","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Redpanda","logo":"https://media.daily.dev/image/upload/s--1R2EIDcb--/f_auto,q_auto/v1769934743/logos/redpanda","url":"https://daily.dev/sources/redpanda"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/building-event-driven-pipelines-with-sqs-and-s3-k7tul0gy8","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"aws,architecture,data-engineering","timeRequired":"PT4M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Redpanda","item":"https://daily.dev/sources/redpanda"},{"@type":"ListItem","position":3,"name":"Building event-driven pipelines with SQS and S3"}]} ```