---
title: "Streaming optimized data to S3 for analytics with Parquet"
url: https://daily.dev/posts/streaming-optimized-data-to-s3-for-analytics-with-parquet-byi9pv92m
source_url: https://www.redpanda.com/blog/streaming-data-s3-analytics-parquet
type: article
source: "Redpanda"
published: 2026-02-01T08:36:37.951Z
updated: 2026-02-01T08:38:42.171Z
tags: ["aws", "big-data", "data-engineering"]
reading_time: 5
upvotes: 0
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming optimized data to S3 for analytics with Parquet

**[Redpanda](https://daily.dev/sources/redpanda)** · 5 min read · 0 upvotes · 0 comments

## Summary

Apache Parquet is a columnar binary format optimized for analytics workloads like Spark and Athena. Redpanda Connect can encode streaming data directly into compressed Parquet files and write them to S3. The tutorial demonstrates configuring a pipeline that reads from a Redpanda topic, batches messages, encodes them using a defined schema with the parquet_encode processor, applies zstd compression, and outputs to S3. This approach enables a single data stream to serve both real-time applications (JSON) and analytics systems (Parquet).

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://www.redpanda.com/blog/streaming-data-s3-analytics-parquet>

---

Tags: [#aws](https://daily.dev/tags/aws), [#big-data](https://daily.dev/tags/big-data), [#data-engineering](https://daily.dev/tags/data-engineering)

[View this post on daily.dev](https://daily.dev/posts/streaming-optimized-data-to-s3-for-analytics-with-parquet-byi9pv92m)