---
title: "How Partition Access Visualizations Reduced our Data Lake S3 Cost by 33%"
url: https://daily.dev/posts/how-partition-access-visualizations-reduced-our-data-lake-s3-cost-by-33--dnz5wsm95
source_url: https://engineeringblog.yelp.com/2026/05/partition-access-visualizations.html
type: article
source: "Yelp Engineering"
published: 2026-05-20T22:07:23.070Z
updated: 2026-05-20T22:07:52.524Z
tags: ["aws-s3", "finops", "data-lake", "apache-iceberg"]
reading_time: 7
upvotes: 0
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# How Partition Access Visualizations Reduced our Data Lake S3 Cost by 33%

**[Yelp Engineering](https://daily.dev/sources/yelp)** · 7 min read · 0 upvotes · 0 comments

## Summary

Yelp's data platform team built a system to visualize S3 partition access patterns by plotting partition key values against access event timestamps. These visualizations reveal distinct usage signatures — daily batch jobs, backfills, and ad hoc queries — attributed to specific IAM roles. Armed with granular usage data, teams confidently adopted more cost-effective S3 Storage Classes (leveraging Intelligent Tiering's 40–81% cost reductions), expanded deletion-based retention policies, and introduced a 'Default Access Retention' mechanism that gates cold data behind IAM policies requiring explicit approval. The combined effort reduced Yelp's petabyte-scale data lake S3 storage cost by 33%. The implementation uses Amazon S3 server access logs aggregated via SQL, and the same usage data also guided prioritization of Apache Iceberg migration efforts toward the most actively used tables.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://engineeringblog.yelp.com/2026/05/partition-access-visualizations.html>

## Similar posts on daily.dev

- [S3 server access logs at scale](https://daily.dev/posts/s3-server-access-logs-at-scale-77agftnrm) · Yelp Engineering · 0 upvotes · 0 comments
- [Yelp Publishes Blueprint for Managing S3 Server-Access Logs at Massive Scale](https://daily.dev/posts/yelp-publishes-blueprint-for-managing-s3-server-access-logs-at-massive-scale-l0en9vc0e) · InfoQ · 2 upvotes · 0 comments
- [AWS Adds Intelligent-Tiering and Replication for S3 Tables](https://daily.dev/posts/aws-adds-intelligent-tiering-and-replication-for-s3-tables-ku8f0ksmd) · InfoQ · 0 upvotes · 0 comments
- [Analyze Cloud Storage estates with Storage Insights datasets](https://daily.dev/posts/analyze-cloud-storage-estates-with-storage-insights-datasets-weoln8qtg) · Google Cloud · 0 upvotes · 0 comments
- [Amazon Quick now supports S3 tables bucket as a data source](https://daily.dev/posts/amazon-quick-now-supports-s3-tables-bucket-as-a-data-source-awsjs4sqe) · AWS · 0 upvotes · 0 comments

---

Tags: [#aws-s3](https://daily.dev/tags/aws-s3), [#finops](https://daily.dev/tags/finops), [#data-lake](https://daily.dev/tags/data-lake), [#apache-iceberg](https://daily.dev/tags/apache-iceberg)

[View this post on daily.dev](https://daily.dev/posts/how-partition-access-visualizations-reduced-our-data-lake-s3-cost-by-33--dnz5wsm95)