--- title: When Catalogs Are Embedded in Storage | daily.dev description: Part 8 of a 15-part Apache Iceberg masterclass explores embedded catalogs, where the storage layer itself manages Iceberg metadata instead of relying on a... canonical: https://daily.dev/posts/when-catalogs-are-embedded-in-storage-htzv1c1dz twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: When Catalogs Are Embedded in Storage | daily.dev og:description: Part 8 of a 15-part Apache Iceberg masterclass explores embedded catalogs, where the storage layer itself manages Iceberg metadata instead of relying on a... og:url: https://daily.dev/posts/when-catalogs-are-embedded-in-storage-htzv1c1dz og:image: https://api.daily.dev/og/posts/HTZv1c1dz.png og:image:alt: When Catalogs Are Embedded in Storage og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # When Catalogs Are Embedded in Storage **[Dremio Blog](https://daily.dev/sources/dremio)** · 6 min read · 0 upvotes · 0 comments ## Summary Part 8 of a 15-part Apache Iceberg masterclass explores embedded catalogs, where the storage layer itself manages Iceberg metadata instead of relying on a separate catalog service. It covers AWS S3 Tables, launched in late 2024 as a dedicated S3 bucket type with built-in compaction, snapshot expiry, and orphan cleanup, and MinIO AI Stor, an on-premises alternative with S3 compatibility and GPU-accelerated analytics. Both trade flexibility for operational simplicity but come with vendor lock-in, limited monitoring, and cross-region latency concerns. The piece also discusses hybrid approaches combining embedded and standalone catalogs like Dremio Open Catalog for unified multi-source analytics. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is AWS S3 Tables and how does it embed the Iceberg catalog? AWS S3 Tables is a dedicated S3 bucket type, launched in late 2024, designed specifically for Apache Iceberg tables. It manages the Iceberg catalog internally, handling metadata files, atomic commits, compaction, and snapshot management within the storage layer, so engines interact with a single endpoint for both catalog operations and data access via a REST-catalog-compatible interface. _Track how storage-embedded catalogs like S3 Tables reshape lakehouse architecture decisions on daily.dev._ ### What are the downsides of using an embedded catalog like S3 Tables instead of a standalone Iceberg catalog? Embedded catalogs create vendor dependency since tables are tied to the storage vendor's lifecycle, and migrating away requires converting every table to a different catalog. They also offer limited monitoring of internal maintenance like compaction schedules, add cross-region latency if compute runs elsewhere, and in S3 Tables' case, limit governance to what AWS IAM provides without column or row-level controls. _Compare catalog trade-offs before locking a lakehouse architecture in, informed by discussions on daily.dev._ ### How does MinIO AI Stor implement an embedded Iceberg catalog for on-premises deployments? MinIO AI Stor embeds Iceberg catalog functionality directly into its S3-compatible storage layer, managing table metadata, file tracking, and maintenance internally as data is written. Because it implements the S3 API, engines like Spark, Trino, and Dremio can interact with it with minimal configuration changes, and it also integrates GPU-aware processing and vector storage for AI workloads alongside Iceberg tables. _Follow on-premises lakehouse tooling like MinIO AI Stor as options evolve, curated on daily.dev._ ## Similar posts on daily.dev - [Iceberg Needs an Open Catalog, Not a Walled Garden](https://daily.dev/posts/iceberg-needs-an-open-catalog-not-a-walled-garden-znv0xeptl) · SingleStore · 0 upvotes · 0 comments - [Apache Iceberg on AWS: Glue Catalog, Athena, EMR, and S3 Tables](https://daily.dev/posts/apache-iceberg-on-aws-glue-catalog-athena-emr-and-s3-tables-qgxux89im) · BigData Boutique blog · 0 upvotes · 0 comments - [Scaling Grab's Data Lake: Our journey to Apache Iceberg adoption](https://daily.dev/posts/scaling-grab-s-data-lake-our-journey-to-apache-iceberg-adoption-ibvihkenm) · Grab Tech Blog · 6 upvotes · 0 comments - [The Iceberg Catalog Layer Explained: Snowflake Horizon, Open Catalog, Polaris, Glue and Nessie Compared](https://daily.dev/posts/the-iceberg-catalog-layer-explained-snowflake-horizon-open-catalog-polaris-glue-and-nessie-compa-hog2inqqf) · Snowflake Community · 0 upvotes · 0 comments --- Tags: [#data-engineering](https://daily.dev/tags/data-engineering), [#apache-iceberg](https://daily.dev/tags/apache-iceberg) [View this post on daily.dev](https://daily.dev/posts/when-catalogs-are-embedded-in-storage-htzv1c1dz) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"When Catalogs Are Embedded in Storage","url":"https://daily.dev/posts/when-catalogs-are-embedded-in-storage-htzv1c1dz","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/when-catalogs-are-embedded-in-storage-htzv1c1dz"},"datePublished":"2026-08-16T08:53:14.283Z","dateModified":"2026-08-16T08:53:49.559Z","description":"Part 8 of a 15-part Apache Iceberg masterclass explores embedded catalogs, where the storage layer itself manages Iceberg metadata instead of relying on a...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/90eb3e00dc57e35e7d35df76c0bd28df?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/90eb3e00dc57e35e7d35df76c0bd28df?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Dremio Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Dremio Blog","logo":"https://media.daily.dev/image/upload/s--MGKANojy--/f_auto,q_auto/v1786870389/logos/dremio?_a=BAMAMicg0","url":"https://daily.dev/sources/dremio"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/when-catalogs-are-embedded-in-storage-htzv1c1dz","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-engineering,apache-iceberg","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Dremio Blog","item":"https://daily.dev/sources/dremio"},{"@type":"ListItem","position":3,"name":"When Catalogs Are Embedded in Storage"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/when-catalogs-are-embedded-in-storage-htzv1c1dz#faq","mainEntity":[{"@type":"Question","name":"What is AWS S3 Tables and how does it embed the Iceberg catalog?","acceptedAnswer":{"@type":"Answer","text":"AWS S3 Tables is a dedicated S3 bucket type, launched in late 2024, designed specifically for Apache Iceberg tables. It manages the Iceberg catalog internally, handling metadata files, atomic commits, compaction, and snapshot management within the storage layer, so engines interact with a single endpoint for both catalog operations and data access via a REST-catalog-compatible interface. Track how storage-embedded catalogs like S3 Tables reshape lakehouse architecture decisions on daily.dev."}},{"@type":"Question","name":"What are the downsides of using an embedded catalog like S3 Tables instead of a standalone Iceberg catalog?","acceptedAnswer":{"@type":"Answer","text":"Embedded catalogs create vendor dependency since tables are tied to the storage vendor's lifecycle, and migrating away requires converting every table to a different catalog. They also offer limited monitoring of internal maintenance like compaction schedules, add cross-region latency if compute runs elsewhere, and in S3 Tables' case, limit governance to what AWS IAM provides without column or row-level controls. Compare catalog trade-offs before locking a lakehouse architecture in, informed by discussions on daily.dev."}},{"@type":"Question","name":"How does MinIO AI Stor implement an embedded Iceberg catalog for on-premises deployments?","acceptedAnswer":{"@type":"Answer","text":"MinIO AI Stor embeds Iceberg catalog functionality directly into its S3-compatible storage layer, managing table metadata, file tracking, and maintenance internally as data is written. Because it implements the S3 API, engines like Spark, Trino, and Dremio can interact with it with minimal configuration changes, and it also integrates GPU-aware processing and vector storage for AI workloads alongside Iceberg tables. Follow on-premises lakehouse tooling like MinIO AI Stor as options evolve, curated on daily.dev."}}]} ```