--- title: State of the Open Lakehouse, September 2026 | daily.dev description: A comprehensive September 2026 status report covers the five Apache projects underpinning the open lakehouse stack: Parquet, Arrow, Iceberg, Polaris, and the... canonical: https://daily.dev/posts/state-of-the-open-lakehouse-september-2026-zuxkpbgvy twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: State of the Open Lakehouse, September 2026 | daily.dev og:description: A comprehensive September 2026 status report covers the five Apache projects underpinning the open lakehouse stack: Parquet, Arrow, Iceberg, Polaris, and the... og:url: https://daily.dev/posts/state-of-the-open-lakehouse-september-2026-zuxkpbgvy og:image: https://api.daily.dev/og/posts/zUXkpBgVy.png og:image:alt: State of the Open Lakehouse, September 2026 og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # State of the Open Lakehouse, September 2026 **[Dremio Blog](https://daily.dev/sources/dremio)** · 35 min read · 10 upvotes · 0 comments ## Summary A comprehensive September 2026 status report covers the five Apache projects underpinning the open lakehouse stack: Parquet, Arrow, Iceberg, Polaris, and the newly incubating Ossie. Parquet passed a versioning policy for forward-incompatible changes and is debating a FILE logical type proposal. Arrow turned ten, shipped 25.0.1, and published a formal security model for untrusted data boundaries. Iceberg 1.11.0 made V3 (deletion vectors, Variant shredding, geospatial types) the stable default while V4 design work progresses in public, including deprecating equality deletes and moving manifests toward Parquet-only. Polaris, now a Top-Level Project, is becoming a control-plane catalog handling identity, credential vending, tags, and federation policy. Ossie, incubating since June 2026, is a new specification for business semantics (metrics, dimensions, joins) aimed at grounding AI agents in consistent definitions. The piece closes with operational warnings (small files, manifest bloat, equality delete accumulation, credential expiry) and a checklist for teams running this stack. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What changed in Apache Iceberg 1.11.0 regarding deletion vectors and table format V3? Iceberg 1.11.0, released May 19, 2026, made the V3 feature set the stable default instead of experimental. Deletion vectors replaced position delete files with a compact per-data-file bitmap, eliminating the long tail of tiny delete files merge-on-read tables used to accumulate. The release also added Variant type shredding, native geospatial types, nanosecond timestamps, and made Spark 4.1 and Flink 2.1 the default build targets. _Teams planning an iceberg V3 migration can track format changes like this as they land on daily.dev._ ### Why are equality deletes being deprecated in Apache Iceberg V4? Equality deletes are being deprecated in Iceberg V4 because they push cost onto every reader indefinitely until compaction resolves them, since resolving an equality delete means evaluating a predicate against every candidate row. The vote to deprecate them and forbid new writes has passed, with a spec PR in progress, so streaming pipelines using equality deletes for upserts need a deletion-vector-based merge-on-read strategy or a compaction cadence instead. _Engineers migrating streaming upsert pipelines off equality deletes can follow iceberg V4 developments on daily.dev._ ### What does the Arrow project's 2026 security model cover and why was it published? Published in February 2026, Arrow's formal security model covers the columnar format, the C Data Interface, and the IPC format, giving guidelines for handling Arrow data from untrusted sources. It exists because Arrow now regularly crosses organizational trust boundaries via Flight endpoints, ADBC drivers, and downloaded IPC files, where a reader trusting hostile offsets and lengths becomes a memory safety problem rather than a simple bug. _Developers exposing Flight or ADBC endpoints can keep up with arrow security guidance via daily.dev._ ## Similar posts on daily.dev - [Open Data Lakehouse: A Practical Guide](https://daily.dev/posts/open-data-lakehouse-a-practical-guide-zxq0hjdsv) · ITNEXT · 0 upvotes · 0 comments --- Tags: [#data-engineering](https://daily.dev/tags/data-engineering), [#apache-iceberg](https://daily.dev/tags/apache-iceberg), [#apache-arrow](https://daily.dev/tags/apache-arrow) [View this post on daily.dev](https://daily.dev/posts/state-of-the-open-lakehouse-september-2026-zuxkpbgvy) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"State of the Open Lakehouse, September 2026","url":"https://daily.dev/posts/state-of-the-open-lakehouse-september-2026-zuxkpbgvy","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/state-of-the-open-lakehouse-september-2026-zuxkpbgvy"},"datePublished":"2026-09-03T14:08:31.242Z","dateModified":"2026-09-03T14:30:05.739Z","description":"A comprehensive September 2026 status report covers the five Apache projects underpinning the open lakehouse stack: Parquet, Arrow, Iceberg, Polaris, and the...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/10ce81700607758c4adf2b7d4b48f870?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/10ce81700607758c4adf2b7d4b48f870?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Dremio Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Dremio Blog","logo":"https://media.daily.dev/image/upload/s--MGKANojy--/f_auto,q_auto/v1786870389/logos/dremio?_a=BAMAMicg0","url":"https://daily.dev/sources/dremio"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/state-of-the-open-lakehouse-september-2026-zuxkpbgvy","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":10},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-engineering,apache-iceberg,apache-arrow","timeRequired":"PT35M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Dremio Blog","item":"https://daily.dev/sources/dremio"},{"@type":"ListItem","position":3,"name":"State of the Open Lakehouse, September 2026"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/state-of-the-open-lakehouse-september-2026-zuxkpbgvy#faq","mainEntity":[{"@type":"Question","name":"What changed in Apache Iceberg 1.11.0 regarding deletion vectors and table format V3?","acceptedAnswer":{"@type":"Answer","text":"Iceberg 1.11.0, released May 19, 2026, made the V3 feature set the stable default instead of experimental. Deletion vectors replaced position delete files with a compact per-data-file bitmap, eliminating the long tail of tiny delete files merge-on-read tables used to accumulate. The release also added Variant type shredding, native geospatial types, nanosecond timestamps, and made Spark 4.1 and Flink 2.1 the default build targets. Teams planning an iceberg V3 migration can track format changes like this as they land on daily.dev."}},{"@type":"Question","name":"Why are equality deletes being deprecated in Apache Iceberg V4?","acceptedAnswer":{"@type":"Answer","text":"Equality deletes are being deprecated in Iceberg V4 because they push cost onto every reader indefinitely until compaction resolves them, since resolving an equality delete means evaluating a predicate against every candidate row. The vote to deprecate them and forbid new writes has passed, with a spec PR in progress, so streaming pipelines using equality deletes for upserts need a deletion-vector-based merge-on-read strategy or a compaction cadence instead. Engineers migrating streaming upsert pipelines off equality deletes can follow iceberg V4 developments on daily.dev."}},{"@type":"Question","name":"What does the Arrow project's 2026 security model cover and why was it published?","acceptedAnswer":{"@type":"Answer","text":"Published in February 2026, Arrow's formal security model covers the columnar format, the C Data Interface, and the IPC format, giving guidelines for handling Arrow data from untrusted sources. It exists because Arrow now regularly crosses organizational trust boundaries via Flight endpoints, ADBC drivers, and downloaded IPC files, where a reader trusting hostile offsets and lengths becomes a memory safety problem rather than a simple bug. Developers exposing Flight or ADBC endpoints can keep up with arrow security guidance via daily.dev."}}]} ```