--- title: Delta Lake Schema Evolution | daily.dev description: A walkthrough of how Delta Lake handles schema evolution, contrasting schema enforcement (the default, which rejects mismatched schemas) with schema evolution... canonical: https://daily.dev/posts/delta-lake-schema-evolution-2yxwgj1fk twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Delta Lake Schema Evolution | daily.dev og:description: A walkthrough of how Delta Lake handles schema evolution, contrasting schema enforcement (the default, which rejects mismatched schemas) with schema evolution... og:url: https://daily.dev/posts/delta-lake-schema-evolution-2yxwgj1fk og:image: https://api.daily.dev/og/posts/2yxwgj1FK.png og:image:alt: Delta Lake Schema Evolution og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Delta Lake Schema Evolution **[Delta Lake Blog](https://daily.dev/sources/delta-lake)** · 8 min read · 0 upvotes · 0 comments ## Summary A walkthrough of how Delta Lake handles schema evolution, contrasting schema enforcement (the default, which rejects mismatched schemas) with schema evolution enabled via mergeSchema (single-write) or autoMerge (global) options. Code examples show appending DataFrames with extra, missing, or completely different columns. The piece also compares this to Parquet tables, which lack real schema enforcement and instead rely on an inconsistent, expensive read-time mergeSchema setting that only reads schema from a single file by default, leading to unpredictable results. Delta Lake's schema-on-write approach using the transaction log is presented as faster and safer than data lakes' schema-on-read model. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is the difference between mergeSchema and autoMerge in Delta Lake? mergeSchema is set per write operation and enables schema evolution only for that single write to a single table, making it the safer, more contained option. autoMerge is a broader setting that activates schema evolution for writes to any table, offering convenience but more risk since it permissively allows appending data with any schema anywhere. mergeSchema is generally recommended unless global schema evolution is truly needed. _daily.dev surfaces engineering deep dives like this for teams weighing Delta Lake configuration trade-offs._ ### Why does appending a DataFrame with extra columns to a Delta table fail by default? Delta Lake enforces schema enforcement by default, which throws an AnalysisException when the incoming DataFrame's schema doesn't match the existing table's schema exactly. This protects tables from unintended structural changes. To bypass it, set the mergeSchema option to true on the write, which enables schema evolution and merges the new columns into the table, filling missing values with null. _Developers debugging Delta Lake schema errors can find practical fixes like this through daily.dev._ ### Why do Parquet tables give inconsistent results when files have different schemas? Spark reads the schema from only a single Parquet file by default and assumes it applies to every file in the table, silently ignoring columns present in other files, which produces unpredictable and inconsistent results across runs. Setting mergeSchema to true when reading forces Spark to inspect every file's schema, but this is slower and is a completely separate setting from Delta Lake's write-time mergeSchema option. _daily.dev helps data engineers compare Parquet and Delta Lake pitfalls before choosing a storage format._ ## Similar posts on daily.dev - [Schema evolution in data pipelines: the engineer's guide](https://daily.dev/posts/schema-evolution-in-data-pipelines-the-engineer-s-guide-yft3nuqgb) · dltHub · 0 upvotes · 0 comments - [Delta Lake Explained: Boost Data Reliability in Cloud Storage](https://daily.dev/posts/delta-lake-explained-boost-data-reliability-in-cloud-storage-oechbmvm4) · databricks · 0 upvotes · 0 comments - [Apache Iceberg vs Delta Lake: Choosing the Right Table Format](https://daily.dev/posts/apache-iceberg-vs-delta-lake-choosing-the-right-table-format-vtnia2ems) · BigData Boutique blog · 7 upvotes · 0 comments --- Tags: [#apache-spark](https://daily.dev/tags/apache-spark), [#big-data](https://daily.dev/tags/big-data), [#database](https://daily.dev/tags/database), [#data-engineering](https://daily.dev/tags/data-engineering) [View this post on daily.dev](https://daily.dev/posts/delta-lake-schema-evolution-2yxwgj1fk) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Delta Lake Schema Evolution","url":"https://daily.dev/posts/delta-lake-schema-evolution-2yxwgj1fk","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/delta-lake-schema-evolution-2yxwgj1fk"},"datePublished":"2026-08-16T08:53:53.733Z","dateModified":"2026-10-15T02:15:42.869Z","description":"A walkthrough of how Delta Lake handles schema evolution, contrasting schema enforcement (the default, which rejects mismatched schemas) with schema evolution...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/1fcabbc5b8d25c7aecfd241ff931f198?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/1fcabbc5b8d25c7aecfd241ff931f198?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Delta Lake Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Delta Lake Blog","logo":"https://media.daily.dev/image/upload/s--7f2q9Asf--/f_auto,q_auto/v1786870393/logos/delta-lake?_a=BAMAMicg0","url":"https://daily.dev/sources/delta-lake"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/delta-lake-schema-evolution-2yxwgj1fk","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"apache-spark,big-data,database,data-engineering","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Delta Lake Blog","item":"https://daily.dev/sources/delta-lake"},{"@type":"ListItem","position":3,"name":"Delta Lake Schema Evolution"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/delta-lake-schema-evolution-2yxwgj1fk#faq","mainEntity":[{"@type":"Question","name":"What is the difference between mergeSchema and autoMerge in Delta Lake?","acceptedAnswer":{"@type":"Answer","text":"mergeSchema is set per write operation and enables schema evolution only for that single write to a single table, making it the safer, more contained option. autoMerge is a broader setting that activates schema evolution for writes to any table, offering convenience but more risk since it permissively allows appending data with any schema anywhere. mergeSchema is generally recommended unless global schema evolution is truly needed. daily.dev surfaces engineering deep dives like this for teams weighing Delta Lake configuration trade-offs."}},{"@type":"Question","name":"Why does appending a DataFrame with extra columns to a Delta table fail by default?","acceptedAnswer":{"@type":"Answer","text":"Delta Lake enforces schema enforcement by default, which throws an AnalysisException when the incoming DataFrame's schema doesn't match the existing table's schema exactly. This protects tables from unintended structural changes. To bypass it, set the mergeSchema option to true on the write, which enables schema evolution and merges the new columns into the table, filling missing values with null. Developers debugging Delta Lake schema errors can find practical fixes like this through daily.dev."}},{"@type":"Question","name":"Why do Parquet tables give inconsistent results when files have different schemas?","acceptedAnswer":{"@type":"Answer","text":"Spark reads the schema from only a single Parquet file by default and assumes it applies to every file in the table, silently ignoring columns present in other files, which produces unpredictable and inconsistent results across runs. Setting mergeSchema to true when reading forces Spark to inspect every file's schema, but this is slower and is a completely separate setting from Delta Lake's write-time mergeSchema option. daily.dev helps data engineers compare Parquet and Delta Lake pitfalls before choosing a storage format."}}]} ```