--- title: Delta Lake on GCP | daily.dev description: A tutorial walks through using Delta Lake with Google Cloud Platform, covering why Delta Lake's transaction log avoids the slow file-listing overhead common... canonical: https://daily.dev/posts/delta-lake-on-gcp-hesiqz5sm twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Delta Lake on GCP | daily.dev og:description: A tutorial walks through using Delta Lake with Google Cloud Platform, covering why Delta Lake's transaction log avoids the slow file-listing overhead common... og:url: https://daily.dev/posts/delta-lake-on-gcp-hesiqz5sm og:image: https://api.daily.dev/og/posts/HeSiQz5Sm.png og:image:alt: Delta Lake on GCP og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Delta Lake on GCP **[Delta Lake Blog](https://daily.dev/sources/delta-lake)** · 9 min read · 0 upvotes · 0 comments ## Summary A tutorial walks through using Delta Lake with Google Cloud Platform, covering why Delta Lake's transaction log avoids the slow file-listing overhead common with cloud object stores. It explains setting up a GCS bucket, configuring Spark with the gcs-connector JAR and service account authentication, and reading/writing Delta tables with both Spark and Polars. It also covers updates, deletes, merges, time travel, and notes that GCS's native atomic object writes mean no external locking provider is needed, unlike with AWS S3. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Do I need a separate locking provider when using Delta Lake with Google Cloud Storage? No, a separate locking provider is not needed with Google Cloud Storage because GCS natively supports atomic operations at the object level. Each file creation or modification is treated as a single transaction, so writes either complete fully or fail without leaving a file in a partial state, unlike AWS S3 which requires additional locking mechanisms for concurrent writes. _Compare storage backend requirements like this on daily.dev before picking a lakehouse setup._ ### What error do I get if my Spark session is not configured correctly for Delta Lake on GCP? An error indicating that Spark has not been set up with the correct GoogleHadoopFileSystem appears when the gcs-connector JAR is missing or the spark.hadoop.fs.gs.impl config is not set correctly. Fixing this requires downloading the gcs-connector JAR, adding it to the Spark session, and setting the file system and authentication configs explicitly. _Track configuration gotchas like this Spark and GCS setup issue on daily.dev._ ### Do I need extra dependencies to read and write Delta tables to Google Cloud Storage with Polars? No extra dependencies are needed to read or write Delta tables to GCS with engines built on delta-rs, such as Polars. What is required is correctly configured GCS access credentials, either through Application Default Credentials (ADC) for automatic detection, or by passing credentials explicitly using Polars' storage_options keyword. _Follow practical setup differences between engines like Spark and Polars on daily.dev._ ## Similar posts on daily.dev - [Delta Lake Explained: Boost Data Reliability in Cloud Storage](https://daily.dev/posts/delta-lake-explained-boost-data-reliability-in-cloud-storage-oechbmvm4) · databricks · 0 upvotes · 0 comments - [650GB of Data \(Delta Lake on S3\). Polars vs DuckDB vs Daft vs Spark.](https://daily.dev/posts/650gb-of-data-delta-lake-on-s3-polars-vs-duckdb-vs-daft-vs-spark--bbkmmear9) · Hacker News · 1 upvotes · 0 comments - [Expensive Delta Lake S3 Storage Mistakes \(And How to Fix Them\)](https://daily.dev/posts/expensive-delta-lake-s3-storage-mistakes-and-how-to-fix-them--iyqaxbwgz) · databricks · 3 upvotes · 0 comments --- Tags: [#gcp](https://daily.dev/tags/gcp), [#data-engineering](https://daily.dev/tags/data-engineering), [#apache-spark](https://daily.dev/tags/apache-spark), [#polars](https://daily.dev/tags/polars) [View this post on daily.dev](https://daily.dev/posts/delta-lake-on-gcp-hesiqz5sm) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Delta Lake on GCP","url":"https://daily.dev/posts/delta-lake-on-gcp-hesiqz5sm","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/delta-lake-on-gcp-hesiqz5sm"},"datePublished":"2026-08-16T08:53:24.169Z","dateModified":"2026-08-16T08:55:19.036Z","description":"A tutorial walks through using Delta Lake with Google Cloud Platform, covering why Delta Lake's transaction log avoids the slow file-listing overhead common...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/8862c40db88be72df78d5de9dfe9d476?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/8862c40db88be72df78d5de9dfe9d476?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Delta Lake Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Delta Lake Blog","logo":"https://media.daily.dev/image/upload/s--7f2q9Asf--/f_auto,q_auto/v1786870393/logos/delta-lake?_a=BAMAMicg0","url":"https://daily.dev/sources/delta-lake"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/delta-lake-on-gcp-hesiqz5sm","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"gcp,data-engineering,apache-spark,polars","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Delta Lake Blog","item":"https://daily.dev/sources/delta-lake"},{"@type":"ListItem","position":3,"name":"Delta Lake on GCP"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/delta-lake-on-gcp-hesiqz5sm#faq","mainEntity":[{"@type":"Question","name":"Do I need a separate locking provider when using Delta Lake with Google Cloud Storage?","acceptedAnswer":{"@type":"Answer","text":"No, a separate locking provider is not needed with Google Cloud Storage because GCS natively supports atomic operations at the object level. Each file creation or modification is treated as a single transaction, so writes either complete fully or fail without leaving a file in a partial state, unlike AWS S3 which requires additional locking mechanisms for concurrent writes. Compare storage backend requirements like this on daily.dev before picking a lakehouse setup."}},{"@type":"Question","name":"What error do I get if my Spark session is not configured correctly for Delta Lake on GCP?","acceptedAnswer":{"@type":"Answer","text":"An error indicating that Spark has not been set up with the correct GoogleHadoopFileSystem appears when the gcs-connector JAR is missing or the spark.hadoop.fs.gs.impl config is not set correctly. Fixing this requires downloading the gcs-connector JAR, adding it to the Spark session, and setting the file system and authentication configs explicitly. Track configuration gotchas like this Spark and GCS setup issue on daily.dev."}},{"@type":"Question","name":"Do I need extra dependencies to read and write Delta tables to Google Cloud Storage with Polars?","acceptedAnswer":{"@type":"Answer","text":"No extra dependencies are needed to read or write Delta tables to GCS with engines built on delta-rs, such as Polars. What is required is correctly configured GCS access credentials, either through Application Default Credentials (ADC) for automatic detection, or by passing credentials explicitly using Polars' storage_options keyword. Follow practical setup differences between engines like Spark and Polars on daily.dev."}}]} ```