--- title: Build data pipelines in less time with Data Agent Kit description: Google Cloud introduces the Data Agent Kit, a free, open-source collection of data engineering tools that integrates into IDEs and CLIs like VS Code, Claude... canonical: https://daily.dev/posts/build-data-pipelines-in-less-time-with-data-agent-kit-k8cquzavs twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Build data pipelines in less time with Data Agent Kit | daily.dev og:description: Google Cloud introduces the Data Agent Kit, a free, open-source collection of data engineering tools that integrates into IDEs and CLIs like VS Code, Claude... og:url: https://daily.dev/posts/build-data-pipelines-in-less-time-with-data-agent-kit-k8cquzavs og:image: https://api.daily.dev/og/posts/k8CquzAVs.png og:image:alt: Build data pipelines in less time with Data Agent Kit og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Build data pipelines in less time with Data Agent Kit **[Google Cloud](https://daily.dev/sources/gcp)** · 9 min read · 1 upvotes · 0 comments ## Summary Google Cloud introduces the Data Agent Kit, a free, open-source collection of data engineering tools that integrates into IDEs and CLIs like VS Code, Claude Code, and Codex. It embeds the Orchestration Pipelines framework, letting data professionals author, deploy, and troubleshoot Apache Airflow DAGs using natural language and a declarative YAML DSL instead of Python boilerplate. A walkthrough demonstrates building an MLOps pipeline for predicting shipping delays using BigQuery, Managed Service for Apache Spark, Gemini Enterprise Agent Platform, and dbt, covering training, daily inference, and automated drift-based retraining. The kit also generates CI/CD workflows for deployment to Managed Airflow and offers in-IDE agentic troubleshooting for pipeline failures. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is the Google Cloud Data Agent Kit and what does it do for Airflow pipelines? The Data Agent Kit is a free, open-source collection of data engineering and data science tools that plugs into IDEs and CLIs such as VS Code, Claude Code, and Codex. It embeds the Orchestration Pipelines framework, letting users author, deploy, and troubleshoot production-grade Apache Airflow DAGs using natural language prompts and a declarative YAML DSL instead of writing Python Airflow boilerplate. _daily.dev surfaces tooling like this for engineers automating Airflow pipeline authoring and deployment._ ### How do I enable agentic troubleshooting for failed Airflow pipelines in Google Cloud's Data Agent Kit? Click the 'Troubleshoot' button inside the IDE after a pipeline fails; the Data Engineering Agent then analyzes the failure context, distinguishes infrastructure quota limits from code-level bugs, and produces a root-cause summary with a suggested inline fix, such as scaling up a compute template for an out-of-memory Spark cluster or a BigQuery quota issue. _daily.dev tracks practical fixes like this for teams debugging Airflow and MLOps pipeline failures._ ## Similar posts on daily.dev - [Exploring the Data Engineering Agent in BigQuery](https://daily.dev/posts/exploring-the-data-engineering-agent-in-bigquery-jh5qckdfs) · Google Cloud · 0 upvotes · 0 comments - [Enabling data scientists to become agentic architects](https://daily.dev/posts/enabling-data-scientists-to-become-agentic-architects-3wmyyqa5j) · Google Cloud · 1 upvotes · 0 comments - [Managed Apache Airflow: Scaling Data & AI Workloads](https://daily.dev/posts/managed-apache-airflow-scaling-data-ai-workloads-1s4kat53h) · Google Cloud · 0 upvotes · 0 comments - [Snowflake Cortex Code CLI adds dbt and Apache Airflow support for AI-powered data pipelines](https://daily.dev/posts/snowflake-cortex-code-cli-adds-dbt-and-apache-airflow-support-for-ai-powered-data-pipelines-yrixxu8v0) · The New Stack · 1 upvotes · 0 comments --- Tags: [#gcp](https://daily.dev/tags/gcp), [#data-engineering](https://daily.dev/tags/data-engineering), [#mlops](https://daily.dev/tags/mlops), [#google-bigquery](https://daily.dev/tags/google-bigquery), [#apache-airflow](https://daily.dev/tags/apache-airflow) [View this post on daily.dev](https://daily.dev/posts/build-data-pipelines-in-less-time-with-data-agent-kit-k8cquzavs) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Build data pipelines in less time with Data Agent Kit","url":"https://daily.dev/posts/build-data-pipelines-in-less-time-with-data-agent-kit-k8cquzavs","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/build-data-pipelines-in-less-time-with-data-agent-kit-k8cquzavs"},"datePublished":"2026-08-31T16:02:00.041Z","dateModified":"2026-08-31T16:02:25.092Z","description":"Google Cloud introduces the Data Agent Kit, a free, open-source collection of data engineering tools that integrates into IDEs and CLIs like VS Code, Claude...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/9df795f90d3290a29c3b5ccad1c3e312?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/9df795f90d3290a29c3b5ccad1c3e312?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Google Cloud","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Google Cloud","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/gcp","url":"https://daily.dev/sources/gcp"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/build-data-pipelines-in-less-time-with-data-agent-kit-k8cquzavs","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"gcp,data-engineering,mlops,google-bigquery,apache-airflow","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Google Cloud","item":"https://daily.dev/sources/gcp"},{"@type":"ListItem","position":3,"name":"Build data pipelines in less time with Data Agent Kit"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/build-data-pipelines-in-less-time-with-data-agent-kit-k8cquzavs#faq","mainEntity":[{"@type":"Question","name":"What is the Google Cloud Data Agent Kit and what does it do for Airflow pipelines?","acceptedAnswer":{"@type":"Answer","text":"The Data Agent Kit is a free, open-source collection of data engineering and data science tools that plugs into IDEs and CLIs such as VS Code, Claude Code, and Codex. It embeds the Orchestration Pipelines framework, letting users author, deploy, and troubleshoot production-grade Apache Airflow DAGs using natural language prompts and a declarative YAML DSL instead of writing Python Airflow boilerplate. daily.dev surfaces tooling like this for engineers automating Airflow pipeline authoring and deployment."}},{"@type":"Question","name":"How do I enable agentic troubleshooting for failed Airflow pipelines in Google Cloud's Data Agent Kit?","acceptedAnswer":{"@type":"Answer","text":"Click the 'Troubleshoot' button inside the IDE after a pipeline fails; the Data Engineering Agent then analyzes the failure context, distinguishes infrastructure quota limits from code-level bugs, and produces a root-cause summary with a suggested inline fix, such as scaling up a compute template for an out-of-memory Spark cluster or a BigQuery quota issue. daily.dev tracks practical fixes like this for teams debugging Airflow and MLOps pipeline failures."}}]} ```