--- title: "The AI-native Data Engineer | OD818" url: https://daily.dev/posts/the-ai-native-data-engineer-od818-nn8ug7pvr source_url: https://www.youtube.com/watch?v=I-m7bUUprzY type: video:youtube source: "Microsoft Developer" published: 2026-06-03T13:34:26.110Z updated: 2026-06-03T13:34:49.403Z tags: ["github", "data-engineering", "apache-spark", "microsoft-fabric"] reading_time: 31 upvotes: 0 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # The AI-native Data Engineer | OD818 **[Microsoft Developer](https://daily.dev/sources/microsoftdeveloper)** · 31 min read · 0 upvotes · 0 comments ## Summary A Microsoft Build session exploring how AI is transforming the data engineering workflow using Microsoft Fabric, VS Code, and GitHub Copilot. Two presenters demonstrate practical AI-native workflows: building a streaming ingestion pipeline from Kafka/EventStream into a medallion architecture (bronze/silver/gold Delta Lake tables) using Copilot in agent mode, and a local development setup where production OneLake data is mounted via Blobfuse into a VS Code dev container running local Spark. Key techniques include authoring AI-friendly codebases with consistent interfaces (e.g., a DataTransformer trait in Scala), using bounded context 'skills' to constrain Copilot's behavior, building deterministic CLI tools for AI to invoke, and parallelizing work across multiple Copilot CLI instances via GitHub Actions. The 'Ralph loop' concept is introduced as a mental model for giving AI a well-defined task within a bounded context. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: --- Tags: [#github](https://daily.dev/tags/github), [#data-engineering](https://daily.dev/tags/data-engineering), [#apache-spark](https://daily.dev/tags/apache-spark), [#microsoft-fabric](https://daily.dev/tags/microsoft-fabric) [View this post on daily.dev](https://daily.dev/posts/the-ai-native-data-engineer-od818-nn8ug7pvr)