--- title: How we built Cloudflare's data platform and an AI agent... description: Cloudflare built Town Lake, a unified data lakehouse platform, to replace fragmented data systems spanning Postgres, ClickHouse, BigQuery, Kafka, and R2. The... canonical: https://daily.dev/posts/how-we-built-cloudflare-s-data-platform-and-an-ai-agent-on-top-of-it-gwui5xgs3 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How we built Cloudflare's data platform and an AI agent on top of it | daily.dev og:description: Cloudflare built Town Lake, a unified data lakehouse platform, to replace fragmented data systems spanning Postgres, ClickHouse, BigQuery, Kafka, and R2. The... og:url: https://daily.dev/posts/how-we-built-cloudflare-s-data-platform-and-an-ai-agent-on-top-of-it-gwui5xgs3 og:image: https://api.daily.dev/og/posts/Gwui5Xgs3.png og:image:alt: How we built Cloudflare's data platform and an AI agent on top of it og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How we built Cloudflare's data platform and an AI agent on top of it **[Cloudflare](https://daily.dev/sources/cloudflare)** · 17 min read · 0 upvotes · 0 comments ## Summary Cloudflare built Town Lake, a unified data lakehouse platform, to replace fragmented data systems spanning Postgres, ClickHouse, BigQuery, Kafka, and R2. The architecture centers on Apache Trino as a federated query engine, Apache Iceberg on R2 for cold/warm storage, DataHub for metadata cataloging, and custom services for access control (Lifeguard), PII detection (Skimmer), and ELT orchestration (Transformer). On top of this, they built Skipper, an AI agent that translates natural-language questions into validated SQL queries using five layers of grounded context: schema metadata, human annotations, code-derived knowledge from SQL transformation definitions, curated data model documents, and runtime introspection. Key engineering lessons include: less prescriptive prompting improves LLM accuracy, tool consolidation reduces model confusion, ingesting actual SQL that produces tables captures meaning better than schema alone, and a memory layer enables the agent to improve over time. The platform now handles 91,760 queries from 324 employees, with billing queries accounting for 53% of load. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Cloudflare Details Unified Data Platform Where Billing Workloads Account for 53% of Queries](https://daily.dev/posts/cloudflare-details-unified-data-platform-where-billing-workloads-account-for-53-of-queries-wktgqiouh) · InfoQ · 0 upvotes · 0 comments - [SQL with GenAI: Building an Apache Iceberg lakehouse on Red Hat OpenShift](https://daily.dev/posts/sql-with-genai-building-an-apache-iceberg-lakehouse-on-red-hat-openshift-hjdynkhts) · Red Hat Developer · 1 upvotes · 0 comments - [How leading tech companies are killing the builder’s tax with Lakebase](https://daily.dev/posts/how-leading-tech-companies-are-killing-the-builder-s-tax-with-lakebase-xfo6upqr9) · databricks · 1 upvotes · 0 comments - [Cross-cloud conversational analytics with borderless Lakehouse](https://daily.dev/posts/cross-cloud-conversational-analytics-with-borderless-lakehouse-g6rjhqjqv) · Google Cloud · 0 upvotes · 0 comments - [NexusOne: The Data Layer for AI](https://daily.dev/posts/nexusone-the-data-layer-for-ai-t6pnryasz) · Medium · 0 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#cloudflare](https://daily.dev/tags/cloudflare), [#apache-iceberg](https://daily.dev/tags/apache-iceberg) [View this post on daily.dev](https://daily.dev/posts/how-we-built-cloudflare-s-data-platform-and-an-ai-agent-on-top-of-it-gwui5xgs3) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How we built Cloudflare's data platform and an AI agent on top of it","url":"https://daily.dev/posts/how-we-built-cloudflare-s-data-platform-and-an-ai-agent-on-top-of-it-gwui5xgs3","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-we-built-cloudflare-s-data-platform-and-an-ai-agent-on-top-of-it-gwui5xgs3"},"datePublished":"2026-05-28T13:09:28.480Z","dateModified":"2026-05-28T13:09:52.329Z","description":"Cloudflare built Town Lake, a unified data lakehouse platform, to replace fragmented data systems spanning Postgres, ClickHouse, BigQuery, Kafka, and R2. The...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/3a857e1b143cfea59fa776bc69f13544?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/3a857e1b143cfea59fa776bc69f13544?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Cloudflare","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Cloudflare","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/38522e1d11354cd6b7af66f9d4316735","url":"https://daily.dev/sources/cloudflare"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-we-built-cloudflare-s-data-platform-and-an-ai-agent-on-top-of-it-gwui5xgs3","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents,cloudflare,apache-iceberg","timeRequired":"PT17M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Cloudflare","item":"https://daily.dev/sources/cloudflare"},{"@type":"ListItem","position":3,"name":"How we built Cloudflare's data platform and an AI agent on top of it"}]} ```