--- title: Cloudflare AI Search: give your agents a search engine... description: Cloudflare has announced developer experience improvements to its AI Search product, which now handles crawling, ingestion, embedding, and retrieval... canonical: https://daily.dev/posts/cloudflare-ai-search-give-your-agents-a-search-engine-for-your-data-ctstzert3 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Cloudflare AI Search: give your agents a search engine for your data | daily.dev og:description: Cloudflare has announced developer experience improvements to its AI Search product, which now handles crawling, ingestion, embedding, and retrieval... og:url: https://daily.dev/posts/cloudflare-ai-search-give-your-agents-a-search-engine-for-your-data-ctstzert3 og:image: https://api.daily.dev/og/posts/cTSTzeRT3.png og:image:alt: Cloudflare AI Search: give your agents a search engine for your data og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Cloudflare AI Search: give your agents a search engine for your data **[Cloudflare](https://daily.dev/sources/cloudflare)** · 10 min read · 1 upvotes · 0 comments ## Summary Cloudflare has announced developer experience improvements to its AI Search product, which now handles crawling, ingestion, embedding, and retrieval automatically without manually stitching together Workers AI, Vectorize, R2, and other primitives. New features include sitemap-free website indexing via a 'Discover' parse option, public /search and /mcp endpoints across namespaces, custom domain support for branded endpoints, and an EmDash CMS plugin for semantic search. Cloudflare also previewed pricing: embedding and reranking are free with default Workers AI models, with base ingestion at $0.75/1M tokens and semantic queries at $0.75/1k. The post walks through how Cloudflare built its own Dev Stack MCP server using AI Search to give coding agents up-to-date, cited documentation across 10 Cloudflare-owned surfaces. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Does Cloudflare AI Search now require a sitemap to index a website? No, a sitemap is no longer required. Cloudflare AI Search previously required websites to have a sitemap for the website integration, but now you can select the 'Discover' parsing option, which crawls a site by following links (powered by Browser Run's /crawl) to add a website without a sitemap as a source. _Track platform updates like this so integrating site search into an agent doesn't stall on daily.dev._ ### How much does Cloudflare AI Search cost and is embedding included for free? During the preview pricing phase, base ingestion costs $0.75 per 1M tokens (5M free monthly), image processing adds $0.50 per 1M tokens, storage is $2/GB-month (10GB free), semantic queries cost $0.75 per 1,000 (2,000 free monthly), and full-text queries cost $0.10 per 1,000. Embedding and reranking are free when using default Workers AI models. AI Search is currently free during beta with billing not yet active. _Compare pricing models like this before committing an app's search budget, tracked on daily.dev._ ### How can I let a coding agent query documentation across multiple websites without authentication? Enable public URLs on an AI Search namespace to get a /search and /mcp endpoint that queries every instance in that namespace at once with no authentication required. Alternatively, bind the namespace to a Cloudflare Worker to create a remote MCP server and make a single multi-instance search call across all sources programmatically. _Wiring live docs into coding agents is easier to keep up with on daily.dev._ ## Similar posts on daily.dev - [Cloudflare Extends AI Search to Make it Easier for Agents and Developers to Search Custom Data](https://daily.dev/posts/cloudflare-extends-ai-search-to-make-it-easier-for-agents-and-developers-to-search-custom-data-cp63j3xne) · InfoQ · 0 upvotes · 0 comments - [AI Search: the search primitive for your agents](https://daily.dev/posts/ai-search-the-search-primitive-for-your-agents-esbieslcq) · Cloudflare · 1 upvotes · 0 comments - [Making AI search smarter](https://daily.dev/posts/making-ai-search-smarter-vdumwudvn) · Cloudflare · 0 upvotes · 0 comments - [Power Up Your AI Agent With Live Web Search, for Fewer Tokens](https://daily.dev/posts/power-up-your-ai-agent-with-live-web-search-for-fewer-tokens-tksh6izca) · SitePoint · 1 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#mcp](https://daily.dev/tags/mcp), [#cloudflare](https://daily.dev/tags/cloudflare), [#vector-search](https://daily.dev/tags/vector-search) [View this post on daily.dev](https://daily.dev/posts/cloudflare-ai-search-give-your-agents-a-search-engine-for-your-data-ctstzert3) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Cloudflare AI Search: give your agents a search engine for your data","url":"https://daily.dev/posts/cloudflare-ai-search-give-your-agents-a-search-engine-for-your-data-ctstzert3","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/cloudflare-ai-search-give-your-agents-a-search-engine-for-your-data-ctstzert3"},"datePublished":"2026-08-06T13:03:17.609Z","dateModified":"2026-09-14T09:12:07.244Z","description":"Cloudflare has announced developer experience improvements to its AI Search product, which now handles crawling, ingestion, embedding, and retrieval...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/394c3dde878e6f66c5e703fcef2a6d07?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/394c3dde878e6f66c5e703fcef2a6d07?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Cloudflare","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Cloudflare","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/38522e1d11354cd6b7af66f9d4316735","url":"https://daily.dev/sources/cloudflare"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/cloudflare-ai-search-give-your-agents-a-search-engine-for-your-data-ctstzert3","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents,mcp,cloudflare,vector-search","timeRequired":"PT10M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Cloudflare","item":"https://daily.dev/sources/cloudflare"},{"@type":"ListItem","position":3,"name":"Cloudflare AI Search: give your agents a search engine for your data"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/cloudflare-ai-search-give-your-agents-a-search-engine-for-your-data-ctstzert3#faq","mainEntity":[{"@type":"Question","name":"Does Cloudflare AI Search now require a sitemap to index a website?","acceptedAnswer":{"@type":"Answer","text":"No, a sitemap is no longer required. Cloudflare AI Search previously required websites to have a sitemap for the website integration, but now you can select the 'Discover' parsing option, which crawls a site by following links (powered by Browser Run's /crawl) to add a website without a sitemap as a source. Track platform updates like this so integrating site search into an agent doesn't stall on daily.dev."}},{"@type":"Question","name":"How much does Cloudflare AI Search cost and is embedding included for free?","acceptedAnswer":{"@type":"Answer","text":"During the preview pricing phase, base ingestion costs $0.75 per 1M tokens (5M free monthly), image processing adds $0.50 per 1M tokens, storage is $2/GB-month (10GB free), semantic queries cost $0.75 per 1,000 (2,000 free monthly), and full-text queries cost $0.10 per 1,000. Embedding and reranking are free when using default Workers AI models. AI Search is currently free during beta with billing not yet active. Compare pricing models like this before committing an app's search budget, tracked on daily.dev."}},{"@type":"Question","name":"How can I let a coding agent query documentation across multiple websites without authentication?","acceptedAnswer":{"@type":"Answer","text":"Enable public URLs on an AI Search namespace to get a /search and /mcp endpoint that queries every instance in that namespace at once with no authentication required. Alternatively, bind the namespace to a Cloudflare Worker to create a remote MCP server and make a single multi-instance search call across all sources programmatically. Wiring live docs into coding agents is easier to keep up with on daily.dev."}}]} ```