--- title: "Say it once: introducing Bot Preference SynC" url: https://daily.dev/posts/say-it-once-introducing-bot-preference-sync-chjkgyjqr source_url: https://blog.cloudflare.com/bot-preference-sync%20 type: article source: "Cloudflare" published: 2026-08-21T23:24:29.069Z updated: 2026-08-21T23:27:17.194Z tags: ["seo", "cloudflare"] reading_time: 8 upvotes: 1 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Say it once: introducing Bot Preference SynC **[Cloudflare](https://daily.dev/sources/cloudflare)** · 8 min read · 1 upvotes · 0 comments ## Summary Cloudflare launched Bot Preference Sync, a feature that automatically updates a site's robots.txt file to match the AI bot policies (Search, Agent, Training) configured in the Cloudflare dashboard, eliminating the need to maintain a separate static file. The feature is rolling out to all customers, free through enterprise, in the coming week, and will be on by default for new signups. Cloudflare also tightened its Transparency requirements: mixed-use crawlers that both search and train must now provide additional disclosures (respecting no-training preferences, offering opt-outs from AI summaries, providing URL-level visibility, and proving that disallowing training doesn't hurt search rankings) to avoid being blocked when Disallow Training is set. Publishers relying on ad revenue get a new default of Disallow Training while staying searchable. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is Cloudflare Bot Preference Sync and how does it update robots.txt? Bot Preference Sync is a Cloudflare feature that automatically writes your configured AI bot preferences for Search, Agent, and Training categories into your site's robots.txt file, prepending the generated directives while preserving any existing Disallow rules. It removes the need to manually maintain a static robots.txt, keeping stated preferences and edge-enforced blocks aligned. It rolls out to all plans, from Free to Enterprise, and is on by default for new customers. _daily.dev helps site owners track platform changes like this before they affect crawler access._ ### What extra transparency requirements does Cloudflare impose on mixed-use AI crawlers that do both search and training? Bots that perform both search and training must respect a no-training preference in robots.txt, let site owners opt out of AI summaries, provide URL-level visibility into which pages were used for training plus search metrics, and publicly demonstrate that disallowing training doesn't hurt search rankings. Crawlers that skip these requirements are blocked outright whenever a site sets Disallow Training, with no benefit of the doubt given. _Track evolving crawler transparency rules on daily.dev before they change your site's AI visibility._ ### What robots.txt default does Cloudflare set for ad-supported publisher sites versus other site owners? Ad-supported or publisher sites that select the option indicating they monetize pages with ads get Training set to Disallow by default, keeping their content out of AI model training while remaining in search indexes so they still receive referral traffic. Other new customers get no blocks or disallows by default for Search, Agent, or Training when onboarding a domain; the choice is left entirely to the customer. _Publishers weighing AI training exposure against search visibility can follow this shift on daily.dev._ ## Similar posts on daily.dev - [Your site, your rules: new AI traffic options for all customers](https://daily.dev/posts/your-site-your-rules-new-ai-traffic-options-for-all-customers-zrn0fx6z7) · Cloudflare · 0 upvotes · 0 comments - [Cloudflare gives AI crawlers a September deadline to pay up](https://daily.dev/posts/cloudflare-gives-ai-crawlers-a-september-deadline-to-pay-up-s5ncxffs7) · The Next Web · 0 upvotes · 0 comments - [Giving users choice with Cloudflare’s new Content Signals Policy](https://daily.dev/posts/giving-users-choice-with-cloudflare-s-new-content-signals-policy-dhuovwbvp) · Cloudflare · 1 upvotes · 0 comments - [Every AI Crawler in 2026: The Reference Table \(And Which Ones Your WAF Is Silently Blocking\)](https://daily.dev/posts/every-ai-crawler-in-2026-the-reference-table-and-which-ones-your-waf-is-silently-blocking--ujfhbfivn) · Security Boulevard · 1 upvotes · 1 comments - [Redirects for AI Training enforces canonical content](https://daily.dev/posts/redirects-for-ai-training-enforces-canonical-content-tvi2dnkcy) · Cloudflare · 0 upvotes · 0 comments --- Tags: [#seo](https://daily.dev/tags/seo), [#cloudflare](https://daily.dev/tags/cloudflare) [View this post on daily.dev](https://daily.dev/posts/say-it-once-introducing-bot-preference-sync-chjkgyjqr)