--- title: Making it easier for websites to express AI training... description: OpenAI will honor a shared robots.txt signal letting website operators state their AI training preferences once for multiple providers. Previously, operators... canonical: https://daily.dev/posts/making-it-easier-for-websites-to-express-ai-training-preferences-ycc0xghgx twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Making it easier for websites to express AI training preferences | daily.dev og:description: OpenAI will honor a shared robots.txt signal letting website operators state their AI training preferences once for multiple providers. Previously, operators... og:url: https://daily.dev/posts/making-it-easier-for-websites-to-express-ai-training-preferences-ycc0xghgx og:image: https://api.daily.dev/og/posts/YcC0xgHgx.png og:image:alt: Making it easier for websites to express AI training preferences og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Making it easier for websites to express AI training preferences **[OpenAI](https://daily.dev/sources/openai)** · 2 min read · 0 upvotes · 0 comments ## Summary OpenAI will honor a shared robots.txt signal letting website operators state their AI training preferences once for multiple providers. Previously, operators had to identify individual crawlers and set separate rules for each provider. The signal draws on the ongoing work of the IETF AI Preferences working group and will roll out in the coming months, with implementation details to follow. It won't replace OpenAI's existing crawler controls, which stay in effect. OpenAI is encouraging other AI providers to honor the same signal. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How can I block or allow AI training crawlers across multiple companies with one robots.txt rule? OpenAI will honor a shared robots.txt signal, developed through the IETF AI Preferences working group, that lets website operators express AI training preferences once for multiple participating providers instead of listing rules for each crawler separately. OpenAI plans to implement it in the coming months and says it will not replace its existing crawler controls, which remain in effect. _Publishers weighing crawler access policy can follow standards shifts like this on daily.dev._ ### Will OpenAI's new robots.txt AI training signal replace its existing crawler controls? No, the shared robots.txt signal for AI training preferences will not replace OpenAI's existing crawler controls. Those existing controls, which give web publishers flexibility over their pages, continue to apply and be respected alongside the new shared signal once it is implemented. _Anyone configuring crawler rules can track changes like this robots.txt update on daily.dev._ ## Similar posts on daily.dev - [Say it once: introducing Bot Preference SynC](https://daily.dev/posts/say-it-once-introducing-bot-preference-sync-chjkgyjqr) · Cloudflare · 1 upvotes · 0 comments - [robots.txt in the age of AI crawlers](https://daily.dev/posts/robots-txt-in-the-age-of-ai-crawlers-nnwkbp4ee) · Flavio Copes · 1 upvotes · 1 comments - [Cloudflare Debuts Markdown for Agents and Content Signals to Guide AI Crawlers](https://daily.dev/posts/cloudflare-debuts-markdown-for-agents-and-content-signals-to-guide-ai-crawlers-xrnlmy5s5) · InfoQ · 1 upvotes · 0 comments - [Your site, your rules: new AI traffic options for all customers](https://daily.dev/posts/your-site-your-rules-new-ai-traffic-options-for-all-customers-zrn0fx6z7) · Cloudflare · 0 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#openai](https://daily.dev/tags/openai) [View this post on daily.dev](https://daily.dev/posts/making-it-easier-for-websites-to-express-ai-training-preferences-ycc0xghgx) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Making it easier for websites to express AI training preferences","url":"https://daily.dev/posts/making-it-easier-for-websites-to-express-ai-training-preferences-ycc0xghgx","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/making-it-easier-for-websites-to-express-ai-training-preferences-ycc0xghgx"},"datePublished":"2026-10-30T18:18:48.376Z","dateModified":"2026-10-30T18:19:08.850Z","description":"OpenAI will honor a shared robots.txt signal letting website operators state their AI training preferences once for multiple providers. Previously, operators...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/9145deb268c870d9b6ce8d233a9aff5a?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/9145deb268c870d9b6ce8d233a9aff5a?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"OpenAI","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"OpenAI","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/984b3632316f4afaa3d5213d5c64fe65","url":"https://daily.dev/sources/openai"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/making-it-easier-for-websites-to-express-ai-training-preferences-ycc0xghgx","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,openai","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"OpenAI","item":"https://daily.dev/sources/openai"},{"@type":"ListItem","position":3,"name":"Making it easier for websites to express AI training preferences"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/making-it-easier-for-websites-to-express-ai-training-preferences-ycc0xghgx#faq","mainEntity":[{"@type":"Question","name":"How can I block or allow AI training crawlers across multiple companies with one robots.txt rule?","acceptedAnswer":{"@type":"Answer","text":"OpenAI will honor a shared robots.txt signal, developed through the IETF AI Preferences working group, that lets website operators express AI training preferences once for multiple participating providers instead of listing rules for each crawler separately. OpenAI plans to implement it in the coming months and says it will not replace its existing crawler controls, which remain in effect. Publishers weighing crawler access policy can follow standards shifts like this on daily.dev."}},{"@type":"Question","name":"Will OpenAI's new robots.txt AI training signal replace its existing crawler controls?","acceptedAnswer":{"@type":"Answer","text":"No, the shared robots.txt signal for AI training preferences will not replace OpenAI's existing crawler controls. Those existing controls, which give web publishers flexibility over their pages, continue to apply and be respected alongside the new shared signal once it is implemented. Anyone configuring crawler rules can track changes like this robots.txt update on daily.dev."}}]} ```