Time magazine is serving a completely different document to AI agents (ClaudeBot, GPTBot, PerplexityBot, etc.) than to human browsers — a clean Markdown version with embedded ads injected by ad-tech partner Mobian. The mechanism is user-agent detection at the CDN edge (Fastly), not content negotiation or URL patterns. The ad units are structured knowledge objects engineered for retrieval, containing FAQ blocks, schema.org JSON-LD, and brand assertions designed to be ingested as cited facts. On ad-bearing pages, the advertiser content makes up 41–70% of the document, positioned before the journalism. A key finding: the stated llms.txt policy disallows Anthropic crawlers, but the edge actively serves them markdown and ads — a direct contradiction. The author also flags that this is textbook cloaking, that disclosure labels don't survive chunking in retrieval systems, and that the only way to audit this behavior is to impersonate crawlers and diff responses across multiple HTTP clients.

8m read timeFrom apievangelist.com
Post cover image
202 Impressions