--- title: What happened to TheNumbers.com | daily.dev description: TheNumbers.com, a 29-year-old film industry data site with 8 million annual visitors, went offline in March 2026 after being overwhelmed by AI bot traffic and... canonical: https://daily.dev/posts/what-happened-to-thenumbers-com-iqgmwf9yo twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: What happened to TheNumbers.com | daily.dev og:description: TheNumbers.com, a 29-year-old film industry data site with 8 million annual visitors, went offline in March 2026 after being overwhelmed by AI bot traffic and... og:url: https://daily.dev/posts/what-happened-to-thenumbers-com-iqgmwf9yo og:image: https://api.daily.dev/og/posts/IQGMwf9yO.png og:image:alt: What happened to TheNumbers.com og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # What happened to TheNumbers.com **[Hacker News](https://daily.dev/sources/hn)** · 15 min read · 0 upvotes · 0 comments ## Summary TheNumbers.com, a 29-year-old film industry data site with 8 million annual visitors, went offline in March 2026 after being overwhelmed by AI bot traffic and what appears to be a targeted intrusion. By early 2026, only 10% of its traffic was human. The site was likely targeted because prediction markets like Polymarket use its data as the official resolution source, giving early access financial value. The team rebuilt on new infrastructure rather than restore compromised legacy files. The incident illustrates three broader trends: AI agents have made web scraping and hacking accessible to non-experts, the economics of the open web (crawl-for-traffic exchange) have broken down, and small independent sites running on old infrastructure are acutely vulnerable. Examples from Read the Docs, iFixit, Wikipedia, and GNOME show the problem is widespread. Cloudflare has launched pay-per-crawl and pay-per-use models as potential mitigations, but their effectiveness depends on AI companies' compliance. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community take How the wider developer community reacted, aggregated from 2 discussions and 133 comments across hackernews (as of 2026-07-25). **TL;DR:** The community is largely critical of AI scrapers destroying free, public-interest websites through unsustainable bandwidth costs, with many commenters sharing personal experiences of the same problem; there is also a heated side debate about whether AI trivializing creative work is a net loss for hobbyists and the open web. **Sentiment:** 10% positive · 35% mixed · 55% skeptical **The case for** - Commenters note that robots.txt disallow rules for major AI crawlers (ClaudeBot, GPTBot, etc.) do offer some opt-out control for site owners. - Some suggest static site generators plus a bot-aware CDN (e.g., Cloudflare) could dramatically reduce hosting costs and bot impact. - A few commenters welcome AI training on their open-source work, seeing it as a way to improve models for everyone. **The pushback** - AI scrapers ignore bulk-download options and hammer dynamic search endpoints, generating terabytes of traffic even when full data exports are freely available. - The scale of AI scraping (entire data-center fleets vs. a few residential proxies) makes the problem qualitatively different from historical scraping. - Prediction market financial incentives create a separate, powerful motive for malicious actors to probe and exploit sites for early data access. - 90% of bot traffic now uses headless Chrome via residential IPs, making robots.txt and IP-based blocking largely ineffective. - AI scraping reduces human traffic and ad/licensing revenue while generating no return visits, undermining the economic viability of free public-interest sites. - Many hobbyist creators say the knowledge that their work feeds commercial AI training—without compensation—is killing their motivation to publish freely. - AWS and cloud providers still offer no hard spending caps, leaving hobbyist operators exposed to runaway bills. **By community** - hackernews (skeptical): Commenters are broadly critical of AI scraper behavior and its economic harm to small sites, while debating whether technical mitigations (CDN, static hosting, robots.txt) are adequate and lamenting the chilling effect on open-web publishing. **Hottest debate:** Whether AI trivializing hobbyist coding and creative work is a net benefit (freeing time) or a genuine theft of joy and motivation, with CamperBob2 and several others in direct, sustained disagreement. **Open questions** - Is there any realistic infrastructure-level mechanism to make bots pay for the traffic they generate, or to enforce penalties for ignoring robots.txt? - Why do frontier AI labs' crawlers still hammer sites that already offer bulk data downloads, and who is accountable for third-party 'vibe-coded' scrapers? - Will the end state of the open web simply be everything behind paywalls, and what happens to hobbyist and public-interest publishing then? - Can Cloudflare or similar services reliably distinguish residential-IP bot traffic from genuine human visitors? **Highlights** > A couple years ago, I was running a website which allowed the public to view all the US government handouts to small businesses during the COVID-19 pandemic. It also tracked all the fraudulent loans being prosecuted by the DOJ, and allowed anyone to run structured queries over the public dataset. There was a "donate" button which took in ~$2k in donations over the lifetime of the site, and you could download the entire underlying dataset (around 10 GB uncompressed) for free directly on the site. Despite the "download all data" link being prominently placed on the front page, the AI scrapers decided it would be more efficient to download terabytes upon terabytes of raw HTML by paginating through every possible facet on the search endpoint. Even with CloudFront caching results and a fairly efficient backend setup, the monthly bill ended up with around $1k just going toward network ingress/egress, so I shut down the site the following month. > — [primitivesuave on hackernews · 4 comments](https://news.ycombinator.com/item?id=49026105) > I think people are missing one of the points of the article. It's not just that agents are hammering the site, it's that there might be lurking vulnerabilities that allow malicious usage, which is why it went down, then came back up with a fraction of the data and a reduced design. The article says (speculates?) that malicious users are trying to get privileged access for an edge in prediction market betting. From the article: > If you could see The Numbers data before everyone else, every single week, you would have a significant edge over all the other traders - learning the answers slightly ahead of publication would allow you to front-run the trades. > — [abetusk on hackernews](https://news.ycombinator.com/item?id=49025489) > > Now you can do something with your time besides writing pointless one-off doo-dads Writing software one-offs to scratch an itch was historically one of my most enjoyable past-times. AI trivializing that has been a very real theft of joy in my life. Solving the actual problem was never the point, it was just motivation to do geek-out and craft some code. AI is rapidly diminishing many interesting hobbies (coding, art, music, writing). Having more free time when there's nothing fun nor exciting to do with it isn't really a benefit. > — [jtuple on hackernews](https://news.ycombinator.com/item?id=49026740) > Why would you produce content and have no one read it or visits your site, but OpenAI and Anthropic make millions from it? At some point it becomes stupid to just give free money to these companies when they steal literally all your traffic and content. As per the article, Anthropic sends 1 page view for every 38,000 views they get. > — [freediddy on hackernews](https://news.ycombinator.com/item?id=49028219) > Because the crawlers would still have hammered their site, though. (The GP post doesn’t actually meaningfully address the issue being raised. Adding BigQuery or whatever would not change the fact that (a) they already offered a method of getting all of the data in a cost effective way, and (b) the issue was that the crawlers hammered the site hard enough to make it economically unviable.) > — [taneq on hackernews](https://news.ycombinator.com/item?id=49027802) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49024691) · 118 points · 133 comments - [hackernews](https://news.ycombinator.com/item?id=49017026) · 2 points · 0 comments ## Similar posts on daily.dev - [How Film Industry Data Website The-Numbers.com Got Mauled By Bots](https://daily.dev/posts/how-film-industry-data-website-the-numbers-com-got-mauled-by-bots-ec9bnxq71) · Hackaday · 0 upvotes · 0 comments - [Dead internet theory becomes measurable fact as AI agents flood the web](https://daily.dev/posts/dead-internet-theory-becomes-measurable-fact-as-ai-agents-flood-the-web-4uwxc0w1l) · The Next Web · 1 upvotes · 0 comments - [The web belongs to the machines now](https://daily.dev/posts/the-web-belongs-to-the-machines-now-oylvpdhmh) · TechCentral · 0 upvotes · 0 comments - [Publishers say no to AI scrapers, block bots at server level](https://daily.dev/posts/publishers-say-no-to-ai-scrapers-block-bots-at-server-level-nite7hp90) · The Register · 1 upvotes · 0 comments - [Online bot traffic will exceed human traffic by 2027, Cloudflare CEO says](https://daily.dev/posts/online-bot-traffic-will-exceed-human-traffic-by-2027-cloudflare-ceo-says-u2nxo9i7j) · TechCrunch · 0 upvotes · 0 comments --- Tags: [#cyber](https://daily.dev/tags/cyber), [#cloudflare](https://daily.dev/tags/cloudflare), [#crawling](https://daily.dev/tags/crawling), [#agentic-ai](https://daily.dev/tags/agentic-ai) [View this post on daily.dev](https://daily.dev/posts/what-happened-to-thenumbers-com-iqgmwf9yo) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"What happened to TheNumbers.com","url":"https://daily.dev/posts/what-happened-to-thenumbers-com-iqgmwf9yo","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/what-happened-to-thenumbers-com-iqgmwf9yo"},"datePublished":"2026-07-23T17:40:33.534Z","dateModified":"2026-07-25T00:24:40.827Z","description":"TheNumbers.com, a 29-year-old film industry data site with 8 million annual visitors, went offline in March 2026 after being overwhelmed by AI bot traffic and...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/644634bbfeb458e2ca4000d7eb76d79b?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/644634bbfeb458e2ca4000d7eb76d79b?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/what-happened-to-thenumbers-com-iqgmwf9yo","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"cyber,cloudflare,crawling,agentic-ai","timeRequired":"PT15M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"What happened to TheNumbers.com"}]} ```