--- title: Who does Anubis actually stop? | daily.dev description: Anubis is an HTTP proxy that uses proof-of-work challenges to block AI scrapers, but it fails at its stated goal. An AI assistant trivially bypassed it by... canonical: https://daily.dev/posts/who-does-anubis-actually-stop--repxcsjqd twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Who does Anubis actually stop? | daily.dev og:description: Anubis is an HTTP proxy that uses proof-of-work challenges to block AI scrapers, but it fails at its stated goal. An AI assistant trivially bypassed it by... og:url: https://daily.dev/posts/who-does-anubis-actually-stop--repxcsjqd og:image: https://api.daily.dev/og/posts/RePXCSjQd.png og:image:alt: Who does Anubis actually stop? og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Who does Anubis actually stop? **[Farid Zakaria’s Blog](https://daily.dev/sources/fzakaria)** · 4 min read · 0 upvotes · 0 comments ## Summary Anubis is an HTTP proxy that uses proof-of-work challenges to block AI scrapers, but it fails at its stated goal. An AI assistant trivially bypassed it by generating a tool (anubis-fetch) that solves the challenge natively, impersonates Chrome TLS fingerprints, and caches cookies. The real cost falls on humans: legitimate users with slow devices, text browsers, screen readers, and RSS readers are blocked or penalized, while bot farms treat the challenge as a one-time amortized cost. Rough estimates suggest millions of daily solves could waste hundreds to thousands of person-years annually. The author argues Anubis is a regressive tax on the open web that inconveniences humans far more than the AI bots it targets. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community take How the wider developer community reacted, aggregated from 4 discussions and 201 comments across hackernews, lobsters (as of 2026-07-26). **TL;DR:** Commenters broadly agree that Anubis is mischaracterized by the article — its real purpose is anti-DoS against mass scrapers, not blocking individual LLM users — but are divided on whether its proof-of-work mechanism is actually effective long-term versus just security-by-obscurity that will erode as scrapers adapt. **Sentiment:** 25% positive · 45% mixed · 30% skeptical **The case for** - Multiple site operators report dramatic, empirical reductions in scraper load after deploying Anubis (e.g., 600k blocked queries/day, bandwidth costs dropping back to free tier). - Anubis stops the majority of 'dumb' scrapers that don't bother implementing PoW, which is sufficient for most small/personal sites. - It is privacy-preserving compared to alternatives like Cloudflare Turnstile or reCAPTCHA, and can be configured without requiring JavaScript. - The pragmatic case: it solves the problem now and buys time, even if it won't last forever. **The pushback** - Proof-of-work is trivially bypassable — an optimized native solver is orders of magnitude faster than the JS version and can be generated in minutes with an LLM. - Scrapers using residential proxy botnets (smart TVs, etc.) rotate IPs constantly, triggering a new challenge each time, negating the cost-increase argument. - Anubis is already being bypassed at scale for high-value targets like Codeberg, suggesting its effectiveness is eroding. - Widespread adoption creates a 'penicillin effect' — it increases selection pressure on scrapers to adapt, ultimately making things worse for everyone while permanently raising the JS-requirement baseline for the web. - The anime-girl branding is off-putting in professional contexts (e.g., Arch Linux wiki), and the free/commercial licensing model creates awkward situations for open-source projects. - It is fundamentally incompatible with the open web / IndieWeb model (RSS feeds, no-JS users, Tor users). **By community** - hackernews (mixed): HN commenters largely correct the article's framing (Anubis targets mass scrapers, not LLM agents) but are split between pragmatists citing real-world effectiveness and skeptics arguing PoW is theoretically and practically weak. - lobsters (mixed): Lobsters discussion is richer and more heated, with strong empirical testimonials from site operators alongside a contentious sub-thread about the ethics of scraping itself and a meta-debate over whether the vibecoding tag was wrongly removed. **Hottest debate:** Whether Anubis' proof-of-work actually increases scraping costs meaningfully, or whether it only works due to obscurity and will become useless as scrapers adapt — with empirical operator reports clashing against theoretical and real-world bypass evidence. **Open questions** - Is there a viable anti-scraping solution that is compatible with the open web (RSS, no-JS, Tor) rather than forcing a JavaScript requirement? - At what point of adoption does Anubis lose effectiveness entirely, and what comes next? - Should robots.txt be considered binding network etiquette, or merely a voluntary convention that scrapers may ignore for archival/research purposes? **Highlights** > Anubis does not filter bots, it rate-limits clients. > — [icefox on lobsters · 42 points](https://lobste.rs/s/ktew3s/who_does_anubis_actually_stop#c_cxeimo) > Anubis is the penicillin of web hosting. Every individual practitioner has a strong incentive to overuse it, because it's extremely effective for them individually. Every additional practitioner that uses it increases the selection pressure on their collective adversary to develop resistance. Eventually, it reaches a tipping point and becomes ineffective. But the ecosystem-level impact of its historical use remains, and makes things worse for everyone. Recently, the baseline expectations of the Web have shifted, and I need to enable JavaScript to read a non-CloudFlare-proxied simple HTML site like LKML. I do not expect this shift will revert once Anubis outlives its effectiveness. > — [greyface- on hackernews · 2 comments](https://news.ycombinator.com/item?id=49052732) > > I'm pretty sure the owners of the websites I'm scraping are not happy about that either. But that's not really my problem. And that attitude is why we have this problem in the first place. > — [spc476 on lobsters · 28 points, 1 comments](https://lobste.rs/s/ktew3s/who_does_anubis_actually_stop#c_aosn7q) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49051505) · 25 points · 105 comments - [hackernews](https://news.ycombinator.com/item?id=48880808) · 19 points · 6 comments - [lobsters](https://lobste.rs/s/ktew3s/who_does_anubis_actually_stop) · 69 points · 86 comments - [hackernews](https://news.ycombinator.com/item?id=48852762) · 9 points · 4 comments ## Similar posts on daily.dev - [Making sure you're not a bot\!](https://daily.dev/posts/making-sure-you-re-not-a-bot--zsawpz3gm) · Lobsters · 0 upvotes · 0 comments - [you don't need anubis](https://daily.dev/posts/you-don-t-need-anubis-8u7ltjtc6) · Hacker News · 3 upvotes · 0 comments - [An update on the scraper situation](https://daily.dev/posts/an-update-on-the-scraper-situation-f9grknbuz) · Hacker News · 0 upvotes · 0 comments - [It took a year to ship WebAssembly in Anubis](https://daily.dev/posts/it-took-a-year-to-ship-webassembly-in-anubis-efjdlvpkj) · Lobsters · 0 upvotes · 2 comments - [Publishers say no to AI scrapers, block bots at server level](https://daily.dev/posts/publishers-say-no-to-ai-scrapers-block-bots-at-server-level-nite7hp90) · The Register · 1 upvotes · 0 comments --- Tags: [#crawling](https://daily.dev/tags/crawling), [#proof-of-work](https://daily.dev/tags/proof-of-work) [View this post on daily.dev](https://daily.dev/posts/who-does-anubis-actually-stop--repxcsjqd) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Who does Anubis actually stop?","url":"https://daily.dev/posts/who-does-anubis-actually-stop--repxcsjqd","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/who-does-anubis-actually-stop--repxcsjqd"},"datePublished":"2026-07-09T21:36:47.340Z","dateModified":"2026-07-26T15:10:41.950Z","description":"Anubis is an HTTP proxy that uses proof-of-work challenges to block AI scrapers, but it fails at its stated goal. An AI assistant trivially bypassed it by...","image":"https://media.daily.dev/image/upload/s--58gMhC4P--/f_auto/v1722860399/public/Placeholder%2012","thumbnailUrl":"https://media.daily.dev/image/upload/s--58gMhC4P--/f_auto/v1722860399/public/Placeholder%2012","isAccessibleForFree":true,"articleSection":"Farid Zakaria’s Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Farid Zakaria’s Blog","logo":"https://media.daily.dev/image/upload/s--YqqvRt90--/f_auto/v1723789280/logos/fzakaria","url":"https://daily.dev/sources/fzakaria"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/who-does-anubis-actually-stop--repxcsjqd","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"crawling,proof-of-work","timeRequired":"PT4M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Farid Zakaria’s Blog","item":"https://daily.dev/sources/fzakaria"},{"@type":"ListItem","position":3,"name":"Who does Anubis actually stop?"}]} ```