A personal site operator explains why visitors sometimes hit a block page: an anti-crawler system is filtering out requests using old or suspicious browser User-Agent strings, mainly older Chrome identifiers, because of a surge in high-volume crawlers (some apparently scraping data for LLM training). The explanation also covers side effects, such as Inoreader and Feedly mistakenly triggering the block by testing feeds with fake old-browser headers, Vivaldi users needing to enable 'User Agent Brand Masking' to avoid being misidentified as Chrome, and archive.today/archive.ph crawlers being indistinguishable from malicious bots (with archive.org suggested as a better-behaved alternative).
Table of contents
A special note to people using Inoreader (the feed reader)A special note to people using Feedly (the feed reader)A special note for people using VivaldiA special note for people using archive.*Questions this post answers
Why does Vivaldi get blocked by sites that filter out old Chrome browsers?
Vivaldi identifies itself using a Chrome-based User-Agent by default, so anti-crawler filters that block old Chrome user agents can mistakenly block Vivaldi too. Enabling the 'User Agent Brand Masking' setting in Vivaldi makes it report itself as Vivaldi instead of Google Chrome, which avoids being caught by this filtering, even on the current version of the browser. daily.dev surfaces practical browser-configuration fixes like this for developers running into odd site-blocking behavior.
Why do feed readers like Feedly and Inoreader sometimes get blocked as bots when fetching RSS feeds?
Feed readers can periodically probe feeds using a fake, old-browser HTTP User-Agent header, and when that fetch fails or returns a block page, they sometimes mistakenly latch onto that error response instead of using their real feed-fetcher User-Agent's results. This causes subscribers to see an anti-crawler block page instead of the actual feed content, even though the reader's normal fetcher isn't blocked. daily.dev helps developers troubleshoot flaky feed-fetching and User-Agent quirks like these before they ship integrations.