--- title: Zombie stats: Where the numbers AI quotes come from description: Only about 40% of statistics cited by AI assistants trace back to a study that actually says the same thing. Researchers checked 1,285 statistics quoted by... canonical: https://daily.dev/posts/zombie-stats-where-the-numbers-ai-quotes-come-from-xuj9hsgrl twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Zombie stats: Where the numbers AI quotes come from | daily.dev og:description: Only about 40% of statistics cited by AI assistants trace back to a study that actually says the same thing. Researchers checked 1,285 statistics quoted by... og:url: https://daily.dev/posts/zombie-stats-where-the-numbers-ai-quotes-come-from-xuj9hsgrl og:image: https://api.daily.dev/og/posts/XUJ9Hsgrl.png og:image:alt: Zombie stats: Where the numbers AI quotes come from og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Zombie stats: Where the numbers AI quotes come from **[Hacker News](https://daily.dev/sources/hn)** · 7 min read · 0 upvotes · 0 comments ## Summary Only about 40% of statistics cited by AI assistants trace back to a study that actually says the same thing. Researchers checked 1,285 statistics quoted by ChatGPT, Perplexity, Gemini, Claude and Google AI Overviews across 200 number-based questions, following each citation up to five pages deep. Another 8% led to a study whose number or context had changed along the way, while 40% dead-ended at an inaccessible source and 11% led nowhere. ChatGPT quoted the fewest numbers per answer (1.3); Claude quoted the most (6.8). Even when all five assistants agreed on a figure, only 2 of 5 traced trails were actually sourced. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What percentage of statistics cited by ChatGPT, Gemini, Claude and other AI assistants actually trace back to a real study? About 40% of statistics cited by AI assistants trace back to a study that reports the same figure. Another 8% reach a study but with the number or context changed. Roughly 40% dead-end at a source that is locked, gone, or silent on the number, and 11% lead nowhere. The test covered 1,285 statistics from ChatGPT, Perplexity, Gemini, Claude and Google AI Overviews. _Anyone fact-checking AI output can track findings like this on daily.dev before citing a number._ ### Which AI assistant is most accurate when citing statistics, ChatGPT or Claude? ChatGPT performed best among five assistants tested and also quoted far fewer numbers per answer, 1.3 compared to Claude's 6.8. The other three assistants, Perplexity, Gemini and Google AI Overviews, landed between 38% and 46% sourced accuracy, too close together to rank meaningfully against each other. _Developers comparing AI assistants for factual reliability can follow this kind of benchmark on daily.dev._ ### Does it mean a statistic is accurate if multiple AI assistants give the same number? No, agreement across AI assistants does not mean a statistic is properly sourced. When all five assistants gave the identical figure on 10 questions, only 2 of the 5 traced trails actually led to a real study. Statistics repeated by three or more assistants did no better, with only 11 of 29 properly sourced, and four were cited by all five with no study behind them at all. _Teams relying on AI-generated figures can avoid false confidence by tracking verification practices on daily.dev._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 58 comments across hackernews (as of 2026-10-29). **TL;DR:** Commenters largely see this as confirming what they already suspected: AI citations are often unreliable or unverifiable, with some pointing out that the report itself reads like AI-generated content and that humans aren't much better at sourcing claims. **Sentiment:** 5% positive · 25% mixed · 70% skeptical **The case for** - Some note that well-prompted LLMs with proper tooling (RAG, provenance tracking) could actually solve the sourcing problem, so it's a fixable product failure rather than an AI limitation. - A few see value in AI forcing people to build critical-thinking skills around verifying claims, similar to evaluating any human-written source. **The pushback** - Many report firsthand experience of citations leading to sources that don't actually support the claim, or to AI-generated blogspam being cited as if reliable. - Several argue people falsely assume AI answers are well-researched and vetted, unlike traditional software, which makes the problem worse. - Some call the report itself low-quality or likely AI-written/thinly veiled marketing (for a product called trakkr), undermining its credibility. - A worry that future source material itself will increasingly be AI-generated, making the verification problem recursive and worse over time. - One thread shows a widely-cited '70% of websites use PHP' statistic may be outdated or methodologically flawed, illustrating how ungrounded numbers propagate anyway. **By community** - hackernews (mixed): Discussion ranges from distrust of AI citation practices and skepticism toward the report itself, to practical discussion of RAG/provenance tooling as potential fixes, with humor about human unreliability too. **Hottest debate:** Whether unreliable sourcing is an inherent flaw of AI systems or simply a failure of poorly built products that could be fixed with better tooling and prompting. **Open questions** - What concrete tooling or standards could reliably track provenance from AI-generated claims back to verified sources? - How should users be made aware that AI answers aren't as rigorously verified as they assume? **Highlights** > I think it's more that the majority of humans DON'T REALIZE it's NOT doing that. Most people think AI answers are carefully sourced, triple vetted, over-researched to death and can trust it with their life, despite the warnings, until they run into nonsense and then dismiss it as a one-off. If there was a "always give the best possible, most researched and trusted answer" flag that was easily visible, or maybe "chatGpt" and "bestAnswersGpt", where the second one did all the verification and was considered much better overall, guess which one people would start using after a while and/or quote, because of improved accuracy? It's not that people don't WANT honest information (well, most people) - they just don't KNOW they're not getting it right now. That's the problem with tech - they're trading on "computers were always right in the past" because there has been no generally available "fuzzy search" type application that needs handholding to give "good" results. Most software either gave the result of computations that are always the same, or enforced static rules (like spellcheck) which were perceived as "always right". > — [hn\_acc1 on hackernews · 1 comments](https://news.ycombinator.com/item?id=50293264) > This article is complete slop and not worth looking at, but it's an issue worth discussing. One of the original php2shell announcement posts (which was clearly AI written) claimed that 70% of websites use PHP. This number has now been repeated by every single article about php2shell, including in the most trusted tech news outlets. I think it could have been approximately true in 2008, but not in 2026. The number is too round anyway. Trying to search for a source to the claim is now impossible. If the basis of a reliable fact is consensus, then every AI model should now consider this fact to be true. In reality consensus is not useful of course, offhand facts are frequently generated by citogenesis. An article mentioning a stat offhand is not a source. A source should take some amount of epistemic responsibility for a claim. > — [rozab on hackernews · 1 comments](https://news.ycombinator.com/item?id=50292078) > Looking at HTTP Archive's September 2026 crawl[1], PHP shows up on 4.22M of 8.74M mobile origins, which is 48.3%. My guess is the 70% figure comes from W3Techs[2], which currently puts PHP at 69.2%, but only of sites "whose server-side programming language we know." Given that PHP is uniquely trivial to fingerprint (via WordPress markup, .php paths, PHPSESSID cookies, etc), it is very likely that it is overrepresented in their dataset[3]. [1] https://httparchive.org/ [2] https://w3techs.com/technologies/overview/programming_langua... [3] https://en.wikipedia.org/wiki/Streetlight_effect > — [cosmojg on hackernews · 1 comments](https://news.ycombinator.com/item?id=50292498) > yeah i work on this at https://groundedai.company/ we build a product called Veracity focused on citation and claim checking. legal is a major focus for us, also scientific and technical publishing, where we detect hallucinated citations in submitted works and all that jazz planning to release something soon focused on the LLM harness layer we've been working on the problem for years. gradually getting traction all completely closed source atm, though we often revisit that discussion i would contest a sibling comment - this is not RAG. 1) because it doesn't necessarily involve 'generation' (e.g our new grounding model doesn't depend on gen ai) and 2) RAG is just enriching the context. even if you can get the retrieval right (not a given) it doesn't guarentee that the LLM will faithfully or correctly use that context, or that all its claims are supported by sources > — [cyclecycle on hackernews](https://news.ycombinator.com/item?id=50294857) > A full 40% of AI stats actually have a specific study behind them? Knowing how the human baseline is that 73.6%* of all statistics are made up on the spot, this is really encouraging. *That particular stat was of course made up, but not by me. > — [falcor84 on hackernews](https://news.ycombinator.com/item?id=50290928) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=50290283) · 98 points · 58 comments ## Similar posts on daily.dev - [Brits believe the bots even though they spout nonsense](https://daily.dev/posts/brits-believe-the-bots-even-though-they-spout-nonsense-xvvh71pq4) · The Register · 1 upvotes · 0 comments - [Publish fake stats, get cited in LLMs](https://daily.dev/posts/publish-fake-stats-get-cited-in-llms-0vhqscrf9) · Hacker News · 2 upvotes · 0 comments - [Your brain on AI](https://daily.dev/posts/your-brain-on-ai-2xpkp11jx) · MIT Technology Review · 2 upvotes · 0 comments - [Study: AI chatbots provide less-accurate information to vulnerable users](https://daily.dev/posts/study-ai-chatbots-provide-less-accurate-information-to-vulnerable-users-8nuiign7m) · MIT News · 1 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#chatgpt](https://daily.dev/tags/chatgpt) [View this post on daily.dev](https://daily.dev/posts/zombie-stats-where-the-numbers-ai-quotes-come-from-xuj9hsgrl) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Zombie stats: Where the numbers AI quotes come from","url":"https://daily.dev/posts/zombie-stats-where-the-numbers-ai-quotes-come-from-xuj9hsgrl","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/zombie-stats-where-the-numbers-ai-quotes-come-from-xuj9hsgrl"},"datePublished":"2026-10-29T16:53:16.421Z","dateModified":"2026-10-29T19:56:02.076Z","description":"Only about 40% of statistics cited by AI assistants trace back to a study that actually says the same thing. Researchers checked 1,285 statistics quoted by...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/70122ff3bdb4ad15164a7342a67de0b0?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/70122ff3bdb4ad15164a7342a67de0b0?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/zombie-stats-where-the-numbers-ai-quotes-come-from-xuj9hsgrl","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,chatgpt","timeRequired":"PT7M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"Zombie stats: Where the numbers AI quotes come from"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/zombie-stats-where-the-numbers-ai-quotes-come-from-xuj9hsgrl#faq","mainEntity":[{"@type":"Question","name":"What percentage of statistics cited by ChatGPT, Gemini, Claude and other AI assistants actually trace back to a real study?","acceptedAnswer":{"@type":"Answer","text":"About 40% of statistics cited by AI assistants trace back to a study that reports the same figure. Another 8% reach a study but with the number or context changed. Roughly 40% dead-end at a source that is locked, gone, or silent on the number, and 11% lead nowhere. The test covered 1,285 statistics from ChatGPT, Perplexity, Gemini, Claude and Google AI Overviews. Anyone fact-checking AI output can track findings like this on daily.dev before citing a number."}},{"@type":"Question","name":"Which AI assistant is most accurate when citing statistics, ChatGPT or Claude?","acceptedAnswer":{"@type":"Answer","text":"ChatGPT performed best among five assistants tested and also quoted far fewer numbers per answer, 1.3 compared to Claude's 6.8. The other three assistants, Perplexity, Gemini and Google AI Overviews, landed between 38% and 46% sourced accuracy, too close together to rank meaningfully against each other. Developers comparing AI assistants for factual reliability can follow this kind of benchmark on daily.dev."}},{"@type":"Question","name":"Does it mean a statistic is accurate if multiple AI assistants give the same number?","acceptedAnswer":{"@type":"Answer","text":"No, agreement across AI assistants does not mean a statistic is properly sourced. When all five assistants gave the identical figure on 10 questions, only 2 of the 5 traced trails actually led to a real study. Statistics repeated by three or more assistants did no better, with only 11 of 29 properly sourced, and four were cited by all five with no study behind them at all. Teams relying on AI-generated figures can avoid false confidence by tracking verification practices on daily.dev."}}]} ```