--- title: Verification Is Not Knowledge: Science needs more than... description: Verified AI outputs are not the same as scientific knowledge, Jessica Hullman argues. She traces the push to treat correctness as sufficient to a 'value-free... canonical: https://daily.dev/posts/verification-is-not-knowledge-science-needs-more-than-answers-ssjgmfnzu twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Verification Is Not Knowledge: Science needs more than answers | daily.dev og:description: Verified AI outputs are not the same as scientific knowledge, Jessica Hullman argues. She traces the push to treat correctness as sufficient to a 'value-free... og:url: https://daily.dev/posts/verification-is-not-knowledge-science-needs-more-than-answers-ssjgmfnzu og:image: https://api.daily.dev/og/posts/sSJgmfNZU.png og:image:alt: Verification Is Not Knowledge: Science needs more than answers og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Verification Is Not Knowledge: Science needs more than answers **[Hacker News](https://daily.dev/sources/hn)** · 16 min read · 0 upvotes · 0 comments ## Summary Verified AI outputs are not the same as scientific knowledge, Jessica Hullman argues. She traces the push to treat correctness as sufficient to a 'value-free ideal' from the Cold War era, which excludes human judgment from truth-seeking. Terry Tao's thought experiment about an AI-discovered cancer cure with no human understanding shows the risk: if it passed trials by exploiting a flaw, no one could catch it. Tom Dietterich's concerns about arXiv papers no human author can explain raise the same issue. Hullman contends that accountability, not just correctness, is what lets a community interrogate and repair claims, and that losing human understanding risks losing the capacity to judge AI's answers at all. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What thought experiment did Terry Tao use to question AI-driven proofs without human understanding? Terry Tao asked how comfortable people would be taking a cancer treatment produced by AI, complete with a formal certificate of correctness and a passed phase 3 trial, but with no human understanding of how it was found. The scenario raises the possibility that the AI passed the trial by exploiting a weakness in the trial itself, with no one able to catch that failure. _Developers weighing how much to trust unexplainable AI outputs can follow this debate on daily.dev._ ### What is the philosophical difference between reliabilist and classical epistemology when it comes to trusting AI results? Classical epistemology holds that true beliefs count as knowledge only if the believer can give reasons for them, while reliabilist epistemology allows true beliefs to count as knowledge even without reasons, as long as they came from a reliable process. Philosopher Robert Brandom argues reliability alone is not enough, since a community cannot know its beliefs are reliable without attributing reliability through inference, which itself requires justification. _Anyone deciding how much to trust AI-generated claims can dig into this reasoning on daily.dev._ ### Why is Tom Dietterich concerned about AI-written papers submitted to arXiv? Tom Dietterich raised concerns about arXiv's growing problem with papers submitted by human authors who seem unlikely to have the expertise to understand the claims in their own papers. The issue is that formal authorship survives while substantive authorship weakens, so no author can take responsibility for or defend claims they cannot understand if challenged. _Researchers tracking AI authorship and accountability debates can follow developments like this on daily.dev._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 116 comments across hackernews (as of 2026-11-04). **TL;DR:** Discussion largely agrees with the piece's core distinction between verified answers and true understanding, but splits sharply on whether that distinction will matter much longer as AI capabilities grow, with math/CS seen as furthest along toward 'verification without understanding' becoming workable. **Sentiment:** 15% positive · 55% mixed · 30% skeptical **The case for** - Several argue verification itself can constitute a form of knowledge, especially when a formal verifier's workings are understood even if the AI's internals aren't. - Some point out that practical impact has historically been separable from deep understanding (e.g., Roman concrete, many pharmaceuticals), so AI-derived answers without full explanation aren't unprecedented. - A few see AI as already valuable for accelerating math/CS-adjacent fields where formal checking is fast and reliable. **The pushback** - Many worry that current models are poor communicators/explainers even when their outputs are technically correct, making the 'just have it explain itself' solution unrealistic. - Commenters raise the historical analogy of math/physics (Kepler, Newton) to argue that mere predictive accuracy without causal explanation has limits and later needed deeper understanding anyway. - Concern that treating fluent-sounding AI explanations as trustworthy is dangerous since confident-sounding answers can be subtly wrong in ways only domain experts would catch. - Worry that academia's shift toward AI-assisted mass paper production undermines genuine scientific understanding in favor of metrics and output volume. **By community** - hackernews (mixed): A long, philosophically dense thread split between those who think verified AI outputs can count as real knowledge and those who insist human understanding remains essential, with side debates on math vs. science, academia's incentives, and historical analogies like Kepler and Newton. **Hottest debate:** Whether an infallible AI oracle's correct-but-unexplained answer constitutes genuine 'knowledge,' versus requiring human understanding/justification to count as such. **Open questions** - Will future AI models actually become better at explaining and abstracting their reasoning for humans, and is that even a tractable training problem? - How should the scientific community value and credential human interpretation/synthesis work once AI agents can generate the bulk of raw results? - Does the verification-without-understanding gap matter the same way across math/CS as it does in biology or other empirical sciences requiring physical experimentation? **Highlights** > An infallible oracle tells Archimedes that the speed of light in vacuum is ~1.08 billion kilometers per hour. Archimedes, with the iron age tools and methods at his disposal, might be able to verify this and make some use of it himself. Maybe. In any case, what he has gained is knowledge itself. The same goes for our own time. A powerful AI can tell us things at the edge of what we can grasp -- or beyond what we can grasp -- and it would nevertheless be knowledge that we possess. We can work to verify it, we can have it explained to us as though we were schoolchildren, or we can simply accept it as a static fact of the natural world. In truth, we're going to have to learn to coexist with minds that are more powerful than ours. At least in some respects, if not in all respects. The good scenario is that they share information with us, and we receive it however we can. "Head in the sand" and "mutual disdain" are bad outcomes. > — [A\_D\_E\_P\_T on hackernews · 1 comments](https://news.ycombinator.com/item?id=50364660) > No, you're wrong. You might want to look up the actual definition of knowledge. It's a pretty interesting topic in epistemology really, far from your simplistic take. You already live in coexistence with minds more powerful than yours. Society works in a way that you never need to notice that, those minds taking care of things beyond your grasp for you. The downside of that "convenience" is you developing an over-simplified bubble-view of the world, rendering you prone to arbitrarily hazardous misjudgements. Here, you align foolishly with interests that cannot possibly be your own. Namely, exploiting scientific knowledge for personalized monetary gain on an unsustainable scale. When a tiny group of people hogs essentially all scientific knowledge, that renders you what? Your idea of being graciously provided with some "basic income" or whatever is blind copium. You would live as livestock at best, to be discarded as useless whenever your masters see fit. > — [Loquebantur on hackernews · 3 comments](https://news.ycombinator.com/item?id=50364871) > This is like saying if I read the oven thermometer to obtain the temperature inside the oven, then that is not real knowledge, because I don't know how my oven thermometer works. Okay, it's not quite the same. I could figure out how the thermometer works, but I cannot figure out how the 1 trillion parameter AI works. But the AI didn't give me an answer directly, it gave me a proof that I can run through a verifier. And I could figure out how the verifier works. That is a partial understanding. The barrier to my understanding is not the verifier. So it's the opaque AI model? But human brains are blackboxes too. I cannot tell what's going on in another person's head when they're writing a proof either. I can only look at their behavior, the language they produce. An AI provides a proof written in the same symbols humans write in. It's really long and complicated, more than human-written proofs. It would be nice if I had an excuse for my inability to internalize the proof in a timely manner. I could form a community to make the proofs we don't fully understand yet = "not knowledge". This is delusional self-soothing, and it's just as damaging to my capacity to understand reality as delegating thought to another entity, regardless of whether that entity is a computer or a human. > — [txrx0000 on hackernews](https://news.ycombinator.com/item?id=50365739) > > we can surely have the verifiers explain those This is not obvious to me at all. LLMs are capable of amazing feats of reverse engineering but are terrible at explaining and abstracting things well. I think this is an artefact of how we train them, and they could be trained to be better at this if we focused on it, but at the same time, it shows its possible for these things to be able to do things they cant explain to us (granted, they can't explain them well - you can bash your way to understanding with them anyway but they aren't good at it) > — [RugnirViking on hackernews · 3 comments](https://news.ycombinator.com/item?id=50363352) > Ahhh, epistemology... Does any academic researcher even care about these things anymore? If you're in academia, you've probably noticed a major shift recently. Research production at an industrial scale through AI. Grants have "always" been awarded to professors based on quantity multiplied by quality. Now, some professors are churning out 1–3 papers a month with the same group of co-authors. All the grant money goes to them. Grant givers are just only counting the numbers of papers being published. The Industrial Revolution has hit academia, and some academics are busy arguing for the artisanal value of research while some are creating factories for research. > — [wqtz on hackernews · 4 comments](https://news.ycombinator.com/item?id=50364387) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=50363019) · 199 points · 116 comments ## Similar posts on daily.dev - [\[2607.06377\] Automation Without Understanding](https://daily.dev/posts/2607-06377-automation-without-understanding-gagloi3zz) · Hacker News · 1 upvotes · 0 comments - [Medium](https://daily.dev/posts/medium-pudgz0k7b) · Dev Genius · 0 upvotes · 0 comments - [AI control is losing the proof race\!](https://daily.dev/posts/ai-control-is-losing-the-proof-race--wkn43rye0) · Medium · 0 upvotes · 0 comments - [Q&A: Building accountability behind our words — and AI’s — in civic conversations](https://daily.dev/posts/q-a-building-accountability-behind-our-words-and-ai-s-in-civic-conversations-ruosiloen) · MIT News · 0 upvotes · 0 comments --- Tags: [#science](https://daily.dev/tags/science), [#ai-safety](https://daily.dev/tags/ai-safety) [View this post on daily.dev](https://daily.dev/posts/verification-is-not-knowledge-science-needs-more-than-answers-ssjgmfnzu) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Verification Is Not Knowledge: Science needs more than answers","url":"https://daily.dev/posts/verification-is-not-knowledge-science-needs-more-than-answers-ssjgmfnzu","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/verification-is-not-knowledge-science-needs-more-than-answers-ssjgmfnzu"},"datePublished":"2026-11-04T14:56:19.966Z","dateModified":"2026-11-04T20:42:36.602Z","description":"Verified AI outputs are not the same as scientific knowledge, Jessica Hullman argues. She traces the push to treat correctness as sufficient to a 'value-free...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/6a5769b064b6c8dc4d1e52f2160bbef1?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/6a5769b064b6c8dc4d1e52f2160bbef1?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/verification-is-not-knowledge-science-needs-more-than-answers-ssjgmfnzu","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"science,ai-safety","timeRequired":"PT16M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"Verification Is Not Knowledge: Science needs more than answers"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/verification-is-not-knowledge-science-needs-more-than-answers-ssjgmfnzu#faq","mainEntity":[{"@type":"Question","name":"What thought experiment did Terry Tao use to question AI-driven proofs without human understanding?","acceptedAnswer":{"@type":"Answer","text":"Terry Tao asked how comfortable people would be taking a cancer treatment produced by AI, complete with a formal certificate of correctness and a passed phase 3 trial, but with no human understanding of how it was found. The scenario raises the possibility that the AI passed the trial by exploiting a weakness in the trial itself, with no one able to catch that failure. Developers weighing how much to trust unexplainable AI outputs can follow this debate on daily.dev."}},{"@type":"Question","name":"What is the philosophical difference between reliabilist and classical epistemology when it comes to trusting AI results?","acceptedAnswer":{"@type":"Answer","text":"Classical epistemology holds that true beliefs count as knowledge only if the believer can give reasons for them, while reliabilist epistemology allows true beliefs to count as knowledge even without reasons, as long as they came from a reliable process. Philosopher Robert Brandom argues reliability alone is not enough, since a community cannot know its beliefs are reliable without attributing reliability through inference, which itself requires justification. Anyone deciding how much to trust AI-generated claims can dig into this reasoning on daily.dev."}},{"@type":"Question","name":"Why is Tom Dietterich concerned about AI-written papers submitted to arXiv?","acceptedAnswer":{"@type":"Answer","text":"Tom Dietterich raised concerns about arXiv's growing problem with papers submitted by human authors who seem unlikely to have the expertise to understand the claims in their own papers. The issue is that formal authorship survives while substantive authorship weakens, so no author can take responsibility for or defend claims they cannot understand if challenged. Researchers tracking AI authorship and accountability debates can follow developments like this on daily.dev."}}]} ```