--- title: Cammarata says neural nets were never tangled; the... description: A sprawling 26,000-word post from Goodfire researcher Nick Cammarata argues that neural network superposition — the idea that unrelated concepts get crammed... canonical: https://daily.dev/posts/cammarata-says-neural-nets-were-never-tangled-the-interpretability-crowd-isn-t-buying-it-mkiimpf7w copyright: © 2026 Daily Dev Ltd. terms: https://daily.dev/tos/ twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Cammarata says neural nets were never tangled; the interpretability crowd isn't buying it | daily.dev og:description: A sprawling 26,000-word post from Goodfire researcher Nick Cammarata argues that neural network superposition — the idea that unrelated concepts get crammed... og:url: https://daily.dev/posts/cammarata-says-neural-nets-were-never-tangled-the-interpretability-crowd-isn-t-buying-it-mkiimpf7w og:image: https://api.daily.dev/og/posts/mkIimPf7W.png og:image:alt: Cammarata says neural nets were never tangled; the interpretability crowd isn't buying it og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Cammarata says neural nets were never tangled; the interpretability crowd isn't buying it **[Trends](https://daily.dev/sources/trends)** · 2 min read · 3 upvotes · 0 comments ## Summary A sprawling 26,000-word post from Goodfire researcher Nick Cammarata argues that neural network superposition — the idea that unrelated concepts get crammed arbitrarily into the same neurons — was largely a misreading of the evidence. Reanalyzing the famous cat-car neuron from Inception v1's Zoom In work, Cammarata argues shared visual features like a V shape explain the apparent tangle, not arbitrary packing. He claims modern architectures with residual connections and LayerNorm reach 99-100% interpretable neurons, and that preliminary audits of Qwen3-32B and trillion-parameter models like Kimi K3 show 90%+ of neurons have one main interpretable job. Interpretability researchers Chris Potts, Jacob Hilton and Neel Nanda pushed back hard, noting current understanding covers less than 0.01% of top LLMs' inner workings and that single-neuron readings don't explain circuit-level behavior. A technical companion paper is promised but not yet available. ## Content A 26,000-word blog post from Nick Cammarata, a researcher at Goodfire, is dividing the interpretability world. His claim: neural networks are far cleaner than the field assumed, and "superposition" (unrelated concepts crammed arbitrarily into the same neurons) was largely a misreading. ## The pushback The skeptics are named and serious. Chris Potts, Jacob Hilton and Neel Nanda have all argued the claims are overhyped. Their sharpest point is scale: current understanding of top LLMs covers less than 0.01% of their inner workings. They also say that reading a single neuron doesn't explain how neurons combine into circuits. ## The fans @omarsar0 calls it "the type of interpretability work that we need more of," and quotes the line that carries the post's boldest promise: "We can now understand most neurons in models up to trillions of parameters, and think the outlook for mechanistic interpretability may be far better than we thought." ## The evidence The argument starts by going back to the famous cat-car neuron in Inception v1 from the 2020 Zoom In work. Cammarata's reading is that cats and cars share a V shape, and the dog-car circuit reflects a shared pose. Neither is arbitrary packing. From there he goes bigger: - InceptionV1's uninterpretable last layer comes from architecture and tooling limits, not superposition. Modern architectures with residual connections and LayerNorm reach 99-100% interpretable neurons at the same size. - Most apparent polysemanticity is what he calls "recursive generalization." A neuron grows outward along related but non-obvious axes. An "iy" neuron generalizes to "diy," then to "do it yourself." - Preliminary audits of Qwen3-32B and trillion-parameter models like Kimi K3 suggest 90%+ of neurons have one main interpretable job. ## Why people care If he's right, interpretability becomes "normal science" on the networks directly. You wouldn't need decomposition techniques to untangle them first. Cammarata now thinks full understanding of LLMs may be reachable with enough effort, which would matter a lot for AI safety and control. A technical companion paper is promised. Until it lands, this is a reframing backed by a very long essay and some early audits, set against critics who say a neuron-level story isn't a model-level one. My bet is both sides are partly right, and the argument will turn on how "interpretable" gets defined. ## Questions this post answers ### What is Nick Cammarata's argument against neural network superposition? Nick Cammarata, a researcher at Goodfire, argues in a 26,000-word post that superposition (unrelated concepts crammed into the same neurons) was largely a misreading of the evidence. He reanalyzes the Inception v1 cat-car neuron from the 2020 Zoom In work, arguing cats and cars share a V shape rather than being arbitrarily packed together, and claims modern architectures reach 99-100% interpretable neurons. _Developers tracking the interpretability debate can follow both sides of this argument on daily.dev._ ### Why do interpretability researchers like Neel Nanda and Jacob Hilton disagree with claims that neural networks are highly interpretable? Neel Nanda, Jacob Hilton and Chris Potts argue the claims are overhyped because current understanding of top large language models covers less than 0.01% of their inner workings. They also point out that explaining a single neuron does not explain how neurons combine into circuits, meaning a neuron-level story isn't necessarily a model-level one. _Those weighing AI safety claims against actual research progress can find the counterarguments on daily.dev._ ### What percentage of neurons were found interpretable in audits of Qwen3-32B and Kimi K3? Preliminary audits of Qwen3-32B and trillion-parameter models like Kimi K3 suggested over 90% of neurons have one main interpretable job, according to Nick Cammarata's analysis. He attributes earlier uninterpretable results, such as Inception v1's last layer, to architecture and tooling limitations rather than inherent superposition, noting modern architectures with residual connections and LayerNorm perform far better. _Engineers evaluating interpretability tooling for large models can track findings like this on daily.dev._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 94 comments across hackernews (as of 2026-10-19). **TL;DR:** Reactions are split between people who find the essay's reframing of polysemanticity as misread generalization compelling and well-argued, and others who see it as overlong, unproven, AI-assisted-sounding, and overreaching relative to prior work on superposition and feature compression. **Sentiment:** 20% positive · 40% mixed · 40% skeptical **The case for** - Some found the generalization explanation (e.g., the Chinese '羊' compression example) intuitive and a plausible account of why small models look messy. - A few praised the linked interactive neuron-microscope demo as the first non-trivial network example they felt they understood. - Several appreciated that the piece at least attempts rigorous, falsifiable claims about specific neurons and circuits rather than vague assertions. **The pushback** - Many felt the essay is excessively long, repetitive, and poorly summarized, requiring outside TL;DRs or glossaries to parse. - Commenters pushed back that the claims overreach what's actually proven, noting prior work on feature compression/condensation isn't addressed or reconciled. - Some argued interpretability claims can't really be verified outside domains humans already understand (e.g., image/language), so the approach may not generalize to harder domains like medical imaging. - Technical skepticism that sparse-autoencoder-style interpretability methods are reliable or can surface genuinely novel (not already-known) concepts. - Complaints about the site itself: laggy/heavy 3D rendering, fingerprinting scripts, and page crashes. - Some doubted the author's credibility or framing, including unrelated personal criticism of his past public statements. **By community** - hackernews (mixed): Discussion ranges from technically engaged skepticism and debate over what 'clean' even means, to praise for specific examples, to complaints about length, site performance, and the author's credibility. **Hottest debate:** Whether the 'recursive generalization' explanation genuinely overturns superposition as the cause of polysemanticity, or merely reframes it without resolving open questions about feature compression. **Open questions** - Does this interpretability approach generalize to domains (like medical imaging) where ground truth can't be verified by human recognition? - How does the generalization account relate to or contradict existing work on feature/geometric condensation in multimodal and other models? - Can these methods actually surface novel, previously-unknown concepts, or only confirm concepts researchers already suspect to look for? **Highlights** > 羊. This is a Chinese character that can refer to goats, rams, and several other kinds of "medium-sized animal with thick wooly coats and horns". English has no direct equivalent; we have to be too specific when we talk about these animals. A model with a small number of neurons has to compact its concepts in order to fit within the parameter count while reducing loss. OP's article shows that the compaction is usually generalization: "goat" and "ram" get compressed into "羊". The canonical example of superposition, an image model where "car" feeds into a bunch of "dog" features, was shown to be actually generalization. In a sense, the model was so small that its optimizer didn't have the parameter budget to distinguish between "car" and "dog"; it has a single set of "thing that moves fast" features. So basically, really small models are merging features into semantic buckets that are so generalized they don't make sense to big human brains unless we really, really squint. The buckets chosen happen to be what has the lowest loss, of course. > — [kmeisthax on hackernews](https://news.ycombinator.com/item?id=50107822) > The claims seem to be an overreach of what is actually proven. We've known about progressive feature complexity and the impacts of early training structures on later training for a really long time (since CNN's, The Lottery Ticket Hypothesis, etc...) and neither of those phenomenon are mutually exclusive with parameter overloading. The Multimodal Gap work (https://arxiv.org/abs/2203.02053) seems to imply networks have a tendency towards condensed representations that are clearly not semantic and a bunch of other considerations are already on the table for discussion. I don't think these have all been considered sufficiently to start claiming we are around the corner from mechanistic interpretation. > — [bluejay2387 on hackernews · 1 comments](https://news.ycombinator.com/item?id=50108530) > Typically the analysis works in the reverse direction: we have a neuron (or SAE feature, or whatever) and we look at examples of data that makes it activate, then inferring an interpretation. This is a lot harder than probing for a known concept, and one of the points of this article is that if you try harder to understand the activation pattern, you will very often find that there is something interpretable there, even if at first glance it looked random. > — [aesthesia on hackernews](https://news.ycombinator.com/item?id=50109361) > But they don't. That's the whole reason they work in the first place. Every neuron in the layer encodes fractionally something about the concepts embedded in the whole thing. It's more like a hologram than a series of connected facts. > — [jacquesm on hackernews · 1 comments](https://news.ycombinator.com/item?id=50107054) > So we have here a massive evidentiary brief, the point of which is to argue that it is easier than we thought to, like, "sequence the AI genome" (in my estimation). We revisit famously-messy neurons, examine various tool failures, check places where messy things still hide, and catalog how "polysemanticity" is represented variously in the network. Once sequenced, the thinking goes, we'll be able to do things like: explain and anticipate failures; predict what happens when you modulate the pathway an input takes on its way to becoming output; reconstruct algorithms the models have learned. I'm not 100% sure this will work out like the mechinterp people think. If the neural representations and pathways are "clean" (organized into a a meaningful-to-humans hierarchy), it seems at least very plausible that the behaviors that AI xrisk people are worried about will mostly boil down to straightforward synthesis (i.e., mostly-linear-interpolation) of mostly-well-cataloged knowledge. And if that is true, then it does not seem hopeful that there would be a way to prevent the output they are worried about, other than just removing a bunch of the knowledge. So maybe there is no alignment solution at all. > — [antics on hackernews](https://news.ycombinator.com/item?id=50108927) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=50105647) · 273 points · 94 comments ## Similar posts on daily.dev - [The Aristotelian Correction: Why Neural Networks May Only Converge Locally](https://daily.dev/posts/the-aristotelian-correction-why-neural-networks-may-only-converge-locally-ifflkjxcr) · Medium · 0 upvotes · 0 comments - [How artificial neural networks build their own learning machinery](https://daily.dev/posts/how-artificial-neural-networks-build-their-own-learning-machinery-wbguw3n0l) · MIT News · 0 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#neural-networks](https://daily.dev/tags/neural-networks), [#ai-safety](https://daily.dev/tags/ai-safety) [View this post on daily.dev](https://daily.dev/posts/cammarata-says-neural-nets-were-never-tangled-the-interpretability-crowd-isn-t-buying-it-mkiimpf7w) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Cammarata says neural nets were never tangled; the interpretability crowd isn't buying it","url":"https://daily.dev/posts/cammarata-says-neural-nets-were-never-tangled-the-interpretability-crowd-isn-t-buying-it-mkiimpf7w","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/cammarata-says-neural-nets-were-never-tangled-the-interpretability-crowd-isn-t-buying-it-mkiimpf7w"},"datePublished":"2026-10-19T18:09:18.596Z","dateModified":"2026-10-19T18:09:59.255Z","description":"A sprawling 26,000-word post from Goodfire researcher Nick Cammarata argues that neural network superposition — the idea that unrelated concepts get crammed...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/2ba6298c45e8673319931a6fc12a374e?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/2ba6298c45e8673319931a6fc12a374e?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Trends","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Trends","logo":"https://media.daily.dev/image/upload/s--ZfSp3asX--/f_auto,q_auto/v1780996004/logos/trends?_a=BAMAMiWQ0","url":"https://daily.dev/sources/trends"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/cammarata-says-neural-nets-were-never-tangled-the-interpretability-crowd-isn-t-buying-it-mkiimpf7w","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":3},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,neural-networks,ai-safety","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Trends","item":"https://daily.dev/sources/trends"},{"@type":"ListItem","position":3,"name":"Cammarata says neural nets were never tangled; the interpretability crowd isn't buying it"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/cammarata-says-neural-nets-were-never-tangled-the-interpretability-crowd-isn-t-buying-it-mkiimpf7w#faq","mainEntity":[{"@type":"Question","name":"What is Nick Cammarata's argument against neural network superposition?","acceptedAnswer":{"@type":"Answer","text":"Nick Cammarata, a researcher at Goodfire, argues in a 26,000-word post that superposition (unrelated concepts crammed into the same neurons) was largely a misreading of the evidence. He reanalyzes the Inception v1 cat-car neuron from the 2020 Zoom In work, arguing cats and cars share a V shape rather than being arbitrarily packed together, and claims modern architectures reach 99-100% interpretable neurons. Developers tracking the interpretability debate can follow both sides of this argument on daily.dev."}},{"@type":"Question","name":"Why do interpretability researchers like Neel Nanda and Jacob Hilton disagree with claims that neural networks are highly interpretable?","acceptedAnswer":{"@type":"Answer","text":"Neel Nanda, Jacob Hilton and Chris Potts argue the claims are overhyped because current understanding of top large language models covers less than 0.01% of their inner workings. They also point out that explaining a single neuron does not explain how neurons combine into circuits, meaning a neuron-level story isn't necessarily a model-level one. Those weighing AI safety claims against actual research progress can find the counterarguments on daily.dev."}},{"@type":"Question","name":"What percentage of neurons were found interpretable in audits of Qwen3-32B and Kimi K3?","acceptedAnswer":{"@type":"Answer","text":"Preliminary audits of Qwen3-32B and trillion-parameter models like Kimi K3 suggested over 90% of neurons have one main interpretable job, according to Nick Cammarata's analysis. He attributes earlier uninterpretable results, such as Inception v1's last layer, to architecture and tooling limitations rather than inherent superposition, noting modern architectures with residual connections and LayerNorm perform far better. Engineers evaluating interpretability tooling for large models can track findings like this on daily.dev."}}]} ```