--- title: A 10-year-old CNN outperforms frontier LLMs for skin... description: A dermatologist tested GPT-6 Astra and Claude Fable 5.1 against a fine-tuned Inception v3 CNN on 200 biopsy-proven skin lesions. The CNN got the correct... canonical: https://daily.dev/posts/a-10-year-old-cnn-outperforms-frontier-llms-for-skin-cancer-diagnosis-f1nlebege twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis | daily.dev og:description: A dermatologist tested GPT-6 Astra and Claude Fable 5.1 against a fine-tuned Inception v3 CNN on 200 biopsy-proven skin lesions. The CNN got the correct... og:url: https://daily.dev/posts/a-10-year-old-cnn-outperforms-frontier-llms-for-skin-cancer-diagnosis-f1nlebege og:image: https://api.daily.dev/og/posts/f1nlEBeGe.png og:image:alt: A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis **[Hacker News](https://daily.dev/sources/hn)** · 6 min read · 1 upvotes · 0 comments ## Summary A dermatologist tested GPT-6 Astra and Claude Fable 5.1 against a fine-tuned Inception v3 CNN on 200 biopsy-proven skin lesions. The CNN got the correct diagnosis 62% of the time, versus 42% for each LLM. It also identified 13 of 30 melanomas as the top choice, versus 8 and 11 for the LLMs. Inception v3 is a 2015 architecture fine-tuned for about 20 minutes on 68,011 ISIC Archive images. Asked as a patient whether to see a doctor, both LLMs advised seeing one almost every time, even for harmless lesions. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Can GPT-6 Astra or Claude Fable 5.1 accurately diagnose skin cancer from photos? Not reliably. In a test of 200 biopsy-proven skin lesions, GPT-6 Astra and Claude Fable 5.1 each identified the correct diagnosis only 42% of the time (84 and 85 cases respectively), compared to 62% for a fine-tuned Inception v3 CNN. Both LLMs also frequently misdiagnosed melanomas, correctly naming only 8 and 11 of 30 as their top choice. _Anyone weighing LLMs against specialized models for medical image tasks can track findings like this on daily.dev._ ### How does a 10-year-old CNN like Inception v3 compare to modern LLMs for image classification tasks? A fine-tuned Inception v3, originally published by Google in 2015, outperformed two frontier LLMs on dermatology image classification, scoring 62% diagnostic accuracy versus 42% for each LLM across 200 skin lesions. It was fine-tuned for about 20 minutes on 68,011 dermoscopic and clinical images from the ISIC Archive using a single H100 GPU, showing specialized smaller models can beat general-purpose frontier models on narrow visual tasks. _Developers deciding between fine-tuned CNNs and general LLMs for vision tasks can follow comparisons like this on daily.dev._ ### Is it safe to rely on AI chatbots to tell patients whether a skin lesion needs a doctor visit? No, both GPT-6 Astra and Claude Fable 5.1 advised seeing a doctor for nearly every lesion when asked as a patient, including 99 and 98 of 100 biopsy-proven melanomas, suggesting extreme over-caution rather than useful triage. When used diagnostically instead of as a patient-facing chatbot, the same models misdischarged harmless lesions and underdiagnosed melanoma at higher rates than a specialized CNN. _Teams building patient-facing medical AI tools can weigh evidence like this through daily.dev before shipping._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 104 comments across hackernews (as of 2026-10-25). **TL;DR:** Most commenters see the result as unsurprising: a narrow, fine-tuned classifier beating general-purpose LLMs at a specific task is expected, not newsworthy, though a few flag methodological caveats like benchmaxxing and small/old dataset fragility. **Sentiment:** 10% positive · 35% mixed · 55% skeptical **The case for** - A few note LLMs are still useful as a fast, low-cost way to bootstrap or stepping-stone toward specialized classifiers. - Some argue building and fine-tuning narrow classifiers has gotten much easier thanks to LLM tooling. **The pushback** - Many argue the comparison is unfair since a model fine-tuned on in-domain data will naturally beat a general-purpose model never trained for that narrow task. - Several call the result 'benchmaxxing' and warn the fine-tuned CNN likely doesn't generalize to real-world ('wild') images outside its narrow training distribution. - Some say the CNN architecture is outdated and that modern ensembles of vision transformers already perform much better and have passed clinical trials. - Commenters note narrow classifiers are often fragile, breaking down with different lighting, skin tones, or out-of-distribution images. - A few dispute the framing itself, arguing it's well known and not surprising that LLMs underperform specialized models outside their strong domains like code. **By community** - hackernews (skeptical): Commenters largely view the finding as unsurprising and partly a methodological artifact, while still debating the broader implications for when classifiers versus LLMs make sense. **Hottest debate:** Whether this result is a meaningful finding about LLM limitations or simply an unfair/benchmaxxed comparison between a narrowly fine-tuned model and general-purpose ones. **Open questions** - Would a frontier LLM perform closer to the CNN if given the same fine-tuning or tool access to a specialized classifier? - How would the CNN perform on more diverse, 'wild' images outside the curated ISIC dataset? - How would newer models (e.g. Chinese LLMs, vision transformer ensembles) compare in the same test? **Highlights** > > I used Inception v3 pretrained on everyday photographs and then fine-tuned all layers on 68,011 dermoscopic and clinical images from the ISIC Archive on a single H100 GPU. >These 200 test lesions, and every other photograph of the same lesion or patient, were removed from its data (including the validation set) before training, so it had never seen them, and they were not used to choose the model or set its threshold. This is just benchmaxxing. You can always beat a benchmark with a small model by finetuning on a train/test split. But it doesn't generalize well to 'wild' images, because they are out-of-domain. > — [Legend2440 on hackernews · 3 comments](https://news.ycombinator.com/item?id=50234247) > This is like all those “I fine tuned a model for that beats ” stories. It’s normally possible with sufficient data to build train/test splits and fine tune a narrow model in a way that it outperforms general models, it’s a given that’s it’s possible with the right task definition and data. The problem is the narrow models are always super fragile. You change the lighting, zoom level, skin tone, or give it a picture of a cat (or more dangerously a misaligned image) instead of a skin lesion, and it breaks completely. None of this is new, it’s why narrow classifiers peaked almost 10 years ago and why large general transformer models took over. If you have a task where the data distribution is always identical and has no long tail or outliers then inception or resnet or whatever is great. Problem is these are vanishingly rare. > — [andy99 on hackernews](https://news.ycombinator.com/item?id=50234849) > Which CNN? A fine-tuned version of Inception v3, fine-tuned on 68,011 dermatology images Which LLM? GPT-6 Astra and Claude Fable 5.1, not specified exactly which version Obviously this is a bit like comparing an 100 iq radiologist, 20 years experience, with a just-tested-at-iq-130 6 year old and observing the radiologist beat the crap out of the kid. Was anybody really expecting a different outcome? Let's see after fine tuning. Btw: the actual lesson from that network when it came out, which is really important is that Inception v6, the Google trained version (ie. can recognize cats, halloween masks and airplanes) beat, with a wide margin, the version that was trained from scratch on clinical images (and still didn't do so well, 62%. Not even really good enough for a first triage). > — [spwa4 on hackernews · 1 comments](https://news.ycombinator.com/item?id=50234200) > Here's a good heuristic for predicting the ability of a modern frontier LLM at any non-automatable task: Is it hugely overrepresented in their pre-training dataset (e.g. code), and were they specifically post-trained on it using reinforcement learning (e.g. coding)? If the answer to one or both is yes, you should expect them to be very good at it. Otherwise, you should expect them to be, at best, as good as an average human who's read a few books on the subject and is good at bullshitting. > — [chilmers on hackernews · 2 comments](https://news.ycombinator.com/item?id=50234565) > > I compared to a 10-year old CNN and to my surprise found that this was superior. I'm surprised he's surprised. > A small, old, specialised network using an outdated architecture recognises skin lesions far better than two frontier LLMs. Yes of course. A 30m params Katago network with one playout is superhuman in playing go, while frontier LLMs have over 10,000 times more parameters and aren't any good at playing go. They're good at other things (such as helping you create a network that's good at playing go). > — [tasuki on hackernews](https://news.ycombinator.com/item?id=50235094) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=50233779) · 153 points · 104 comments ## Similar posts on daily.dev - [Medium](https://daily.dev/posts/medium-bsu2jkucp) · Medium · 0 upvotes · 0 comments - [Why GPT-5.4, Claude, and Gemini can’t agree on basic, real-world facts](https://daily.dev/posts/why-gpt-5-4-claude-and-gemini-can-t-agree-on-basic-real-world-facts-q7pcokdcw) · The New Stack · 0 upvotes · 0 comments - [Microsoft AI tool beat generalist doctors at step-by-step diagnosis](https://daily.dev/posts/microsoft-ai-tool-beat-generalist-doctors-at-step-by-step-diagnosis-cev9jn1gr) · The Next Web · 0 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#deep-learning](https://daily.dev/tags/deep-learning), [#computer-vision](https://daily.dev/tags/computer-vision), [#healthcare](https://daily.dev/tags/healthcare) [View this post on daily.dev](https://daily.dev/posts/a-10-year-old-cnn-outperforms-frontier-llms-for-skin-cancer-diagnosis-f1nlebege) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis","url":"https://daily.dev/posts/a-10-year-old-cnn-outperforms-frontier-llms-for-skin-cancer-diagnosis-f1nlebege","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/a-10-year-old-cnn-outperforms-frontier-llms-for-skin-cancer-diagnosis-f1nlebege"},"datePublished":"2026-10-25T18:07:25.288Z","dateModified":"2026-10-25T19:38:37.460Z","description":"A dermatologist tested GPT-6 Astra and Claude Fable 5.1 against a fine-tuned Inception v3 CNN on 200 biopsy-proven skin lesions. The CNN got the correct...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/18b08632f4099e3bc54cb8fa31674882?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/18b08632f4099e3bc54cb8fa31674882?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/a-10-year-old-cnn-outperforms-frontier-llms-for-skin-cancer-diagnosis-f1nlebege","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,deep-learning,computer-vision,healthcare","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/a-10-year-old-cnn-outperforms-frontier-llms-for-skin-cancer-diagnosis-f1nlebege#faq","mainEntity":[{"@type":"Question","name":"Can GPT-6 Astra or Claude Fable 5.1 accurately diagnose skin cancer from photos?","acceptedAnswer":{"@type":"Answer","text":"Not reliably. In a test of 200 biopsy-proven skin lesions, GPT-6 Astra and Claude Fable 5.1 each identified the correct diagnosis only 42% of the time (84 and 85 cases respectively), compared to 62% for a fine-tuned Inception v3 CNN. Both LLMs also frequently misdiagnosed melanomas, correctly naming only 8 and 11 of 30 as their top choice. Anyone weighing LLMs against specialized models for medical image tasks can track findings like this on daily.dev."}},{"@type":"Question","name":"How does a 10-year-old CNN like Inception v3 compare to modern LLMs for image classification tasks?","acceptedAnswer":{"@type":"Answer","text":"A fine-tuned Inception v3, originally published by Google in 2015, outperformed two frontier LLMs on dermatology image classification, scoring 62% diagnostic accuracy versus 42% for each LLM across 200 skin lesions. It was fine-tuned for about 20 minutes on 68,011 dermoscopic and clinical images from the ISIC Archive using a single H100 GPU, showing specialized smaller models can beat general-purpose frontier models on narrow visual tasks. Developers deciding between fine-tuned CNNs and general LLMs for vision tasks can follow comparisons like this on daily.dev."}},{"@type":"Question","name":"Is it safe to rely on AI chatbots to tell patients whether a skin lesion needs a doctor visit?","acceptedAnswer":{"@type":"Answer","text":"No, both GPT-6 Astra and Claude Fable 5.1 advised seeing a doctor for nearly every lesion when asked as a patient, including 99 and 98 of 100 biopsy-proven melanomas, suggesting extreme over-caution rather than useful triage. When used diagnostically instead of as a patient-facing chatbot, the same models misdischarged harmless lesions and underdiagnosed melanoma at higher rates than a specialized CNN. Teams building patient-facing medical AI tools can weigh evidence like this through daily.dev before shipping."}}]} ```