--- title: Inside 847 Production Clinical AI Notes — Sebastian Fox,... description: A talk from Composo's Sebastian Fox examines real-world failures in ambient AI scribes used in clinical settings. Analysis of production notes from three... canonical: https://daily.dev/posts/inside-847-production-clinical-ai-notes-sebastian-fox-composo-xkbkczb0i twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Inside 847 Production Clinical AI Notes — Sebastian Fox, Composo | daily.dev og:description: A talk from Composo's Sebastian Fox examines real-world failures in ambient AI scribes used in clinical settings. Analysis of production notes from three... og:url: https://daily.dev/posts/inside-847-production-clinical-ai-notes-sebastian-fox-composo-xkbkczb0i og:image: https://api.daily.dev/og/posts/xKBkCZB0i.png og:image:alt: Inside 847 Production Clinical AI Notes — Sebastian Fox, Composo og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Inside 847 Production Clinical AI Notes — Sebastian Fox, Composo **[AI Engineer](https://daily.dev/sources/aidotengineer)** · 19 min read · 1 upvotes · 0 comments ## Summary A talk from Composo's Sebastian Fox examines real-world failures in ambient AI scribes used in clinical settings. Analysis of production notes from three leading ambient scribe tools found roughly 1 in 20 contained serious errors capable of causing patient harm, nearly 1 in 5 had important omissions, and over 1 in 10 contained hallucinations, with none of this tracked via adverse event reporting. The talk argues that even sophisticated LLM-judge evaluation systems with detailed rubrics miss most of these errors because the hard part isn't spotting a difference between transcript and note, it's knowing which differences matter - a form of contextual, tacit expert judgment ('taste') that can't be fully written down in a static rubric. The proposed fix is a continuous loop: discover failure modes from real production outputs, capture expert corrections and reasoning (not just scores), and calibrate each new output against retrieved similar past judgments rather than a frozen rubric or fine-tuned weights - keeping the evaluation standard alive as it evolves. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How often do AI-generated clinical notes from ambient scribes contain serious errors in real production use? About 1 in 20 AI-generated clinical notes contain an error serious enough to cause significant harm to the patient, based on the largest real-world study of these notes in production. Widening the lens to all errors, nearly 1 in 5 notes have an important omission and more than 1 in 10 contain a hallucination, and these numbers come from real patient encounters, not testing. _Teams shipping high-stakes AI features track findings like this on daily.dev to gauge real-world reliability risks._ ### Why do LLM-as-judge evaluation systems fail to catch dangerous omissions in AI-generated notes? LLM judges miss most dangerous omissions because they lack a sense of what clinically matters in a specific context, not because they lack intelligence or a rubric. Testing showed one in five notes that a strong rubric-based frontier-model judge marked as clean still contained a serious buried error, usually an omission, because the standard of what counts as important was never written down for the judge to check against. _Anyone building LLM-judge pipelines can follow this reasoning on daily.dev before trusting a rubric-only evaluator._ ### What is a better approach than a static rubric for evaluating AI outputs in high-stakes domains? Instead of a fixed rubric or fine-tuned weights, build a repeating loop that discovers failure modes from real production outputs, captures expert reasoning and corrections on those cases, and calibrates each new output by retrieving the most similar past judgments into the judge's context at inference time. This keeps the evaluation standard current as guidelines and models shift, and lets you trace exactly what moved a score. _Developers designing AI evaluation pipelines can find approaches like this on daily.dev when static rubrics fall short._ ## Similar posts on daily.dev - [Sick and wrong: Ontario auditors find doctors' AI note takers routinely blow basic facts](https://daily.dev/posts/sick-and-wrong-ontario-auditors-find-doctors-ai-note-takers-routinely-blow-basic-facts-jwyn37ihh) · Hacker News · 1 upvotes · 0 comments - [Google Newsletter Issue Agent-CLI Eval](https://daily.dev/posts/google-newsletter-issue-agent-cli-eval-zbtoft45i) · Daily Dose of Data Science \| Avi Chawla \| Substack · 3 upvotes · 1 comments - [That Is Embarrassing: Why Frontier AI Still Makes Things Up, and What to Do About It](https://daily.dev/posts/that-is-embarrassing-why-frontier-ai-still-makes-things-up-and-what-to-do-about-it-y2kdfc0zu) · Towards Data Science · 1 upvotes · 0 comments - [Claude and NotebookLM proved that AI research isn't the gimmick I thought it was](https://daily.dev/posts/claude-and-notebooklm-proved-that-ai-research-isn-t-the-gimmick-i-thought-it-was-ppyezmkam) · XDA Developers · 1 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#healthcare](https://daily.dev/tags/healthcare), [#ai-safety](https://daily.dev/tags/ai-safety) [View this post on daily.dev](https://daily.dev/posts/inside-847-production-clinical-ai-notes-sebastian-fox-composo-xkbkczb0i) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Inside 847 Production Clinical AI Notes — Sebastian Fox, Composo","url":"https://daily.dev/posts/inside-847-production-clinical-ai-notes-sebastian-fox-composo-xkbkczb0i","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/inside-847-production-clinical-ai-notes-sebastian-fox-composo-xkbkczb0i"},"datePublished":"2026-08-22T17:08:53.342Z","dateModified":"2026-09-14T07:06:42.286Z","description":"A talk from Composo's Sebastian Fox examines real-world failures in ambient AI scribes used in clinical settings. Analysis of production notes from three...","image":"https://i.ytimg.com/vi/yqF6XhzbWBk/sddefault.jpg","thumbnailUrl":"https://i.ytimg.com/vi/yqF6XhzbWBk/sddefault.jpg","isAccessibleForFree":true,"articleSection":"AI Engineer","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"AI Engineer","logo":"https://media.daily.dev/image/upload/s--u5PucxNT--/f_auto/v1724338940/logos/aidotengineer","url":"https://daily.dev/sources/aidotengineer"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/inside-847-production-clinical-ai-notes-sebastian-fox-composo-xkbkczb0i","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,healthcare,ai-safety","timeRequired":"PT19M","video":{"@type":"VideoObject","name":"Inside 847 Production Clinical AI Notes — Sebastian Fox, Composo","description":"A talk from Composo's Sebastian Fox examines real-world failures in ambient AI scribes used in clinical settings. Analysis of production notes from three...","thumbnailUrl":"https://i.ytimg.com/vi/yqF6XhzbWBk/sddefault.jpg","uploadDate":"2026-08-22T17:08:53.342Z","duration":"PT19M","url":"https://api.daily.dev/r/xKBkCZB0i","embedUrl":"https://www.youtube.com/embed/yqF6XhzbWBk"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"AI Engineer","item":"https://daily.dev/sources/aidotengineer"},{"@type":"ListItem","position":3,"name":"Inside 847 Production Clinical AI Notes — Sebastian Fox, Composo"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/inside-847-production-clinical-ai-notes-sebastian-fox-composo-xkbkczb0i#faq","mainEntity":[{"@type":"Question","name":"How often do AI-generated clinical notes from ambient scribes contain serious errors in real production use?","acceptedAnswer":{"@type":"Answer","text":"About 1 in 20 AI-generated clinical notes contain an error serious enough to cause significant harm to the patient, based on the largest real-world study of these notes in production. Widening the lens to all errors, nearly 1 in 5 notes have an important omission and more than 1 in 10 contain a hallucination, and these numbers come from real patient encounters, not testing. Teams shipping high-stakes AI features track findings like this on daily.dev to gauge real-world reliability risks."}},{"@type":"Question","name":"Why do LLM-as-judge evaluation systems fail to catch dangerous omissions in AI-generated notes?","acceptedAnswer":{"@type":"Answer","text":"LLM judges miss most dangerous omissions because they lack a sense of what clinically matters in a specific context, not because they lack intelligence or a rubric. Testing showed one in five notes that a strong rubric-based frontier-model judge marked as clean still contained a serious buried error, usually an omission, because the standard of what counts as important was never written down for the judge to check against. Anyone building LLM-judge pipelines can follow this reasoning on daily.dev before trusting a rubric-only evaluator."}},{"@type":"Question","name":"What is a better approach than a static rubric for evaluating AI outputs in high-stakes domains?","acceptedAnswer":{"@type":"Answer","text":"Instead of a fixed rubric or fine-tuned weights, build a repeating loop that discovers failure modes from real production outputs, captures expert reasoning and corrections on those cases, and calibrates each new output by retrieving the most similar past judgments into the judge's context at inference time. This keeps the evaluation standard current as guidelines and models shift, and lets you trace exactly what moved a score. Developers designing AI evaluation pipelines can find approaches like this on daily.dev when static rubrics fall short."}}]} ```