--- title: 7 Things That Break LLM Apps in Production | daily.dev description: Seven common failure modes that break LLM-powered apps in production, drawn from real incidents. Covers missing timeouts and retries (a hung HTTP call with no... canonical: https://daily.dev/posts/7-things-that-break-llm-apps-in-production-kyzofungo twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: 7 Things That Break LLM Apps in Production | daily.dev og:description: Seven common failure modes that break LLM-powered apps in production, drawn from real incidents. Covers missing timeouts and retries (a hung HTTP call with no... og:url: https://daily.dev/posts/7-things-that-break-llm-apps-in-production-kyzofungo og:image: https://api.daily.dev/og/posts/KYzOFUnGo.png og:image:alt: 7 Things That Break LLM Apps in Production og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # 7 Things That Break LLM Apps in Production **[The T-Shaped Dev](https://daily.dev/sources/thetshaped-dev)** · 10 min read · 3 upvotes · 0 comments ## Summary Seven common failure modes that break LLM-powered apps in production, drawn from real incidents. Covers missing timeouts and retries (a hung HTTP call with no timeout silently killed an email-drafting agent for five hours), prompt injection risks when untrusted input reaches the context window, writing tests that assert on schema shape rather than exact strings, unbounded token costs from naive chat history loops, silent context-window overflow causing models to ignore instructions, the need for a lightweight eval harness to measure prompt changes, and the importance of validating and parsing every model output rather than trusting it. Each issue includes concrete Python code patterns and TypeScript equivalents where relevant. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How should I handle timeouts and retries when calling an LLM API in production? Wrap every model call with a hard timeout that actually cancels the in-flight request (asyncio.timeout() in Python, AbortSignal.timeout() in TypeScript), a capped number of retries on 429s, 5xx errors, and timeouts, and exponential backoff with jitter so retrying clients don't hammer the API in lockstep when it recovers. A timeout that doesn't cancel the request is decorative and won't prevent an outage. _Teams hardening LLM integrations against outages track patterns like this one on daily.dev._ ### Why do LLMs seem to ignore instructions when given long prompts? Models reliably lose track of content buried in the middle of long prompts even when it's under the context window limit, and content past the window limit gets silently truncated. This is a context assembly problem, not a prompt wording problem — fix it by setting a token budget, ranking content by relevance to fill that budget, and placing the most important instructions at the start or end of the prompt. _Developers debugging context-window issues can follow LLM engineering practices on daily.dev._ ### How do I test an LLM application when outputs aren't deterministic? Assert on shape and properties instead of exact strings: check that the output parses into a defined schema (pydantic in Python, zod in TypeScript), contains required fields, and includes deterministic elements like a specific order ID or a status from a fixed set, while leaving the prose wording unchecked. This tests the contract the model must honor rather than its exact phrasing. _Engineers building eval suites for LLM features can track testing approaches like this on daily.dev._ ## Similar posts on daily.dev - [How to Deal With Errors and Failures in LLM-Powered Applications](https://daily.dev/posts/how-to-deal-with-errors-and-failures-in-llm-powered-applications-0uawtim2f) · ByteByteGo · 5 upvotes · 0 comments - [What You Cannot See Will Break Your LLM App: A Practitioner Guide to Production Observability](https://daily.dev/posts/what-you-cannot-see-will-break-your-llm-app-a-practitioner-guide-to-production-observability-edxs5oglz) · DevOps.com · 1 upvotes · 0 comments - [The hard part of LLMs isn’t the model. It’s everything around it](https://daily.dev/posts/the-hard-part-of-llms-isn-t-the-model-it-s-everything-around-it-vpj88fgsq) · All Things Open · 0 upvotes · 0 comments - [How to Turn Your LLM Prototype into a Production-Ready System](https://daily.dev/posts/how-to-turn-your-llm-prototype-into-a-production-ready-system-vli6ilmld) · Towards Data Science · 1 upvotes · 0 comments - [27 questions to ask when choosing an LLM](https://daily.dev/posts/27-questions-to-ask-when-choosing-an-llm-ru20obl3q) · InfoWorld · 0 upvotes · 0 comments --- Tags: [#python](https://daily.dev/tags/python), [#llm](https://daily.dev/tags/llm), [#ai-agents](https://daily.dev/tags/ai-agents), [#prompt-injection](https://daily.dev/tags/prompt-injection) [View this post on daily.dev](https://daily.dev/posts/7-things-that-break-llm-apps-in-production-kyzofungo) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"7 Things That Break LLM Apps in Production","url":"https://daily.dev/posts/7-things-that-break-llm-apps-in-production-kyzofungo","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/7-things-that-break-llm-apps-in-production-kyzofungo"},"datePublished":"2026-07-08T10:25:04.474Z","dateModified":"2026-09-13T18:25:50.203Z","description":"Seven common failure modes that break LLM-powered apps in production, drawn from real incidents. Covers missing timeouts and retries (a hung HTTP call with no...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/21fcc21f6e8d79934b118baf5194ec0e?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/21fcc21f6e8d79934b118baf5194ec0e?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"The T-Shaped Dev","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"The T-Shaped Dev","logo":"https://media.daily.dev/image/upload/s--TaOz2Ogr--/f_auto,q_auto/v1774964066/logos/thetshaped-dev?_a=BAMAMiWQ0","url":"https://daily.dev/sources/thetshaped-dev"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/7-things-that-break-llm-apps-in-production-kyzofungo","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":3},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"python,llm,ai-agents,prompt-injection","timeRequired":"PT10M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"The T-Shaped Dev","item":"https://daily.dev/sources/thetshaped-dev"},{"@type":"ListItem","position":3,"name":"7 Things That Break LLM Apps in Production"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/7-things-that-break-llm-apps-in-production-kyzofungo#faq","mainEntity":[{"@type":"Question","name":"How should I handle timeouts and retries when calling an LLM API in production?","acceptedAnswer":{"@type":"Answer","text":"Wrap every model call with a hard timeout that actually cancels the in-flight request (asyncio.timeout() in Python, AbortSignal.timeout() in TypeScript), a capped number of retries on 429s, 5xx errors, and timeouts, and exponential backoff with jitter so retrying clients don't hammer the API in lockstep when it recovers. A timeout that doesn't cancel the request is decorative and won't prevent an outage. Teams hardening LLM integrations against outages track patterns like this one on daily.dev."}},{"@type":"Question","name":"Why do LLMs seem to ignore instructions when given long prompts?","acceptedAnswer":{"@type":"Answer","text":"Models reliably lose track of content buried in the middle of long prompts even when it's under the context window limit, and content past the window limit gets silently truncated. This is a context assembly problem, not a prompt wording problem — fix it by setting a token budget, ranking content by relevance to fill that budget, and placing the most important instructions at the start or end of the prompt. Developers debugging context-window issues can follow LLM engineering practices on daily.dev."}},{"@type":"Question","name":"How do I test an LLM application when outputs aren't deterministic?","acceptedAnswer":{"@type":"Answer","text":"Assert on shape and properties instead of exact strings: check that the output parses into a defined schema (pydantic in Python, zod in TypeScript), contains required fields, and includes deterministic elements like a specific order ID or a status from a fixed set, while leaving the prose wording unchecked. This tests the contract the model must honor rather than its exact phrasing. Engineers building eval suites for LLM features can track testing approaches like this on daily.dev."}}]} ```