--- title: Why does the same prompt at temperature 0 return... description: When you send the same prompt to an LLM at temperature 0, you might still get different outputs — not because of floating-point randomness, but because of... canonical: https://daily.dev/posts/why-does-the-same-prompt-at-temperature-0-return-different-answers--crimw9jlj twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Why does the same prompt at temperature 0 return different answers? | daily.dev og:description: When you send the same prompt to an LLM at temperature 0, you might still get different outputs — not because of floating-point randomness, but because of... og:url: https://daily.dev/posts/why-does-the-same-prompt-at-temperature-0-return-different-answers--crimw9jlj og:image: https://api.daily.dev/og/posts/CrImw9JLJ.png og:image:alt: Why does the same prompt at temperature 0 return different answers? og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Why does the same prompt at temperature 0 return different answers? **[Medium](https://daily.dev/sources/medium_js)** · 8 min read · 0 upvotes · 0 comments ## Summary When you send the same prompt to an LLM at temperature 0, you might still get different outputs — not because of floating-point randomness, but because of batch size variation caused by other users sharing the same GPU server. The GPU selects different computation strategies (reduction kernels) depending on batch size, leading to different rounding at different moments, which can flip a single token and cascade into entirely different outputs. Research shows this can cause up to 9 percentage point accuracy swings and 9,000-token length differences on the same model. Practical mitigations include reporting match rates across multiple eval runs, avoiding exact string equality in tests, freezing hardware/engine configs for published results, using deterministic kernels (at ~33% throughput cost) for self-hosted setups, and re-running evals before acting on apparent regressions. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [How Does Prompt Caching Work and When Does It Actually Cut LLM Costs?](https://daily.dev/posts/how-does-prompt-caching-work-and-when-does-it-actually-cut-llm-costs--vnardpi3a) · DigitalOcean Community · 1 upvotes · 0 comments - [6 settings I always change before running a local LLM](https://daily.dev/posts/6-settings-i-always-change-before-running-a-local-llm-4x3sb5ebb) · XDA Developers · 0 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#gpu](https://daily.dev/tags/gpu), [#mlops](https://daily.dev/tags/mlops), [#ai-inference](https://daily.dev/tags/ai-inference) [View this post on daily.dev](https://daily.dev/posts/why-does-the-same-prompt-at-temperature-0-return-different-answers--crimw9jlj) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Why does the same prompt at temperature 0 return different answers?","url":"https://daily.dev/posts/why-does-the-same-prompt-at-temperature-0-return-different-answers--crimw9jlj","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/why-does-the-same-prompt-at-temperature-0-return-different-answers--crimw9jlj"},"datePublished":"2026-08-03T13:14:44.394Z","dateModified":"2026-08-03T13:15:47.258Z","description":"When you send the same prompt to an LLM at temperature 0, you might still get different outputs — not because of floating-point randomness, but because of...","image":"https://media.daily.dev/image/upload/s--qPvKM23u--/f_auto/v1722860399/public/Placeholder%2009","thumbnailUrl":"https://media.daily.dev/image/upload/s--qPvKM23u--/f_auto/v1722860399/public/Placeholder%2009","isAccessibleForFree":true,"articleSection":"Medium","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Medium","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/medium","url":"https://daily.dev/sources/medium_js"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/why-does-the-same-prompt-at-temperature-0-return-different-answers--crimw9jlj","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,gpu,mlops,ai-inference","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Medium","item":"https://daily.dev/sources/medium_js"},{"@type":"ListItem","position":3,"name":"Why does the same prompt at temperature 0 return different answers?"}]} ```