--- title: How Language Models Pick Words (Softmax & Temperature) description: A detailed walkthrough of how language models select the next token using softmax and temperature. Starting from Markov chains as an intuitive analogy, the... canonical: https://daily.dev/posts/how-language-models-pick-words-softmax-temperature--fbpaf9mbq twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How Language Models Pick Words (Softmax & Temperature) | daily.dev og:description: A detailed walkthrough of how language models select the next token using softmax and temperature. Starting from Markov chains as an intuitive analogy, the... og:url: https://daily.dev/posts/how-language-models-pick-words-softmax-temperature--fbpaf9mbq og:image: https://api.daily.dev/og/posts/FbpAF9mBQ.png og:image:alt: How Language Models Pick Words (Softmax & Temperature) og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How Language Models Pick Words (Softmax & Temperature) **[The Coding Train](https://daily.dev/sources/thecodingtrain)** · 35 min read · 4 upvotes · 0 comments ## Summary A detailed walkthrough of how language models select the next token using softmax and temperature. Starting from Markov chains as an intuitive analogy, the video explains how raw neural network outputs (logits) are converted to probabilities via the softmax function, and how dividing the exponent by a temperature value squashes or stretches those probabilities. Low temperature makes the model more deterministic (top token dominates), while high temperature flattens probabilities toward uniform randomness. The concepts are demonstrated with p5.js code, a Markov chain text generator, and a live transformers.js language model. Also briefly covers related sampling strategies: argmax, top-K, and top-P. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What does the temperature parameter do when sampling from a language model? Temperature adjusts how flat or peaked the probability distribution is before sampling the next token. A temperature of one leaves probabilities unchanged, a temperature above one flattens the distribution making unlikely tokens more likely (more random output), and a temperature below one sharpens it, making the highest-probability token even more dominant. It works by dividing the logits by the temperature value before applying the exponential in softmax. _Developers tuning LLM output randomness can dig deeper into sampling parameters on daily.dev._ ### What is the difference between top-k and top-p sampling in text generation? Top-k sampling restricts the next-token choice to only the k highest-probability tokens before sampling, so a k of one is equivalent to greedy arg-max selection. Top-p (nucleus) sampling instead picks the smallest set of top tokens whose cumulative probability exceeds a threshold p, so a top-p of 50% only includes tokens until their combined probability crosses that 50% mark. _Choosing between top-k, top-p, and temperature for text generation is easier with resources on daily.dev._ ### Why does a language model give the same output every time when using transformers.js text generation by default? By default, the transformers.js text generation pipeline does not perform sampling and instead uses arg-max, always picking the single token with the highest probability, which is equivalent to a temperature of zero. To get varied outputs on repeated calls, the do_sample property must be set to true in the generate function, optionally combined with a temperature value. _Anyone debugging deterministic vs. varied LLM output can find practical fixes on daily.dev._ ## Similar posts on daily.dev - [The Statistics of Token Selection: Logits, Temperature, and Top-P Walkthrough](https://daily.dev/posts/the-statistics-of-token-selection-logits-temperature-and-top-p-walkthrough-iibdwjgtw) · Machine Learning Mastery · 1 upvotes · 0 comments - [How LLMs Choose Their Words: A Practical Walk-Through of Logits, Softmax and Sampling](https://daily.dev/posts/how-llms-choose-their-words-a-practical-walk-through-of-logits-softmax-and-sampling-cu0ugeeus) · Machine Learning Mastery · 2 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#javascript](https://daily.dev/tags/javascript), [#llm](https://daily.dev/tags/llm) [View this post on daily.dev](https://daily.dev/posts/how-language-models-pick-words-softmax-temperature--fbpaf9mbq) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How Language Models Pick Words (Softmax & Temperature)","url":"https://daily.dev/posts/how-language-models-pick-words-softmax-temperature--fbpaf9mbq","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-language-models-pick-words-softmax-temperature--fbpaf9mbq"},"datePublished":"2026-08-03T16:31:33.272Z","dateModified":"2026-09-13T19:30:39.955Z","description":"A detailed walkthrough of how language models select the next token using softmax and temperature. Starting from Markov chains as an intuitive analogy, the...","image":"https://i.ytimg.com/vi/t_QuHSwB0HY/sddefault.jpg","thumbnailUrl":"https://i.ytimg.com/vi/t_QuHSwB0HY/sddefault.jpg","isAccessibleForFree":true,"articleSection":"The Coding Train","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"The Coding Train","logo":"https://media.daily.dev/image/upload/s--V7pjhVEY--/f_auto/v1716024620/logos/thecodingtrain","url":"https://daily.dev/sources/thecodingtrain"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-language-models-pick-words-softmax-temperature--fbpaf9mbq","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":4},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,javascript,llm","timeRequired":"PT35M","video":{"@type":"VideoObject","name":"How Language Models Pick Words (Softmax & Temperature)","description":"A detailed walkthrough of how language models select the next token using softmax and temperature. Starting from Markov chains as an intuitive analogy, the...","thumbnailUrl":"https://i.ytimg.com/vi/t_QuHSwB0HY/sddefault.jpg","uploadDate":"2026-08-03T16:31:33.272Z","duration":"PT35M","url":"https://api.daily.dev/r/FbpAF9mBQ","embedUrl":"https://www.youtube.com/embed/t_QuHSwB0HY"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"The Coding Train","item":"https://daily.dev/sources/thecodingtrain"},{"@type":"ListItem","position":3,"name":"How Language Models Pick Words (Softmax & Temperature)"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-language-models-pick-words-softmax-temperature--fbpaf9mbq#faq","mainEntity":[{"@type":"Question","name":"What does the temperature parameter do when sampling from a language model?","acceptedAnswer":{"@type":"Answer","text":"Temperature adjusts how flat or peaked the probability distribution is before sampling the next token. A temperature of one leaves probabilities unchanged, a temperature above one flattens the distribution making unlikely tokens more likely (more random output), and a temperature below one sharpens it, making the highest-probability token even more dominant. It works by dividing the logits by the temperature value before applying the exponential in softmax. Developers tuning LLM output randomness can dig deeper into sampling parameters on daily.dev."}},{"@type":"Question","name":"What is the difference between top-k and top-p sampling in text generation?","acceptedAnswer":{"@type":"Answer","text":"Top-k sampling restricts the next-token choice to only the k highest-probability tokens before sampling, so a k of one is equivalent to greedy arg-max selection. Top-p (nucleus) sampling instead picks the smallest set of top tokens whose cumulative probability exceeds a threshold p, so a top-p of 50% only includes tokens until their combined probability crosses that 50% mark. Choosing between top-k, top-p, and temperature for text generation is easier with resources on daily.dev."}},{"@type":"Question","name":"Why does a language model give the same output every time when using transformers.js text generation by default?","acceptedAnswer":{"@type":"Answer","text":"By default, the transformers.js text generation pipeline does not perform sampling and instead uses arg-max, always picking the single token with the highest probability, which is equivalent to a temperature of zero. To get varied outputs on repeated calls, the do_sample property must be set to true in the generate function, optionally combined with a temperature value. Anyone debugging deterministic vs. varied LLM output can find practical fixes on daily.dev."}}]} ```