--- title: Decoding Strategies and Output Control | daily.dev description: A detailed walkthrough of how language models generate text through decoding strategies. Covers greedy decoding, temperature sampling, top-k and nucleus... canonical: https://daily.dev/posts/decoding-strategies-and-output-control-lshztkmvt twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Decoding Strategies and Output Control | daily.dev og:description: A detailed walkthrough of how language models generate text through decoding strategies. Covers greedy decoding, temperature sampling, top-k and nucleus... og:url: https://daily.dev/posts/decoding-strategies-and-output-control-lshztkmvt og:image: https://api.daily.dev/og/posts/lSHZtKMvT.png og:image:alt: Decoding Strategies and Output Control og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Decoding Strategies and Output Control **[Machine Learning Mastery](https://daily.dev/sources/mlm)** · 8 min read · 0 upvotes · 0 comments ## Summary A detailed walkthrough of how language models generate text through decoding strategies. Covers greedy decoding, temperature sampling, top-k and nucleus (top-p) sampling, repetition penalties, beam search, stop conditions, and structured output constraints. Each strategy is explained with code examples using Hugging Face Transformers and GPT-2, including trade-offs between determinism, diversity, quality, and inference cost. Constrained decoding for JSON and fixed-label outputs is also discussed, along with pointers to key research papers. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why does beam search often produce generic or bland text in open-ended chat generation? Beam search favors high-probability continuations at each step, which tends to produce generic, safe text rather than interesting or varied output. It works well when there is a well-defined sequence-level objective like translation in older sequence-to-sequence systems, but for open-ended chat it is usually avoided because of this bias plus its higher compute and memory cost from tracking multiple candidate sequences. _Comparing decoding methods for a chat product is easier with practical writeups like these surfaced on daily.dev._ ### What is the difference between top-k sampling and nucleus (top-p) sampling when generating text with a language model? Top-k sampling restricts token choices to a fixed number of the highest-scoring tokens, while nucleus (top-p) sampling instead selects the smallest set of tokens whose cumulative probability exceeds a threshold p, so the number of candidate tokens varies depending on how flat or sharp the distribution is at each step. Both are typically combined with temperature scaling before sampling. _Developers tuning generation quality can track sampling technique trade-offs through daily.dev._ ### Why can repetition penalties hurt output quality in LLM generation? Repetition penalties reduce the scores of already-generated tokens to discourage loops, but some content legitimately requires exact repetition, such as code, names, citations, and structured formats. Applying the penalty indiscriminately can break correctness in these cases, so it should only be used when repetition is actually a measured problem, and combining it with top-k or nucleus sampling changes its effect further. _Anyone debugging repetitive or broken LLM output can find grounded technique breakdowns on daily.dev._ ## Similar posts on daily.dev - [Generative AI, Part 1 — Decoding & Control: How to Make Models Write What You Want](https://daily.dev/posts/generative-ai-part-1-decoding-control-how-to-make-models-write-what-you-want-nybxojtvs) · C\# Corner · 0 upvotes · 0 comments - [The Machine Learning Practitioner’s Guide to Speculative Decoding](https://daily.dev/posts/the-machine-learning-practitioner-s-guide-to-speculative-decoding-3z4bou5te) · Machine Learning Mastery · 2 upvotes · 0 comments - [An Animated Guide to Large Language Models](https://daily.dev/posts/an-animated-guide-to-large-language-models-mp91hdghn) · Machine Learning Pills · 9 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#transformers](https://daily.dev/tags/transformers) [View this post on daily.dev](https://daily.dev/posts/decoding-strategies-and-output-control-lshztkmvt) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Decoding Strategies and Output Control","url":"https://daily.dev/posts/decoding-strategies-and-output-control-lshztkmvt","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/decoding-strategies-and-output-control-lshztkmvt"},"datePublished":"2026-08-03T14:41:55.256Z","dateModified":"2026-10-15T02:22:53.396Z","description":"A detailed walkthrough of how language models generate text through decoding strategies. Covers greedy decoding, temperature sampling, top-k and nucleus...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/a069d2c38106dbe55713e7d7d7107c99?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/a069d2c38106dbe55713e7d7d7107c99?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Machine Learning Mastery","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Machine Learning Mastery","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/795781ffb32446c39eba040feb64f505","url":"https://daily.dev/sources/mlm"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/decoding-strategies-and-output-control-lshztkmvt","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,transformers","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Machine Learning Mastery","item":"https://daily.dev/sources/mlm"},{"@type":"ListItem","position":3,"name":"Decoding Strategies and Output Control"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/decoding-strategies-and-output-control-lshztkmvt#faq","mainEntity":[{"@type":"Question","name":"Why does beam search often produce generic or bland text in open-ended chat generation?","acceptedAnswer":{"@type":"Answer","text":"Beam search favors high-probability continuations at each step, which tends to produce generic, safe text rather than interesting or varied output. It works well when there is a well-defined sequence-level objective like translation in older sequence-to-sequence systems, but for open-ended chat it is usually avoided because of this bias plus its higher compute and memory cost from tracking multiple candidate sequences. Comparing decoding methods for a chat product is easier with practical writeups like these surfaced on daily.dev."}},{"@type":"Question","name":"What is the difference between top-k sampling and nucleus (top-p) sampling when generating text with a language model?","acceptedAnswer":{"@type":"Answer","text":"Top-k sampling restricts token choices to a fixed number of the highest-scoring tokens, while nucleus (top-p) sampling instead selects the smallest set of tokens whose cumulative probability exceeds a threshold p, so the number of candidate tokens varies depending on how flat or sharp the distribution is at each step. Both are typically combined with temperature scaling before sampling. Developers tuning generation quality can track sampling technique trade-offs through daily.dev."}},{"@type":"Question","name":"Why can repetition penalties hurt output quality in LLM generation?","acceptedAnswer":{"@type":"Answer","text":"Repetition penalties reduce the scores of already-generated tokens to discourage loops, but some content legitimately requires exact repetition, such as code, names, citations, and structured formats. Applying the penalty indiscriminately can break correctness in these cases, so it should only be used when repetition is actually a measured problem, and combining it with top-k or nucleus sampling changes its effect further. Anyone debugging repetitive or broken LLM output can find grounded technique breakdowns on daily.dev."}}]} ```