> ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. --- title: "How LLMs Actually Work" url: https://daily.dev/agentic-ai-hub/how-llms-actually-work/ description: "A large language model predicts the next token given the tokens so far, and that is essentially all it does." lastUpdated: "2026-07-22" --- A large language model predicts the next token given the tokens so far, and that is essentially all it does. Everything else, including writing code, planning a trip, and passing the bar exam, is emergent behavior that falls out of getting extremely good at that single objective over a large enough model and corpus. **Next-token prediction.** The model consumes a sequence of tokens and outputs a probability distribution over what comes next across its entire vocabulary. Sampling from that distribution (greedily, or with temperature/top-p randomness) gives you one token. Append it, feed the whole thing back in, and repeat. Fluent paragraphs are just this loop run hundreds of times. The important intuition is that the model has no plan for the sentence it's about to write. It commits one token at a time, and its apparent foresight comes from having learned, during training, what well-formed continuations look like. This is also why telling a model to "think step by step" used to help so much. You were handing it more tokens to reason over before it had to commit. Today's reasoning models do that internally by default, so the explicit instruction matters much less than it once did (more in [Prompting & Context Engineering](/agentic-ai-hub/prompting-context-engineering/)). **Training vs. inference.** These are two different regimes. *Training* is where the weights change. The model sees text, predicts the next token, is scored against the actual next token, and its billions of parameters are nudged via gradient descent to be a little less wrong. This happens once (well, in expensive batches) and costs enormous compute. *Inference* is what happens every time you send a prompt. The weights are frozen, and the model just runs the forward pass to produce tokens. When people say a model "learned" something from your conversation, they're usually wrong. Nothing about your chat changed the weights. The model only "knows" your conversation because it's sitting in the context window (more on that below). **Pre-training → post-training.** A modern assistant is built in stages: - **Pre-training** is the giant next-token-prediction run over a broad slice of the internet, books, and code. It produces a *base model*, a raw next-token predictor that's shockingly knowledgeable but not helpful. Ask it a question and it might continue with more questions, because that's a plausible continuation of text on the web. - **Supervised fine-tuning (SFT)** teaches the base model to behave like an assistant by training it on curated examples of instructions paired with good responses. This is where "answer the question" becomes the default behavior. - **RLHF (Reinforcement Learning from Human Feedback)** refines it further using human preference judgments. Humans (or a model trained to imitate them) rank candidate responses. A reward model learns those preferences, and the LLM is optimized to produce responses the reward model scores highly. RLHF is why models are polite, hedge appropriately, and refuse harmful requests. **RLAIF (Reinforcement Learning from AI Feedback)** swaps human labelers for an AI labeler (often guided by a written "constitution" of principles) to scale up the preference data cheaply. [Anthropic's Constitutional AI work](https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback) is the canonical example. - **DPO (Direct Preference Optimization)** is a more recent, simpler alternative to the full RLHF pipeline: it optimizes the model directly on preference pairs without training a separate reward model or running RL, which makes it cheaper and more stable to tune. See the [original DPO paper](https://arxiv.org/abs/2305.18290). - **GRPO and RLVR (the 2026 reasoning wave).** The newest post-training leans on reinforcement learning against *verifiable* rewards instead of human preference. **RLVR** (RL from Verifiable Rewards) trains on signals a computer can check automatically, like whether the tests pass or the math is correct, and **GRPO** (Group Relative Policy Optimization, popularized by DeepSeek) is a cheaper RL algorithm that drops the separate value model older methods needed. Together they are the engine behind today's reasoning models. See [this post-training overview](https://huggingface.co/blog/karina-zadorozhny/guide-to-llm-post-training-algorithms). Collectively, SFT + preference optimization is **post-training**, the phase that turns a knowledgeable predictor into a usable assistant. A huge fraction of a model's "personality" and instruction-following quality is decided here, not in pre-training. **What "reasoning" / test-time-compute models do differently.** A newer class of models (OpenAI's o-series, DeepSeek-R1, Claude's extended-thinking modes, Gemini's thinking variants) is trained to spend more compute *at inference time* by generating a long internal chain of thought before answering. Rather than committing to an answer immediately, they produce (often hidden) intermediate reasoning tokens that explore, check, and backtrack, and this measurably improves performance on math, coding, and multi-step logic. The key shift is economic. Instead of only scaling training compute, you scale *test-time* compute, trading latency and tokens for accuracy. These models are typically trained with reinforcement learning against verifiable rewards (did the code pass? is the math correct?), which is why they excel where answers can be checked. The practical takeaway is to use them when a problem genuinely decomposes into steps, and to expect them to be slower and pricier per query. See [Core Concepts](/agentic-ai-hub/core-concepts/) for when reasoning is worth it. **Why context ≠ memory.** This trips up nearly everyone. The context window is working memory that exists *only for the duration of a single request*. When you have a long chat, the interface re-sends the accumulated transcript on every turn. The model isn't remembering, it's re-reading. Close the session, or overflow the window, and it's gone. Persistent memory (a model that recalls you across sessions) is an *engineering layer* on top. Your app stores facts in a database or vector store and re-injects them into context. The model itself is stateless between calls. Internalize this and a lot of behavior stops being mysterious: why the model "forgets" something from 50 messages ago (it fell off the front of the window), why it can't recall your last conversation (nothing persisted it), and why repeating context helps. **Tokenization intuition.** Models don't see characters or words. They see *tokens*, subword chunks produced by a tokenizer (typically a byte-pair-encoding scheme). Common English words are often one token. Rarer words split into pieces ("tokenization" → `token` + `ization`), and whitespace and punctuation carry tokens too. Rough rule of thumb for English: **~4 characters or ~0.75 words per token**, so 1,000 tokens ≈ 750 words. This has real consequences. Token boundaries are why models historically fumbled "how many r's in strawberry" (the word is a couple of opaque tokens, not individual letters). It's why code and non-English languages can be less token-efficient (more tokens per character means more cost and less effective context). And it's why every pricing and context-limit number is quoted in tokens, not words. When you reason about cost and context budgets, think in tokens.