--- title: The Soft Search That Taught Machines to Understand description: A conceptual walkthrough of the scaled dot-product attention mechanism that underpins modern LLMs like ChatGPT, Claude, and Gemini. Starting from a familiar... canonical: https://daily.dev/posts/the-soft-search-that-taught-machines-to-understand-vswbbvpnk twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: The Soft Search That Taught Machines to Understand | daily.dev og:description: A conceptual walkthrough of the scaled dot-product attention mechanism that underpins modern LLMs like ChatGPT, Claude, and Gemini. Starting from a familiar... og:url: https://daily.dev/posts/the-soft-search-that-taught-machines-to-understand-vswbbvpnk og:image: https://api.daily.dev/og/posts/VswbBvPNk.png og:image:alt: The Soft Search That Taught Machines to Understand og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # The Soft Search That Taught Machines to Understand **[Medium](https://daily.dev/sources/medium_js)** · 10 min read · 0 upvotes · 0 comments ## Summary A conceptual walkthrough of the scaled dot-product attention mechanism that underpins modern LLMs like ChatGPT, Claude, and Gemini. Starting from a familiar Python dictionary lookup, it builds up the intuition for why queries, keys, and values are separate learned projections, how dot products measure vector similarity, why softmax is used instead of argmax (gradient flow during training), and how the weighted blend of value vectors lets a model resolve ambiguity by drawing on multiple context sources simultaneously. The full Attention(Q,K,V) = softmax(QKᵀ/√dₖ)V formula is assembled step by step. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Multi-Head Attention Explained: Queries, Keys, and Values Made Simple](https://daily.dev/posts/multi-head-attention-explained-queries-keys-and-values-made-simple-6tda3mel9) · DigitalOcean Community · 0 upvotes · 0 comments - [The Q, K, V Matrices](https://daily.dev/posts/the-q-k-v-matrices-aeoio0pms) · Hacker News · 0 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#llm](https://daily.dev/tags/llm), [#nlp](https://daily.dev/tags/nlp) [View this post on daily.dev](https://daily.dev/posts/the-soft-search-that-taught-machines-to-understand-vswbbvpnk) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"The Soft Search That Taught Machines to Understand","url":"https://daily.dev/posts/the-soft-search-that-taught-machines-to-understand-vswbbvpnk","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/the-soft-search-that-taught-machines-to-understand-vswbbvpnk"},"datePublished":"2026-07-23T03:33:28.534Z","dateModified":"2026-07-23T03:34:58.406Z","description":"A conceptual walkthrough of the scaled dot-product attention mechanism that underpins modern LLMs like ChatGPT, Claude, and Gemini. Starting from a familiar...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/0d959432f711eca399d155b22cfb0c9c?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/0d959432f711eca399d155b22cfb0c9c?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Medium","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Medium","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/medium","url":"https://daily.dev/sources/medium_js"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/the-soft-search-that-taught-machines-to-understand-vswbbvpnk","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,llm,nlp","timeRequired":"PT10M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Medium","item":"https://daily.dev/sources/medium_js"},{"@type":"ListItem","position":3,"name":"The Soft Search That Taught Machines to Understand"}]} ```