--- title: What is Paged Attention? | daily.dev description: PagedAttention solves the GPU memory waste problem in LLM inference by applying operating system virtual memory paging concepts to the KV cache. Instead of... canonical: https://daily.dev/posts/what-is-paged-attention--dyjrqpqku twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: What is Paged Attention? | daily.dev og:description: PagedAttention solves the GPU memory waste problem in LLM inference by applying operating system virtual memory paging concepts to the KV cache. Instead of... og:url: https://daily.dev/posts/what-is-paged-attention--dyjrqpqku og:image: https://api.daily.dev/og/posts/DYjrqPQku.png og:image:alt: What is Paged Attention? og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # What is Paged Attention? **[Medium](https://daily.dev/sources/medium_js)** · 12 min read · 1 upvotes · 0 comments ## Summary PagedAttention solves the GPU memory waste problem in LLM inference by applying operating system virtual memory paging concepts to the KV cache. Instead of pre-allocating contiguous memory blocks sized for worst-case sequence length (causing 60-80% waste from internal/external fragmentation), it divides the KV cache into fixed-size blocks tracked via a block table, allocating them on-demand. It also enables memory sharing across sequences via copy-on-write for beam search and parallel sampling. This technique, pioneered in the vLLM paper (Kwon et al., 2023, UC Berkeley), delivers 2-4x throughput gains and near-zero memory waste, and now powers vLLM and similar inference engines. The piece includes analogies, diagrams, and simplified Python pseudocode for a block allocator. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is PagedAttention and why does it improve LLM inference memory usage? PagedAttention is a memory management technique for the KV cache in LLM inference that borrows virtual memory paging from operating systems. It splits the KV cache into fixed-size blocks tracked via a block table instead of pre-allocating contiguous memory for a worst-case sequence length. This eliminates most internal and external fragmentation, which traditional systems suffered at rates of 60-80% wasted memory. _Developers optimizing LLM serving costs follow deep dives like this on daily.dev to keep GPU memory usage efficient._ ### How much throughput improvement does vLLM's PagedAttention provide over Hugging Face Transformers or FasterTransformer? vLLM with PagedAttention delivers roughly 2x to 4x higher throughput on the same GPU compared to systems like Hugging Face Transformers or FasterTransformer, according to the original vLLM paper and community benchmarks (Kwon et al., 2023, UC Berkeley). It also achieves near-zero memory waste versus 60-80% waste in naive pre-allocation systems. _Teams choosing an LLM inference engine compare throughput numbers like these via daily.dev before committing to a stack._ ### How does PagedAttention share memory across parallel sampling or beam search requests? PagedAttention lets multiple sequences point their block tables to the same physical KV cache blocks for a shared prompt prefix, avoiding duplication. Only when a sequence generates tokens that diverge from the others does it get a new block, a technique called copy-on-write, directly modeled on the fork() mechanism in Unix processes. _Engineers tuning beam search or parallel sampling setups track memory-saving techniques like this on daily.dev._ ## Similar posts on daily.dev - [Paged Attention in LLMs](https://daily.dev/posts/paged-attention-in-llms-2bvrxmoiy) · Daily Dose of Data Science \| Avi Chawla \| Substack · 0 upvotes · 0 comments - [vLLM Teardown Part 2: The Virtual Memory for LLMs](https://daily.dev/posts/vllm-teardown-part-2-the-virtual-memory-for-llms-uzjhgrxwa) · Medium · 0 upvotes · 0 comments - [The KV Cache Tax: Why Inference Servers Run Out of Memory Before Compute](https://daily.dev/posts/the-kv-cache-tax-why-inference-servers-run-out-of-memory-before-compute-yzoonznpj) · Towards Data Science · 0 upvotes · 0 comments - [Why An LLM’s Memory Gets Expensive and How to Fix It](https://daily.dev/posts/why-an-llm-s-memory-gets-expensive-and-how-to-fix-it-tfmmfttjf) · ByteByteGo · 2 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#ai-inference](https://daily.dev/tags/ai-inference), [#vllm](https://daily.dev/tags/vllm) [View this post on daily.dev](https://daily.dev/posts/what-is-paged-attention--dyjrqpqku) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"What is Paged Attention?","url":"https://daily.dev/posts/what-is-paged-attention--dyjrqpqku","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/what-is-paged-attention--dyjrqpqku"},"datePublished":"2026-08-17T23:02:06.322Z","dateModified":"2026-09-13T21:58:44.268Z","description":"PagedAttention solves the GPU memory waste problem in LLM inference by applying operating system virtual memory paging concepts to the KV cache. Instead of...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f1c92a53fb07a02979c67e59bac6c329?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f1c92a53fb07a02979c67e59bac6c329?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Medium","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Medium","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/medium","url":"https://daily.dev/sources/medium_js"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/what-is-paged-attention--dyjrqpqku","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,ai-inference,vllm","timeRequired":"PT12M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Medium","item":"https://daily.dev/sources/medium_js"},{"@type":"ListItem","position":3,"name":"What is Paged Attention?"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/what-is-paged-attention--dyjrqpqku#faq","mainEntity":[{"@type":"Question","name":"What is PagedAttention and why does it improve LLM inference memory usage?","acceptedAnswer":{"@type":"Answer","text":"PagedAttention is a memory management technique for the KV cache in LLM inference that borrows virtual memory paging from operating systems. It splits the KV cache into fixed-size blocks tracked via a block table instead of pre-allocating contiguous memory for a worst-case sequence length. This eliminates most internal and external fragmentation, which traditional systems suffered at rates of 60-80% wasted memory. Developers optimizing LLM serving costs follow deep dives like this on daily.dev to keep GPU memory usage efficient."}},{"@type":"Question","name":"How much throughput improvement does vLLM's PagedAttention provide over Hugging Face Transformers or FasterTransformer?","acceptedAnswer":{"@type":"Answer","text":"vLLM with PagedAttention delivers roughly 2x to 4x higher throughput on the same GPU compared to systems like Hugging Face Transformers or FasterTransformer, according to the original vLLM paper and community benchmarks (Kwon et al., 2023, UC Berkeley). It also achieves near-zero memory waste versus 60-80% waste in naive pre-allocation systems. Teams choosing an LLM inference engine compare throughput numbers like these via daily.dev before committing to a stack."}},{"@type":"Question","name":"How does PagedAttention share memory across parallel sampling or beam search requests?","acceptedAnswer":{"@type":"Answer","text":"PagedAttention lets multiple sequences point their block tables to the same physical KV cache blocks for a shared prompt prefix, avoiding duplication. Only when a sequence generates tokens that diverge from the others does it get a new block, a technique called copy-on-write, directly modeled on the fork() mechanism in Unix processes. Engineers tuning beam search or parallel sampling setups track memory-saving techniques like this on daily.dev."}}]} ```