--- title: Stanford's Prefix Sliding cuts long-reasoning inference... description: A Stanford paper introduces Prefix Sliding, a technique for long-reasoning LLM inference that keeps the fixed task prefix plus a sliding window of recent... canonical: https://daily.dev/posts/stanford-s-prefix-sliding-cuts-long-reasoning-inference-costs-by-3x-without-retraining-qkug5faso twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Stanford's Prefix Sliding cuts long-reasoning inference costs by 3x without retraining | daily.dev og:description: A Stanford paper introduces Prefix Sliding, a technique for long-reasoning LLM inference that keeps the fixed task prefix plus a sliding window of recent... og:url: https://daily.dev/posts/stanford-s-prefix-sliding-cuts-long-reasoning-inference-costs-by-3x-without-retraining-qkug5faso og:image: https://api.daily.dev/og/posts/qkuG5faSO.png og:image:alt: Stanford's Prefix Sliding cuts long-reasoning inference costs by 3x without retraining og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Stanford's Prefix Sliding cuts long-reasoning inference costs by 3x without retraining **[Collections](https://daily.dev/sources/collections)** · 2 min read · 3 upvotes · 1 comments ## Summary A Stanford paper introduces Prefix Sliding, a technique for long-reasoning LLM inference that keeps the fixed task prefix plus a sliding window of recent reasoning tokens, discarding stale middle tokens instead of applying full attention to the entire chain of thought. On Qwen3-1.7B, a 4,096-token window scored 33.9% on AIME25 versus 34.2% with full attention while running about 3x faster, with no retraining required. It also outperformed pure sliding windows, repeated summarization, and last-k deletion, and enabled RL rollouts past 100,000 tokens that full attention makes prohibitively expensive. ## Content A new Stanford paper called [Prefix Sliding](https://arxiv.org/abs/2506.26070) makes a simple but useful observation: when a model reasons for a long time, most of the middle tokens in that chain of thought barely matter. The problem it's solving is real. Long reasoning traces use full attention, meaning every new token looks back over the entire growing context. The harder the problem, the longer the trace, the more expensive each step becomes. Cost scales with difficulty in the worst possible way. The authors measured where attention actually goes during long reasoning. The answer: mostly to the prefix (task instructions, available tools) and to the most recent tokens. The intermediate reasoning steps in between get relatively little attention as generation continues. So Prefix Sliding just drops them. It keeps the fixed prefix intact and maintains a sliding window of recent tokens - a few thousand at most - discarding older middle tokens as new ones come in. Total memory stays capped regardless of how long the model reasons. The accuracy hit is minimal. On Qwen3-1.7B, a 4,096-token window scored 33.9% on AIME25 versus 34.2% with full attention. In exchange, inference runs about 3x faster with no retraining required. Each new token also has constant attention cost once the window fills, rather than growing more expensive with every step - which is what allowed the authors to run RL rollouts past 100,000 tokens. Compared to alternatives like pure sliding windows, repeated summarization, and last-k deletion, Prefix Sliding came out ahead on the speed/accuracy tradeoff. If you're running agents that do extended reasoning, this is worth reading. ## Questions this post answers ### What is Prefix Sliding and how does it reduce inference costs for long reasoning models? Prefix Sliding is a Stanford technique that keeps a model's fixed prefix (task instructions, tools) intact while maintaining only a sliding window of a few thousand recent tokens, discarding older intermediate reasoning tokens as generation continues. This caps memory regardless of reasoning length and yields roughly 3x faster inference with constant per-token attention cost, requiring no retraining of the underlying model. _Anyone optimizing agent inference costs can track techniques like prefix sliding through daily.dev._ ### Does dropping intermediate reasoning tokens hurt accuracy on math benchmarks like AIME25? The accuracy hit is minimal. On Qwen3-1.7B, using a 4,096-token sliding window scored 33.9% on AIME25 compared to 34.2% with full attention over the entire reasoning trace, a difference of only 0.3 percentage points while running about 3x faster. _Developers weighing speed against accuracy tradeoffs in reasoning models can follow benchmarks like this on daily.dev._ ### How does Prefix Sliding compare to other methods for managing long context in reasoning models? Prefix Sliding outperformed pure sliding windows, repeated summarization, and last-k token deletion on the speed/accuracy tradeoff for long chain-of-thought reasoning. Its advantage comes from preserving the fixed prefix (instructions and tools) separately from the sliding window of recent tokens, rather than treating all context uniformly. _daily.dev helps engineers comparing context-management strategies for long-running reasoning agents stay current._ ## Community discussion Top comments from developers on daily.dev. **@trevorsuna** · 2 upvotes > Keeping the task prefix and recent reasoning while dropping the low-value middle is a neat way to cap attention cost. The AIME result is promising; I’d also test tool-using agents and long debugging sessions, where an older observation may look unimportant until a later step needs it. ## Similar posts on daily.dev - [How Production LLMs Reason Better At Inference Time](https://daily.dev/posts/how-production-llms-reason-better-at-inference-time-weygdosue) · Daily Dose of Data Science \| Avi Chawla \| Substack · 1 upvotes · 0 comments - [Sliding Window Attention: Efficient Long-Context Modeling](https://daily.dev/posts/sliding-window-attention-efficient-long-context-modeling-wjssh8o3w) · DigitalOcean Community · 2 upvotes · 0 comments --- Tags: [#reinforcement-learning](https://daily.dev/tags/reinforcement-learning), [#qwen](https://daily.dev/tags/qwen) [View this post on daily.dev](https://daily.dev/posts/stanford-s-prefix-sliding-cuts-long-reasoning-inference-costs-by-3x-without-retraining-qkug5faso) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Stanford's Prefix Sliding cuts long-reasoning inference costs by 3x without retraining","url":"https://daily.dev/posts/stanford-s-prefix-sliding-cuts-long-reasoning-inference-costs-by-3x-without-retraining-qkug5faso","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/stanford-s-prefix-sliding-cuts-long-reasoning-inference-costs-by-3x-without-retraining-qkug5faso"},"datePublished":"2026-09-04T01:38:13.862Z","dateModified":"2026-09-13T19:33:54.088Z","description":"A Stanford paper introduces Prefix Sliding, a technique for long-reasoning LLM inference that keeps the fixed task prefix plus a sliding window of recent...","image":"https://pbs.twimg.com/media/HRVgihNaoAA8BXC.jpg","thumbnailUrl":"https://pbs.twimg.com/media/HRVgihNaoAA8BXC.jpg","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/stanford-s-prefix-sliding-cuts-long-reasoning-inference-costs-by-3x-without-retraining-qkug5faso","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":3},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"reinforcement-learning,qwen","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"Stanford's Prefix Sliding cuts long-reasoning inference costs by 3x without retraining"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/stanford-s-prefix-sliding-cuts-long-reasoning-inference-costs-by-3x-without-retraining-qkug5faso","comment":[{"@type":"Comment","text":"Keeping the task prefix and recent reasoning while dropping the low-value middle is a neat way to cap attention cost. The AIME result is promising; I’d also test tool-using agents and long debugging sessions, where an older observation may look unimportant until a later step needs it.","datePublished":"2026-09-04T05:59:24.815Z","url":"https://daily.dev/posts/qkuG5faSO#c-VxL0STLHV","author":{"@type":"Person","name":"Trevor Suna","url":"https://daily.dev/trevorsuna","image":"https://media.daily.dev/image/upload/s--dZ7gXxpp--/f_auto/v1784081551/avatars/avatar_EMoP47rpuw8DNjhp6R1b6?_a=BAMAMicg0"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/stanford-s-prefix-sliding-cuts-long-reasoning-inference-costs-by-3x-without-retraining-qkug5faso#faq","mainEntity":[{"@type":"Question","name":"What is Prefix Sliding and how does it reduce inference costs for long reasoning models?","acceptedAnswer":{"@type":"Answer","text":"Prefix Sliding is a Stanford technique that keeps a model's fixed prefix (task instructions, tools) intact while maintaining only a sliding window of a few thousand recent tokens, discarding older intermediate reasoning tokens as generation continues. This caps memory regardless of reasoning length and yields roughly 3x faster inference with constant per-token attention cost, requiring no retraining of the underlying model. Anyone optimizing agent inference costs can track techniques like prefix sliding through daily.dev."}},{"@type":"Question","name":"Does dropping intermediate reasoning tokens hurt accuracy on math benchmarks like AIME25?","acceptedAnswer":{"@type":"Answer","text":"The accuracy hit is minimal. On Qwen3-1.7B, using a 4,096-token sliding window scored 33.9% on AIME25 compared to 34.2% with full attention over the entire reasoning trace, a difference of only 0.3 percentage points while running about 3x faster. Developers weighing speed against accuracy tradeoffs in reasoning models can follow benchmarks like this on daily.dev."}},{"@type":"Question","name":"How does Prefix Sliding compare to other methods for managing long context in reasoning models?","acceptedAnswer":{"@type":"Answer","text":"Prefix Sliding outperformed pure sliding windows, repeated summarization, and last-k token deletion on the speed/accuracy tradeoff for long chain-of-thought reasoning. Its advantage comes from preserving the fixed prefix (instructions and tools) separately from the sliding window of recent tokens, rather than treating all context uniformly. daily.dev helps engineers comparing context-management strategies for long-running reasoning agents stay current."}}]} ```