--- title: Speculative Decoding on CPUs: Nearly 4x Faster Token... description: DFlash speculative decoding support for CPUs, newly enabled in vLLM v0.25.0, delivers nearly 4x faster token generation throughput compared to autoregressive... canonical: https://daily.dev/posts/speculative-decoding-on-cpus-nearly-4x-faster-token-generation-with-dflash-suiryhqim twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Speculative Decoding on CPUs: Nearly 4x Faster Token Generation with DFlash | daily.dev og:description: DFlash speculative decoding support for CPUs, newly enabled in vLLM v0.25.0, delivers nearly 4x faster token generation throughput compared to autoregressive... og:url: https://daily.dev/posts/speculative-decoding-on-cpus-nearly-4x-faster-token-generation-with-dflash-suiryhqim og:image: https://api.daily.dev/og/posts/SUiryHqim.png og:image:alt: Speculative Decoding on CPUs: Nearly 4x Faster Token Generation with DFlash og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Speculative Decoding on CPUs: Nearly 4x Faster Token Generation with DFlash **[Towards Data Science](https://daily.dev/sources/tds)** · 9 min read · 0 upvotes · 0 comments ## Summary DFlash speculative decoding support for CPUs, newly enabled in vLLM v0.25.0, delivers nearly 4x faster token generation throughput compared to autoregressive decoding. Testing with Qwen3.5-9B on an AWS r8i instance powered by Intel Xeon 6 processors showed a 3.92x average speedup and 74% cost reduction per token at concurrency 1 across HumanEval, GSM8K, and MT-Bench datasets. The piece explains how DFlash's block-diffusion drafter and target KV injection improve draft quality, walks through acceptance rate and acceptance length metrics, provides a vLLM Docker configuration example, and discusses how to tune num_speculative_tokens for optimal throughput. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What speedup does DFlash speculative decoding give for LLM inference on CPUs with vLLM? DFlash delivers an average 3.92x throughput increase over autoregressive decoding when running Qwen3.5-9B on an Intel Xeon 6 processor (AWS r8i.16xlarge instance) at concurrency 1, translating to a 74.4% cost reduction per generated token. Results varied by dataset: 4.16x on GSM8K, 3.99x on HumanEval, and 3.61x on MT-Bench, with acceptance lengths between 6.01 and 7.08 tokens per round. _Engineers tuning inference costs can follow ongoing coverage of speculative decoding techniques on daily.dev._ ### How do I enable DFlash speculative decoding in vLLM on CPU? Run the vLLM CPU Docker image with a speculative-config flag specifying method 'dflash', the drafter model (such as z-lab/Qwen3.5-9B-DFlash), and num_speculative_tokens (e.g., 15). DFlash CPU support was added in vLLM v0.25.0, requiring VLLM_TARGET_DEVICE=cpu and appropriate KV cache space settings alongside the target model like Qwen3.5-9B. _Developers configuring vLLM speculative decoding can track setup details and gotchas on daily.dev._ ### What is the difference between acceptance rate and acceptance length in speculative decoding, and which should I use to tune block size? Acceptance length, not acceptance rate, should be used to compare progress per verification round because acceptance rate is an average of per-position values that falls as block size grows even when throughput improves. Acceptance length equals accepted draft tokens per round plus one target token; for example, a HumanEval run showed acceptance rate 39.56% but acceptance length 6.93. Optimal block size should be chosen from measured throughput or latency benchmarks, not from acceptance rate alone. _Practitioners benchmarking speculative decoding parameters can follow deeper technical breakdowns on daily.dev._ ## Similar posts on daily.dev - [Researchers Found a Way to Make LLMs 8.5x Faster\!](https://daily.dev/posts/researchers-found-a-way-to-make-llms-8-5x-faster--gy4vk6zg4) · Daily Dose of Data Science \| Avi Chawla \| Substack · 0 upvotes · 0 comments - [Supercharging LLM inference on Google TPUs: Achieving 3X speedups with diffusion-style speculative decoding](https://daily.dev/posts/supercharging-llm-inference-on-google-tpus-achieving-3x-speedups-with-diffusion-style-speculative-d-jiflbzrv8) · Google Developers · 0 upvotes · 0 comments - [Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding](https://daily.dev/posts/boost-inference-performance-up-to-15x-on-nvidia-blackwell-using-dflash-speculative-decoding-yeogdqdwe) · NVIDIA Developer · 0 upvotes · 0 comments - [Parallel All the Way Down: Beyond Single-Token Generation with Speculative Decoding](https://daily.dev/posts/parallel-all-the-way-down-beyond-single-token-generation-with-speculative-decoding-yzx2fn2fo) · vLLM · 0 upvotes · 0 comments --- Tags: [#ai-inference](https://daily.dev/tags/ai-inference), [#vllm](https://daily.dev/tags/vllm) [View this post on daily.dev](https://daily.dev/posts/speculative-decoding-on-cpus-nearly-4x-faster-token-generation-with-dflash-suiryhqim) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Speculative Decoding on CPUs: Nearly 4x Faster Token Generation with DFlash","url":"https://daily.dev/posts/speculative-decoding-on-cpus-nearly-4x-faster-token-generation-with-dflash-suiryhqim","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/speculative-decoding-on-cpus-nearly-4x-faster-token-generation-with-dflash-suiryhqim"},"datePublished":"2026-08-24T14:12:20.590Z","dateModified":"2026-08-24T14:12:49.384Z","description":"DFlash speculative decoding support for CPUs, newly enabled in vLLM v0.25.0, delivers nearly 4x faster token generation throughput compared to autoregressive...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/b9a7bd2a6d6654e2b1e4df1c5fe39e7f?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/b9a7bd2a6d6654e2b1e4df1c5fe39e7f?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Towards Data Science","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Towards Data Science","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/tds","url":"https://daily.dev/sources/tds"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/speculative-decoding-on-cpus-nearly-4x-faster-token-generation-with-dflash-suiryhqim","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-inference,vllm","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Towards Data Science","item":"https://daily.dev/sources/tds"},{"@type":"ListItem","position":3,"name":"Speculative Decoding on CPUs: Nearly 4x Faster Token Generation with DFlash"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/speculative-decoding-on-cpus-nearly-4x-faster-token-generation-with-dflash-suiryhqim#faq","mainEntity":[{"@type":"Question","name":"What speedup does DFlash speculative decoding give for LLM inference on CPUs with vLLM?","acceptedAnswer":{"@type":"Answer","text":"DFlash delivers an average 3.92x throughput increase over autoregressive decoding when running Qwen3.5-9B on an Intel Xeon 6 processor (AWS r8i.16xlarge instance) at concurrency 1, translating to a 74.4% cost reduction per generated token. Results varied by dataset: 4.16x on GSM8K, 3.99x on HumanEval, and 3.61x on MT-Bench, with acceptance lengths between 6.01 and 7.08 tokens per round. Engineers tuning inference costs can follow ongoing coverage of speculative decoding techniques on daily.dev."}},{"@type":"Question","name":"How do I enable DFlash speculative decoding in vLLM on CPU?","acceptedAnswer":{"@type":"Answer","text":"Run the vLLM CPU Docker image with a speculative-config flag specifying method 'dflash', the drafter model (such as z-lab/Qwen3.5-9B-DFlash), and num_speculative_tokens (e.g., 15). DFlash CPU support was added in vLLM v0.25.0, requiring VLLM_TARGET_DEVICE=cpu and appropriate KV cache space settings alongside the target model like Qwen3.5-9B. Developers configuring vLLM speculative decoding can track setup details and gotchas on daily.dev."}},{"@type":"Question","name":"What is the difference between acceptance rate and acceptance length in speculative decoding, and which should I use to tune block size?","acceptedAnswer":{"@type":"Answer","text":"Acceptance length, not acceptance rate, should be used to compare progress per verification round because acceptance rate is an average of per-position values that falls as block size grows even when throughput improves. Acceptance length equals accepted draft tokens per round plus one target token; for example, a HumanEval run showed acceptance rate 39.56% but acceptance length 6.93. Optimal block size should be chosen from measured throughput or latency benchmarks, not from acceptance rate alone. Practitioners benchmarking speculative decoding parameters can follow deeper technical breakdowns on daily.dev."}}]} ```