--- title: "Exploring Speculative Decoding in vLLM on AMD GPUs" url: https://daily.dev/posts/exploring-speculative-decoding-in-vllm-on-amd-gpus-3xgpvjp6n source_url: https://vllm.ai/blog/2026-08-23-speculative-decoding-amd-gpus type: article source: "vLLM" published: 2026-08-23T02:05:56.348Z updated: 2026-08-23T02:06:27.229Z tags: ["amd", "ai-inference", "vllm"] reading_time: 67 upvotes: 1 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Exploring Speculative Decoding in vLLM on AMD GPUs **[vLLM](https://daily.dev/sources/vllm)** · 67 min read · 1 upvotes · 0 comments ## Summary A detailed technical walkthrough of speculative decoding in vLLM running on AMD Instinct MI300X and MI355X GPUs with ROCm. It covers five drafting approaches (native MTP, Gemma 4 MTP, EAGLE-3, DFlash, and DSpark), explains how each draft component uses target-model information and generates candidate tokens, and shows how to configure them via --speculative-config. Extensive benchmarks across Gemma, Qwen, MiniMax, and Kimi models show throughput gains ranging from below baseline to over 2.8x depending on model, workload, and proposal length (num_speculative_tokens). Practical tuning guidance covers sweeping proposal lengths, monitoring acceptance rate and mean accepted length, and matching sweep depth to workload type. A section also outlines how to train a new speculator using the vLLM Speculators toolkit. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What speculative decoding methods does vLLM support for AMD GPUs? vLLM's --speculative-config supports four method values: mtp, eagle3, dflash, and dspark, covering native MTP, Gemma 4 MTP, EAGLE-3, DFlash, and DSpark drafting approaches. Native MTP requires no separate draft checkpoint, while the others need a matching speculator model specified via the model field, plus a chosen num_speculative_tokens value. _Engineers tuning inference throughput can follow ongoing vLLM tooling coverage on daily.dev._ ### How does DFlash differ from EAGLE-3 in speculative decoding? DFlash predicts an entire block of future token positions in parallel in a single forward pass, while EAGLE-3 generates draft tokens autoregressively, with each token depending on the previous one. Both fuse hidden states from multiple target-model layers, but DFlash injects this context as Key/Value pairs available in every draft layer, whereas EAGLE-3 combines it only at the input with the sampled-token embedding. _Teams choosing between parallel and sequential drafting strategies can track comparisons like this on daily.dev._ ### What throughput speedups did DFlash and Gemma 4 MTP achieve on gemma-4-26B-A4B-it? On gemma-4-26B-A4B-it, DFlash reached up to 2.87x throughput on MATH500 and 2.79x on HumanEval, while Gemma 4 MTP reached 2.74x on GSM8K and 2.62x on MBPP. EAGLE-3 measurements on the same model ranged from 2.11x to 2.27x across the four datasets, with proposal length (N) tuned per workload during testing on AMD MI300X/MI355X GPUs. _Practitioners benchmarking inference speedups can follow real-world throughput numbers on daily.dev._ ## Similar posts on daily.dev - [Speculative Decoding on vLLM: A Configuration and Decision Framework](https://daily.dev/posts/speculative-decoding-on-vllm-a-configuration-and-decision-framework-wsfye2aez) · DigitalOcean Community · 0 upvotes · 0 comments - [EAGLE-3 Speculative Decoding on AMD Instinct GPUs: Training and Serving with vLLM and AMD Quark](https://daily.dev/posts/eagle-3-speculative-decoding-on-amd-instinct-gpus-training-and-serving-with-vllm-and-amd-quark-x5yf4izas) · vLLM · 0 upvotes · 0 comments - [Diving into speculative decoding training support for vLLM with Speculators v0.3.0](https://daily.dev/posts/diving-into-speculative-decoding-training-support-for-vllm-with-speculators-v0-3-0-heh3ldntg) · vLLM · 0 upvotes · 0 comments --- Tags: [#amd](https://daily.dev/tags/amd), [#ai-inference](https://daily.dev/tags/ai-inference), [#vllm](https://daily.dev/tags/vllm) [View this post on daily.dev](https://daily.dev/posts/exploring-speculative-decoding-in-vllm-on-amd-gpus-3xgpvjp6n)