--- title: EAGLE-4 speeds up LLM inference with an adaptive... description: Eagle Corp released EAGLE-4, an open-source drafter for speculative decoding that adapts its drafting strategy to batch size, hardware and workload while... canonical: https://daily.dev/posts/eagle-4-speeds-up-llm-inference-with-an-adaptive-speculative-decoding-drafter-2hx4eb8tv twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: EAGLE-4 speeds up LLM inference with an adaptive speculative decoding drafter | daily.dev og:description: Eagle Corp released EAGLE-4, an open-source drafter for speculative decoding that adapts its drafting strategy to batch size, hardware and workload while... og:url: https://daily.dev/posts/eagle-4-speeds-up-llm-inference-with-an-adaptive-speculative-decoding-drafter-2hx4eb8tv og:image: https://api.daily.dev/og/posts/2Hx4EB8tV.png og:image:alt: EAGLE-4 speeds up LLM inference with an adaptive speculative decoding drafter og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # EAGLE-4 speeds up LLM inference with an adaptive speculative decoding drafter **[Collections](https://daily.dev/sources/collections)** · 2 min read · 1 upvotes · 0 comments ## Summary Eagle Corp released EAGLE-4, an open-source drafter for speculative decoding that adapts its drafting strategy to batch size, hardware and workload while preserving the target model's output distribution. It keeps the standard draft-and-verify loop but uses an adaptive draft structure and a cheaper draft LM head. Paired with Qwen3.8-27B on 4x NVIDIA B300, it reaches 961.6 tokens/s. On a single request with the 27B model it decodes 863.8 tokens/s, 1.28x DFlash 2 and 1.40x DSpark, for a roughly 3x speedup on the tested setup. These are vendor-reported figures pending independent verification. ## Content Coding agents rarely make a single model call. They make dozens, and every step waits on tokens. That waiting adds up. The usual fix is a paid "fast mode," which tends to mean premium pricing or burning through credits faster. EAGLE-4 takes a different route. Eagle Corp has released it as an open-source drafter for speculative decoding that you run on your own serving stack. Eagle describes itself as an AI inference platform used by companies including NVIDIA, Meta, Intel, AMD, and Perplexity. ## How it works EAGLE-4 keeps the standard draft-and-verify loop and puts its effort into the drafting side, with an adaptive draft structure and a cheaper draft LM head. The loop goes like this: - A small draft model proposes several tokens ahead. - The target model checks all of them in one forward pass. - Matching tokens are kept, and the first mismatch is replaced with the target's own token. - Repeat from there. Because the big model verifies everything, the output is identical to what it would have produced alone. Eagle's own description is that one drafter adapts its speculative decoding strategy to the batch size, the hardware, and the workload, while keeping the target model's output distribution. ## The numbers The measured target pair is Qwen3.8-27B with its EAGLE-4 draft. Eagle reports 961.6 tokens/s on 4x NVIDIA B300. On a single request with a 27B model, EAGLE-4 decodes 863.8 tokens/s, which is 1.28x DFlash 2 and 1.40x DSpark. On the tested setup the speedup is roughly 3x. These are vendor-reported figures on specific hardware, so treat them as a starting point for your own measurements. ## Why it matters for cost There is no EAGLE software fee. You still pay for your own compute and hosting. But you can read the code, tune it to your hardware and load, and check the speed-versus-cost trade-off on your own workloads instead of taking a vendor's word for it. Speculative decoding has become one of the more dependable ways to get more tokens per second out of a model you've already chosen. I'm interested in how this holds up on a real coding-agent workflow, with before-and-after numbers. @trikcode says they plan to run exactly that test and share the results. ## Questions this post answers ### What is speculative decoding and how does it speed up LLM inference? Speculative decoding uses a small draft model to propose several tokens ahead, then has the larger target model verify all of them in a single forward pass. Matching tokens are kept and the first mismatch is replaced with the target model's own token, then the process repeats. This produces output identical to running the target model alone, but faster. _daily.dev surfaces technical breakdowns like this for engineers tuning LLM inference throughput._ ### How fast is EAGLE-4 compared to DFlash 2 and DSpark for LLM decoding? On a single request with a 27B model, EAGLE-4 decodes 863.8 tokens per second, which is 1.28 times faster than DFlash 2 and 1.40 times faster than DSpark. Paired with Qwen3.8-27B on 4x NVIDIA B300 hardware, it reaches 961.6 tokens per second, with an overall speedup of roughly 3x on the tested setup. These figures are vendor-reported. _Teams comparing inference drafters can track benchmark claims like these on daily.dev before committing._ ## Community take How the wider developer community reacted, aggregated from 3 discussions and 35 comments across x (as of 2026-10-21). **TL;DR:** The community sees EAGLE-4 as a solid, practically useful speculative-decoding advance since it preserves the target model's exact output distribution while speeding things up, but many want to see the headline numbers reproduced at realistic batch sizes, sampling temperatures, and acceptance rates rather than single-request T=0 benchmarks. **Sentiment:** 35% positive · 50% mixed · 15% skeptical **The case for** - Lossless speedup is valued because output distribution stays identical to the target model, unlike other speed tricks that trade away quality. - Adapting drafting to batch size and hardware is seen as practically important rather than a lab-only trick. - Shipping as engine plugins for serving stacks like vLLM/SGLang is seen as making speculative decoding a drop-in rather than a per-engine hack. - Agent/tool-call workloads are seen as a good fit since so much output (JSON, file paths, diffs) is predictable. **The pushback** - Headline numbers are single-request at temperature 0, which is the easiest case for a drafter, and may not hold at higher batch sizes or with sampling. - One commenter's back-of-envelope HBM bandwidth math questions whether the 961.6 tok/s figure is physically plausible without an unusually high tokens-kept-per-pass rate. - Acceptance rates and speedups are expected to drop on multilingual or non-English text versus the ShareGPT/UltraChat data earlier EAGLE heads were trained on. - Speedup may shrink on messy code or math prompts compared to predictable text. - KV-cache size/memory bandwidth, not decode speed, is argued to be the bigger bottleneck for agentic workloads with long contexts. **By community** - x (mixed): Interest in the lossless, adaptive drafting approach is tempered by repeated calls to verify the benchmarks under realistic batch sizes, sampling, and varied workloads. **Hottest debate:** Whether the reported 861-961 tok/s figures reflect real serving conditions or are best-case single-request/T=0 numbers that won't hold up under batching and sampling. **Open questions** - Does the speedup hold at higher batch sizes (e.g. 32) or with temperature sampling, where acceptance rates typically drop? - What acceptance rate does EAGLE-4 achieve on tool-call-heavy or code/math traces versus plain chat? - Is the 961.6 tok/s figure achievable given HBM bandwidth limits, or does it require an unusually high tokens-kept-per-pass rate? - How does performance compare across languages, given known acceptance-rate drops on non-English text in prior EAGLE versions? - What is the actual cost/power tradeoff alongside the speed gains? **Highlights** > @rohanpaul_ai 27b in bf16 is about 54 gb and 4 b300s read 8 tb/s each, so one token per forward pass tops out near 590 a second even with perfect hbm the 961 headline needs at least 1.6 tokens kept per pass, lightseeks writeup has greedy at about 4.75 and plain decoding around 263 > — [girlquants on x](https://x.com/girlquants/status/2112722238952907016) > @rohanpaul_ai The headline numbers are all single request at T=0, the best case for drafting. Does the speedup hold at batch 32 or with temperature sampling, where acceptance rates usually drop? > — [Bogirl0 on x](https://x.com/Bogirl0/status/2112712833221673426) > @rohanpaul_ai Speculative decoding is impressive, but most models still struggle with context retention at high speeds. Faster token generation often sacrifices coherence, leading to outputs that miss the mark. Balancing speed with quality is the real challenge. > — [younghrmanager on x](https://x.com/younghrmanager/status/2112714396502028771) > @rohanpaul_ai Speculative decoding deserves the hype, but for agent workloads KV-cache management is the bigger lever — decode speedups mean little when context keeps doubling and memory bandwidth dominates. Which bites your agentic workloads harder: decode latency or KV-cache size? > — [AliciaCrawoiue on x](https://x.com/AliciaCrawoiue/status/2112735942654246987) > @testingcatalog Adapting to batch size isn't the same as adapting to language. Earlier EAGLE heads sat on ShareGPT/UltraChat (mostly EN). AdaSpec shows EAGLE-2 acceptance dropping off English, and Yi et al. needed language-specific drafters for the big multilingual speedups. 961 tok/s is one > — [BrianWesley0711 on x](https://x.com/BrianWesley0711/status/2112712123344195938) **Source threads** - [x](https://x.com/testingcatalog/status/2112665677152829772) · 1 points · 14 comments - [x](https://x.com/rohanpaul_ai/status/2112707351300182516) · 0 points · 13 comments - [x](https://x.com/trikcode/status/2112749359821439466) · 0 points · 8 comments ## Similar posts on daily.dev - [P-EAGLE: Faster LLM inference with Parallel Speculative Decoding in vLLM](https://daily.dev/posts/p-eagle-faster-llm-inference-with-parallel-speculative-decoding-in-vllm-jnlxyxzy3) · vLLM · 1 upvotes · 0 comments - [Speeding up LLM inference with P-EAGLE in vLLM Speculators](https://daily.dev/posts/speeding-up-llm-inference-with-p-eagle-in-vllm-speculators-zzmicyrfu) · Red Hat Developer · 2 upvotes · 0 comments - [Co-Designing AI Models Using Speculative Decoding for Faster LLM Inference](https://daily.dev/posts/co-designing-ai-models-using-speculative-decoding-for-faster-llm-inference-0zr9mkfau) · NVIDIA Developer · 0 upvotes · 0 comments - [Parallel All the Way Down: Beyond Single-Token Generation with Speculative Decoding](https://daily.dev/posts/parallel-all-the-way-down-beyond-single-token-generation-with-speculative-decoding-yzx2fn2fo) · vLLM · 0 upvotes · 0 comments - [Performance improvements with speculative decoding in vLLM for gpt-oss](https://daily.dev/posts/performance-improvements-with-speculative-decoding-in-vllm-for-gpt-oss-lpepzsizp) · Red Hat Developer · 0 upvotes · 0 comments --- Tags: [#nvidia](https://daily.dev/tags/nvidia), [#ai-inference](https://daily.dev/tags/ai-inference) [View this post on daily.dev](https://daily.dev/posts/eagle-4-speeds-up-llm-inference-with-an-adaptive-speculative-decoding-drafter-2hx4eb8tv) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"EAGLE-4 speeds up LLM inference with an adaptive speculative decoding drafter","url":"https://daily.dev/posts/eagle-4-speeds-up-llm-inference-with-an-adaptive-speculative-decoding-drafter-2hx4eb8tv","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/eagle-4-speeds-up-llm-inference-with-an-adaptive-speculative-decoding-drafter-2hx4eb8tv"},"datePublished":"2026-10-21T00:48:18.704Z","dateModified":"2026-10-21T05:06:00.692Z","description":"Eagle Corp released EAGLE-4, an open-source drafter for speculative decoding that adapts its drafting strategy to batch size, hardware and workload while...","image":"https://pbs.twimg.com/media/HVHXTajbkAEVm9C.jpg","thumbnailUrl":"https://pbs.twimg.com/media/HVHXTajbkAEVm9C.jpg","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/eagle-4-speeds-up-llm-inference-with-an-adaptive-speculative-decoding-drafter-2hx4eb8tv","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"nvidia,ai-inference","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"EAGLE-4 speeds up LLM inference with an adaptive speculative decoding drafter"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/eagle-4-speeds-up-llm-inference-with-an-adaptive-speculative-decoding-drafter-2hx4eb8tv#faq","mainEntity":[{"@type":"Question","name":"What is speculative decoding and how does it speed up LLM inference?","acceptedAnswer":{"@type":"Answer","text":"Speculative decoding uses a small draft model to propose several tokens ahead, then has the larger target model verify all of them in a single forward pass. Matching tokens are kept and the first mismatch is replaced with the target model's own token, then the process repeats. This produces output identical to running the target model alone, but faster. daily.dev surfaces technical breakdowns like this for engineers tuning LLM inference throughput."}},{"@type":"Question","name":"How fast is EAGLE-4 compared to DFlash 2 and DSpark for LLM decoding?","acceptedAnswer":{"@type":"Answer","text":"On a single request with a 27B model, EAGLE-4 decodes 863.8 tokens per second, which is 1.28 times faster than DFlash 2 and 1.40 times faster than DSpark. Paired with Qwen3.8-27B on 4x NVIDIA B300 hardware, it reaches 961.6 tokens per second, with an overall speedup of roughly 3x on the tested setup. These figures are vendor-reported. Teams comparing inference drafters can track benchmark claims like these on daily.dev before committing."}}]} ```