vLLM
Read post

Announcing vLLM AFD Plugin: Disaggregating Attention and FFN for Flexible MoE Serving

vLLM AFD Plugin is a new experimental external plugin that introduces Attention-FFN Disaggregation (AFD) for Mixture-of-Experts (MoE) model serving. It separates Attention and FFN computation into independently deployable services, allowing each to scale with its own rank topology. The plugin supports NVIDIA GPUs and Ascend NPUs, synchronous and asynchronous connectors, DeepSeek V2/V3-family models, and multiple execution paths (eager, graph, dual-batch). Performance benchmarks on Ascend 910C show that a 64A16F AFD topology achieves up to +11.3% normalized decode throughput over a conventional EP64 baseline, while asynchronous prefill AFD reduces median TTFT by ~47% at 12 req/s. The plugin integrates via vLLM's plugin entry point without modifying the vLLM source tree, preserving the OpenAI-compatible serving interface. It is currently experimental and pinned to vLLM 0.19.1.

    #ai-inference#deepseek#vllm#mixture-of-experts
Jul 23•9m read time•From vllm.ai
Post cover image
Table of contents
Why Attention-FFN Disaggregation?Inside the ArchitectureA Performance SnapshotGetting StartedCurrent Scope and RoadmapJoin the Community
205 Impressions
vLLM's image
vLLM

76 Followers

•

163 Upvotes

Would you recommend this post?

Copy link
WhatsApp
Facebook
X
New Squad
  • © 2026 Daily Dev Ltd.
  • Guidelines
  • Explore
  • Tags
  • Sources
  • Squads
  • Leaderboard