--- title: "Optimizing vLLM on Arm CPUs" url: https://daily.dev/posts/optimizing-vllm-on-arm-cpus-oezpzfcin source_url: https://vllm.ai/blog/2026-07-29-optimizing-vllm-on-arm-cpus type: article source: "vLLM" published: 2026-07-29T15:13:08.795Z updated: 2026-07-29T16:18:08.360Z tags: ["data-science", "performance", "arm", "ai-inference", "vllm"] reading_time: 9 upvotes: 1 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Optimizing vLLM on Arm CPUs **[vLLM](https://daily.dev/sources/vllm)** · 9 min read · 1 upvotes · 0 comments ## Summary A detailed technical breakdown of optimizations made to vLLM for Arm Neoverse-based CPUs, covering the full inference stack. Key improvements include switching to mimalloc as the default allocator (2.3× throughput gain), leveraging Arm LSE atomics in OpenMP to reduce synchronization overhead (9% throughput, 15% TPOT improvement), enabling oneDNN weight prepacking for dense layers (16% throughput, 60% TPOT improvement), and rewriting paged attention with BFMMLA SIMD instructions (4× faster attention, 12% throughput gain). INT8 W8A8 quantization delivers up to 88% higher throughput and 45% lower TPOT versus the optimized BF16 baseline, while INT8 W4A8 pushes further with up to 29% additional throughput gains. Combined, these changes yield up to 6.2× the October 2025 baseline throughput and a 7.8× TPOT speedup. Usability improvements include pre-built wheels and Docker images, chunked prefill, prefix caching, and expanded model support. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [High-performance quantized LLM inference on Intel CPUs with native PyTorch – PyTorch](https://daily.dev/posts/high-performance-quantized-llm-inference-on-intel-cpus-with-native-pytorch-pytorch-2o8lnytjk) · PyTorch · 0 upvotes · 0 comments - [The State of FP8 KV-Cache and Attention Quantization in vLLM](https://daily.dev/posts/the-state-of-fp8-kv-cache-and-attention-quantization-in-vllm-ovwjvspr4) · vLLM · 1 upvotes · 0 comments --- Tags: [#data-science](https://daily.dev/tags/data-science), [#performance](https://daily.dev/tags/performance), [#arm](https://daily.dev/tags/arm), [#ai-inference](https://daily.dev/tags/ai-inference), [#vllm](https://daily.dev/tags/vllm) [View this post on daily.dev](https://daily.dev/posts/optimizing-vllm-on-arm-cpus-oezpzfcin)