--- title: Building a High-Performance and Portable vLLM Linear... description: A new Helion-based linear backend for vLLM unifies Standard GEMM, Split-K, and Swap-AB kernel variants into a single autotuned implementation targeting NVIDIA... canonical: https://daily.dev/posts/building-a-high-performance-and-portable-vllm-linear-backend-with-helion-pytorch-mais3inea twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Building a High-Performance and Portable vLLM Linear Backend with Helion – PyTorch | daily.dev og:description: A new Helion-based linear backend for vLLM unifies Standard GEMM, Split-K, and Swap-AB kernel variants into a single autotuned implementation targeting NVIDIA... og:url: https://daily.dev/posts/building-a-high-performance-and-portable-vllm-linear-backend-with-helion-pytorch-mais3inea og:image: https://api.daily.dev/og/posts/maiS3Inea.png og:image:alt: Building a High-Performance and Portable vLLM Linear Backend with Helion – PyTorch og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Building a High-Performance and Portable vLLM Linear Backend with Helion – PyTorch **[PyTorch](https://daily.dev/sources/pytorch)** · 14 min read · 0 upvotes · 0 comments ## Summary A new Helion-based linear backend for vLLM unifies Standard GEMM, Split-K, and Swap-AB kernel variants into a single autotuned implementation targeting NVIDIA Hopper GPUs. Using per-shape autotuning and a hybrid dispatch strategy (Helion under CUDA Graph replay for small num_tokens, falling back to CUTLASS/DeepGEMM for larger shapes), the backend achieves geometric mean kernel speedups of 1.11x-1.18x over default CUTLASS, FlashInfer, and DeepGEMM kernels across FP8_Dynamic, W8A8_INT8, and Block_FP8 quantization formats. End-to-end serving benchmarks across Qwen3 model sizes show consistent throughput gains exceeding 10% for some workloads. The piece also discusses a performance-usability-maintainability tradeoff triangle, LLM-guided autotuning search using Claude Opus 4.8, and a proposed upstream adoption model where Helion kernels ship with default configs while workload-specific tuning is delegated to end users. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much faster is the Helion linear backend compared to vLLM's default CUTLASS and DeepGEMM kernels on Hopper GPUs? Helion achieves geometric mean speedups of 1.110x for FP8_Dynamic over CUTLASS, 1.178x for W8A8_INT8 over CUTLASS, 1.149x for Block_FP8 over FlashInfer, and 1.177x for Block_FP8 over DeepGEMM at the kernel level. End-to-end serving benchmarks across Qwen3 models show consistent throughput gains, exceeding 10% improvement for some workloads, when tested on an NVIDIA H100 80GB HBM3 GPU. _daily.dev surfaces benchmarks like these for engineers evaluating GEMM backend choices for LLM serving._ ### What is the hybrid dispatch strategy used in the vLLM Helion linear backend? Dispatch is based on runtime num_tokens and CUDA Graph coverage: for small shapes up to a max_helion_size threshold (set to 32 in this work), the backend routes to Helion kernels under CUDA Graph replay, while larger shapes fall back to the default CUTLASS or DeepGEMM kernels. This limits fine-grained tuning to the small num_tokens range dominating decoding workloads, cutting both runtime CPU overhead and config maintenance burden. _daily.dev helps engineers track emerging kernel dispatch patterns as they design inference serving pipelines._ ### What GEMM algorithm variants can a single Helion kernel implementation cover for quantized linear layers? A single Helion GEMM implementation can cover Standard GEMM, Split-K, and Swap-AB variants by exposing split_k (power-of-two values up to 256) and swap_ab (boolean) as tunable parameters rather than hand-coding separate kernels and dispatch heuristics. An ahead-of-time autotuner then benchmarks combinations and selects the best variant and config per input shape, supporting FP8_Dynamic, W8A8_INT8, and Block_FP8 quantization formats. _daily.dev keeps developers deciding between kernel DSLs informed on new unification approaches like this one._ ## Similar posts on daily.dev - [Portable vLLM Model Inference Kernels in Helion – PyTorch](https://daily.dev/posts/portable-vllm-model-inference-kernels-in-helion-pytorch-ay8n7jjmi) · PyTorch · 0 upvotes · 0 comments - [Portable Paged Attention in Helion – PyTorch](https://daily.dev/posts/portable-paged-attention-in-helion-pytorch-p3relozri) · PyTorch · 0 upvotes · 0 comments - [From 200 lines to 15: How Helion is rewriting the rules of GPU programming](https://daily.dev/posts/from-200-lines-to-15-how-helion-is-rewriting-the-rules-of-gpu-programming-ewda7l8wq) · Red Hat Developer · 0 upvotes · 0 comments - [Helion on TPU: Towards Hardware Heterogeneous Kernel Authoring – PyTorch](https://daily.dev/posts/helion-on-tpu-towards-hardware-heterogeneous-kernel-authoring-pytorch-uvlidlhmw) · PyTorch · 1 upvotes · 0 comments - [vLLM × HPC-Ops: High-Performance Attention and MoE Backends from Tencent Hunyuan](https://daily.dev/posts/vllm-hpc-ops-high-performance-attention-and-moe-backends-from-tencent-hunyuan-ivtebw42s) · vLLM · 2 upvotes · 0 comments --- Tags: [#pytorch](https://daily.dev/tags/pytorch), [#ai-inference](https://daily.dev/tags/ai-inference), [#vllm](https://daily.dev/tags/vllm) [View this post on daily.dev](https://daily.dev/posts/building-a-high-performance-and-portable-vllm-linear-backend-with-helion-pytorch-mais3inea) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Building a High-Performance and Portable vLLM Linear Backend with Helion – PyTorch","url":"https://daily.dev/posts/building-a-high-performance-and-portable-vllm-linear-backend-with-helion-pytorch-mais3inea","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/building-a-high-performance-and-portable-vllm-linear-backend-with-helion-pytorch-mais3inea"},"datePublished":"2026-10-02T19:58:40.836Z","dateModified":"2026-10-02T20:08:55.492Z","description":"A new Helion-based linear backend for vLLM unifies Standard GEMM, Split-K, and Swap-AB kernel variants into a single autotuned implementation targeting NVIDIA...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/6c6e8ee318693c4de6f5f85f45160b68?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/6c6e8ee318693c4de6f5f85f45160b68?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"PyTorch","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"PyTorch","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/5fdab3d75a834a97a52d433d9c7e5ff9","url":"https://daily.dev/sources/pytorch"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/building-a-high-performance-and-portable-vllm-linear-backend-with-helion-pytorch-mais3inea","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"pytorch,ai-inference,vllm","timeRequired":"PT14M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"PyTorch","item":"https://daily.dev/sources/pytorch"},{"@type":"ListItem","position":3,"name":"Building a High-Performance and Portable vLLM Linear Backend with Helion – PyTorch"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/building-a-high-performance-and-portable-vllm-linear-backend-with-helion-pytorch-mais3inea#faq","mainEntity":[{"@type":"Question","name":"How much faster is the Helion linear backend compared to vLLM's default CUTLASS and DeepGEMM kernels on Hopper GPUs?","acceptedAnswer":{"@type":"Answer","text":"Helion achieves geometric mean speedups of 1.110x for FP8_Dynamic over CUTLASS, 1.178x for W8A8_INT8 over CUTLASS, 1.149x for Block_FP8 over FlashInfer, and 1.177x for Block_FP8 over DeepGEMM at the kernel level. End-to-end serving benchmarks across Qwen3 models show consistent throughput gains, exceeding 10% improvement for some workloads, when tested on an NVIDIA H100 80GB HBM3 GPU. daily.dev surfaces benchmarks like these for engineers evaluating GEMM backend choices for LLM serving."}},{"@type":"Question","name":"What is the hybrid dispatch strategy used in the vLLM Helion linear backend?","acceptedAnswer":{"@type":"Answer","text":"Dispatch is based on runtime num_tokens and CUDA Graph coverage: for small shapes up to a max_helion_size threshold (set to 32 in this work), the backend routes to Helion kernels under CUDA Graph replay, while larger shapes fall back to the default CUTLASS or DeepGEMM kernels. This limits fine-grained tuning to the small num_tokens range dominating decoding workloads, cutting both runtime CPU overhead and config maintenance burden. daily.dev helps engineers track emerging kernel dispatch patterns as they design inference serving pipelines."}},{"@type":"Question","name":"What GEMM algorithm variants can a single Helion kernel implementation cover for quantized linear layers?","acceptedAnswer":{"@type":"Answer","text":"A single Helion GEMM implementation can cover Standard GEMM, Split-K, and Swap-AB variants by exposing split_k (power-of-two values up to 256) and swap_ab (boolean) as tunable parameters rather than hand-coding separate kernels and dispatch heuristics. An ahead-of-time autotuner then benchmarks combinations and selects the best variant and config per input shape, supporting FP8_Dynamic, W8A8_INT8, and Block_FP8 quantization formats. daily.dev keeps developers deciding between kernel DSLs informed on new unification approaches like this one."}}]} ```