--- title: Portable vLLM Model Inference Kernels in Helion – PyTorch description: Helion, a PyTorch-native kernel DSL, was integrated into vLLM to replace forward-pass GPU kernels for FP8 inference using Qwen3 models. Custom Helion kernels... canonical: https://daily.dev/posts/portable-vllm-model-inference-kernels-in-helion-pytorch-ay8n7jjmi twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Portable vLLM Model Inference Kernels in Helion – PyTorch | daily.dev og:description: Helion, a PyTorch-native kernel DSL, was integrated into vLLM to replace forward-pass GPU kernels for FP8 inference using Qwen3 models. Custom Helion kernels... og:url: https://daily.dev/posts/portable-vllm-model-inference-kernels-in-helion-pytorch-ay8n7jjmi og:image: https://api.daily.dev/og/posts/aY8N7jJMI.png og:image:alt: Portable vLLM Model Inference Kernels in Helion – PyTorch og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Portable vLLM Model Inference Kernels in Helion – PyTorch **[PyTorch](https://daily.dev/sources/pytorch)** · 12 min read · 0 upvotes · 0 comments ## Summary Helion, a PyTorch-native kernel DSL, was integrated into vLLM to replace forward-pass GPU kernels for FP8 inference using Qwen3 models. Custom Helion kernels were implemented for quantization, normalization, and fusion-heavy operations, then benchmarked against CUTLASS, torch.compile, and existing CUDA kernels on NVIDIA H100 and B200 GPUs. Non-GEMM kernels showed consistent speedups of 1.1x–2.3x over baselines. End-to-end throughput gains of up to ~1.09x were observed on H100, with speculative decoding scenarios showing up to 1.33x TTFT improvement. GEMM kernels underperformed on B200 due to current Triton limitations on Blackwell GPUs, with ongoing work on a CuteDSL backend to address this. Autotuning overhead remains a practical caveat, requiring full-day sweeps for large kernels. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Portable Paged Attention in Helion – PyTorch](https://daily.dev/posts/portable-paged-attention-in-helion-pytorch-p3relozri) · PyTorch · 0 upvotes · 0 comments - [Helion: A High-Level DSL for Performant and Portable ML Kernels – PyTorch](https://daily.dev/posts/helion-a-high-level-dsl-for-performant-and-portable-ml-kernels-pytorch-przi31los) · PyTorch · 0 upvotes · 0 comments - [Helion on TPU: Towards Hardware Heterogeneous Kernel Authoring – PyTorch](https://daily.dev/posts/helion-on-tpu-towards-hardware-heterogeneous-kernel-authoring-pytorch-uvlidlhmw) · PyTorch · 1 upvotes · 0 comments --- Tags: [#pytorch](https://daily.dev/tags/pytorch), [#vllm](https://daily.dev/tags/vllm) [View this post on daily.dev](https://daily.dev/posts/portable-vllm-model-inference-kernels-in-helion-pytorch-ay8n7jjmi) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Portable vLLM Model Inference Kernels in Helion – PyTorch","url":"https://daily.dev/posts/portable-vllm-model-inference-kernels-in-helion-pytorch-ay8n7jjmi","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/portable-vllm-model-inference-kernels-in-helion-pytorch-ay8n7jjmi"},"datePublished":"2026-06-10T17:02:40.151Z","dateModified":"2026-06-10T17:03:08.971Z","description":"Helion, a PyTorch-native kernel DSL, was integrated into vLLM to replace forward-pass GPU kernels for FP8 inference using Qwen3 models. Custom Helion kernels...","image":"https://media.daily.dev/image/upload/s--CxzD6vbw--/f_auto/v1722860399/public/Placeholder%2005","thumbnailUrl":"https://media.daily.dev/image/upload/s--CxzD6vbw--/f_auto/v1722860399/public/Placeholder%2005","isAccessibleForFree":true,"articleSection":"PyTorch","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"PyTorch","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/5fdab3d75a834a97a52d433d9c7e5ff9","url":"https://daily.dev/sources/pytorch"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/portable-vllm-model-inference-kernels-in-helion-pytorch-ay8n7jjmi","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"pytorch,vllm","timeRequired":"PT12M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"PyTorch","item":"https://daily.dev/sources/pytorch"},{"@type":"ListItem","position":3,"name":"Portable vLLM Model Inference Kernels in Helion – PyTorch"}]} ```