--- title: Engineering TTS Inference in vLLM-Omni | daily.dev description: vLLM-Omni's engineering team details how they optimized TTS inference for four models: Qwen3-TTS, VoxCPM2, Fish Speech S2 Pro, and Higgs Audio V3. Key... canonical: https://daily.dev/posts/engineering-tts-inference-in-vllm-omni-bwg4hvcgj twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Engineering TTS Inference in vLLM-Omni | daily.dev og:description: vLLM-Omni's engineering team details how they optimized TTS inference for four models: Qwen3-TTS, VoxCPM2, Fish Speech S2 Pro, and Higgs Audio V3. Key... og:url: https://daily.dev/posts/engineering-tts-inference-in-vllm-omni-bwg4hvcgj og:image: https://api.daily.dev/og/posts/bWg4hVCgJ.png og:image:alt: Engineering TTS Inference in vLLM-Omni og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Engineering TTS Inference in vLLM-Omni **[vLLM](https://daily.dev/sources/vllm)** · 21 min read · 0 upvotes · 0 comments ## Summary vLLM-Omni's engineering team details how they optimized TTS inference for four models: Qwen3-TTS, VoxCPM2, Fish Speech S2 Pro, and Higgs Audio V3. Key challenges include decoupling streaming chunk sizes from decode windows to balance TTFP and audio quality, batching per-request Python preprocessing to reduce hot-path overhead, applying whole-model torch.compile to reduce kernel launch boundaries, moving multi-codebook decode state to GPU-resident tensors, and implementing model-specific Triton attention kernels for pure decode shapes. Results include a 61.5% audio throughput improvement for Qwen3-TTS, 172% for VoxCPM2, and 2.70× speedup for Higgs Audio V3. The post also documents rejected designs like staging-overlap under dynamic batching and explains why PIECEWISE CUDA Graph lost to eager plus local MLP graph for Higgs v3. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Experience and Lessons Learned from Serving Multi-Stage Qwen3-Omni in vLLM-Omni](https://daily.dev/posts/experience-and-lessons-learned-from-serving-multi-stage-qwen3-omni-in-vllm-omni-l3fswtvdc) · vLLM · 0 upvotes · 0 comments - [Inside the vLLM-Omni architecture: Serving Qwen3-Omni](https://daily.dev/posts/inside-the-vllm-omni-architecture-serving-qwen3-omni-o8bxkmvxf) · Red Hat Developer · 0 upvotes · 0 comments - [MiniMax H3 on vLLM-Omni: From System-Wide Optimization to Real-Time Serving with FastVideo’s FastH3](https://daily.dev/posts/minimax-h3-on-vllm-omni-from-system-wide-optimization-to-real-time-serving-with-fastvideo-s-fasth3-mhkh8zrjp) · vLLM · 0 upvotes · 0 comments --- Tags: [#text-to-speech](https://daily.dev/tags/text-to-speech), [#vllm](https://daily.dev/tags/vllm) [View this post on daily.dev](https://daily.dev/posts/engineering-tts-inference-in-vllm-omni-bwg4hvcgj) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Engineering TTS Inference in vLLM-Omni","url":"https://daily.dev/posts/engineering-tts-inference-in-vllm-omni-bwg4hvcgj","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/engineering-tts-inference-in-vllm-omni-bwg4hvcgj"},"datePublished":"2026-06-28T08:21:11.884Z","dateModified":"2026-06-28T08:21:39.409Z","description":"vLLM-Omni's engineering team details how they optimized TTS inference for four models: Qwen3-TTS, VoxCPM2, Fish Speech S2 Pro, and Higgs Audio V3. Key...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/86ee478c9afd395510dca3d28fb983fa?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/86ee478c9afd395510dca3d28fb983fa?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"vLLM","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"vLLM","logo":"https://media.daily.dev/image/upload/s--hTxEuls9--/f_auto/v1744613054/logos/vllm","url":"https://daily.dev/sources/vllm"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/engineering-tts-inference-in-vllm-omni-bwg4hvcgj","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"text-to-speech,vllm","timeRequired":"PT21M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"vLLM","item":"https://daily.dev/sources/vllm"},{"@type":"ListItem","position":3,"name":"Engineering TTS Inference in vLLM-Omni"}]} ```