--- title: Qwen3.8-27B & How to Serve it Fast | daily.dev description: Qwen released the Qwen3.8-27B model following the massive 2.4 trillion parameter Qwen3.8 Max, positioning it as the practical, locally-runnable option.... canonical: https://daily.dev/posts/qwen3-8-27b-how-to-serve-it-fast-ra7gzwwsp twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Qwen3.8-27B & How to Serve it Fast | daily.dev og:description: Qwen released the Qwen3.8-27B model following the massive 2.4 trillion parameter Qwen3.8 Max, positioning it as the practical, locally-runnable option.... og:url: https://daily.dev/posts/qwen3-8-27b-how-to-serve-it-fast-ra7gzwwsp og:image: https://api.daily.dev/og/posts/ra7gZwWSp.png og:image:alt: Qwen3.8-27B & How to Serve it Fast og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Qwen3.8-27B & How to Serve it Fast **[Sam Witteveen](https://daily.dev/sources/samwitteveenai)** · 18 min read · 0 upvotes · 0 comments ## Summary Qwen released the Qwen3.8-27B model following the massive 2.4 trillion parameter Qwen3.8 Max, positioning it as the practical, locally-runnable option. Benchmarks show it substantially beating Qwen3.6-27B and Meta's Muse Glimmer, and scoring close to GLM 5.2 and DeepSeek V4 Pro on Artificial Analysis intelligence and agentic indices. The bulk of the coverage focuses on how to actually run it well: choosing between BF16, FP8, Unsloth NVFP4 quantizations, and MLX builds for Mac, plus tuning the reasoning/thinking level (none, low, medium, x-high), since thinking tokens can balloon from a few hundred to over 20,000 depending on the setting. Testing on an RTX Pro 6000 GPU, speculative decoding (MTP=3) gave large speedups, with VLLM's FP8 version reaching 80-120 tokens/sec, Unsloth's quant around 120 tokens/sec, and SGLang serving an NVFP4 weight variant with GSpark speculative decoding hitting 150-220 tokens/sec, the fastest configuration tested. Obliterated (uncensored) versions were found to loop repeatedly during thinking and are not recommended. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What tokens per second can I get serving Qwen3.8-27B with SGLang and NVFP4 quantization on an RTX Pro 6000? SGLang serving the NVFP4 quantized weights with GSpark speculative decoding on an RTX Pro 6000 (96GB VRAM) reaches roughly 150-220 tokens per second, averaging just under 200, with the full 262K context window enabled. This outperformed VLLM's FP8 version (80-120 tok/s with speculative decoding) and the Unsloth quantization (around 120 tok/s). _daily.dev surfaces real-world throughput benchmarks like this for developers optimizing local LLM inference setups._ ### How many thinking tokens does Qwen3.8-27B use on x-high reasoning compared to medium? On x-high reasoning, Qwen3.8-27B can consume 17,500 to over 22,000 thinking tokens for a single task, sometimes exceeding a 32k output limit before finishing. On medium reasoning the same tasks used roughly 1,000-2,000 thinking tokens, and low reasoning often used similar or fewer tokens than medium, making medium the practical sweet spot for balancing quality and speed. _developers tuning reasoning budgets for local models can track these tradeoffs on daily.dev._ ### Should I use an obliterated (uncensored) version of Qwen3.8-27B? Obliterated versions of Qwen3.8-27B, including ones from teams like Black Frost AI, tend to get stuck in repeated loops during the thinking phase, making them unreliable for practical use. Fine-tuned uncensored variants exist as an alternative, but as of initial testing the obliterated builds are not recommended over the standard FP8 or Unsloth quantized versions. _daily.dev helps developers compare model fine-tunes and quantizations before committing to a local setup._ ## Similar posts on daily.dev - [I dropped Qwen and Gemma for a 27B model that runs smoothly on my 8GB graphics card](https://daily.dev/posts/i-dropped-qwen-and-gemma-for-a-27b-model-that-runs-smoothly-on-my-8gb-graphics-card-h5gi3rwyo) · XDA Developers · 0 upvotes · 0 comments --- Tags: [#data-science](https://daily.dev/tags/data-science), [#vllm](https://daily.dev/tags/vllm), [#qwen](https://daily.dev/tags/qwen) [View this post on daily.dev](https://daily.dev/posts/qwen3-8-27b-how-to-serve-it-fast-ra7gzwwsp) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Qwen3.8-27B & How to Serve it Fast","url":"https://daily.dev/posts/qwen3-8-27b-how-to-serve-it-fast-ra7gzwwsp","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/qwen3-8-27b-how-to-serve-it-fast-ra7gzwwsp"},"datePublished":"2026-08-18T13:13:57.749Z","dateModified":"2026-08-18T17:19:04.552Z","description":"Qwen released the Qwen3.8-27B model following the massive 2.4 trillion parameter Qwen3.8 Max, positioning it as the practical, locally-runnable option....","image":"https://i.ytimg.com/vi/PTuGGdDuyPI/sddefault.jpg","thumbnailUrl":"https://i.ytimg.com/vi/PTuGGdDuyPI/sddefault.jpg","isAccessibleForFree":true,"articleSection":"Sam Witteveen","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Sam Witteveen","logo":"https://media.daily.dev/image/upload/s--gJm-KsgL--/f_auto/v1711727006/logos/samwitteveenai","url":"https://daily.dev/sources/samwitteveenai"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/qwen3-8-27b-how-to-serve-it-fast-ra7gzwwsp","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-science,vllm,qwen","timeRequired":"PT18M","video":{"@type":"VideoObject","name":"Qwen3.8-27B & How to Serve it Fast","description":"Qwen released the Qwen3.8-27B model following the massive 2.4 trillion parameter Qwen3.8 Max, positioning it as the practical, locally-runnable option....","thumbnailUrl":"https://i.ytimg.com/vi/PTuGGdDuyPI/sddefault.jpg","uploadDate":"2026-08-18T13:13:57.749Z","duration":"PT18M","url":"https://api.daily.dev/r/ra7gZwWSp","embedUrl":"https://www.youtube.com/embed/PTuGGdDuyPI"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Sam Witteveen","item":"https://daily.dev/sources/samwitteveenai"},{"@type":"ListItem","position":3,"name":"Qwen3.8-27B & How to Serve it Fast"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/qwen3-8-27b-how-to-serve-it-fast-ra7gzwwsp#faq","mainEntity":[{"@type":"Question","name":"What tokens per second can I get serving Qwen3.8-27B with SGLang and NVFP4 quantization on an RTX Pro 6000?","acceptedAnswer":{"@type":"Answer","text":"SGLang serving the NVFP4 quantized weights with GSpark speculative decoding on an RTX Pro 6000 (96GB VRAM) reaches roughly 150-220 tokens per second, averaging just under 200, with the full 262K context window enabled. This outperformed VLLM's FP8 version (80-120 tok/s with speculative decoding) and the Unsloth quantization (around 120 tok/s). daily.dev surfaces real-world throughput benchmarks like this for developers optimizing local LLM inference setups."}},{"@type":"Question","name":"How many thinking tokens does Qwen3.8-27B use on x-high reasoning compared to medium?","acceptedAnswer":{"@type":"Answer","text":"On x-high reasoning, Qwen3.8-27B can consume 17,500 to over 22,000 thinking tokens for a single task, sometimes exceeding a 32k output limit before finishing. On medium reasoning the same tasks used roughly 1,000-2,000 thinking tokens, and low reasoning often used similar or fewer tokens than medium, making medium the practical sweet spot for balancing quality and speed. developers tuning reasoning budgets for local models can track these tradeoffs on daily.dev."}},{"@type":"Question","name":"Should I use an obliterated (uncensored) version of Qwen3.8-27B?","acceptedAnswer":{"@type":"Answer","text":"Obliterated versions of Qwen3.8-27B, including ones from teams like Black Frost AI, tend to get stuck in repeated loops during the thinking phase, making them unreliable for practical use. Fine-tuned uncensored variants exist as an alternative, but as of initial testing the obliterated builds are not recommended over the standard FP8 or Unsloth quantized versions. daily.dev helps developers compare model fine-tunes and quantizations before committing to a local setup."}}]} ```