--- title: Qwen 3.8 27B: a local model that punches way above its... description: A hands-on account of running Qwen 3.8 27B locally on a Lenovo ThinkStation with an Nvidia GB10 Grace Blackwell chip and on a Mac with 48GB RAM. The model... canonical: https://daily.dev/posts/qwen-3-8-27b-a-local-model-that-punches-way-above-its-weight-class-s1n5mlvep twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Qwen 3.8 27B: a local model that punches way above its weight class | daily.dev og:description: A hands-on account of running Qwen 3.8 27B locally on a Lenovo ThinkStation with an Nvidia GB10 Grace Blackwell chip and on a Mac with 48GB RAM. The model... og:url: https://daily.dev/posts/qwen-3-8-27b-a-local-model-that-punches-way-above-its-weight-class-s1n5mlvep og:image: https://api.daily.dev/og/posts/S1n5MLvep.png og:image:alt: Qwen 3.8 27B: a local model that punches way above its weight class og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Qwen 3.8 27B: a local model that punches way above its weight class **[Collections](https://daily.dev/sources/collections)** · 4 min read · 13 upvotes · 1 comments ## Summary A hands-on account of running Qwen 3.8 27B locally on a Lenovo ThinkStation with an Nvidia GB10 Grace Blackwell chip and on a Mac with 48GB RAM. The model successfully reverse-engineered the license verification of a purchased commercial app offline in about 30 minutes, self-correcting after a failed integrity check. Separately, paired with Hermes Agent as a local coding agent, it reportedly beat Opus 4.6 Max on SWEBench Pro, OSWorld, co-workbench, and IFBench, though it trails on pure reasoning benchmarks like HLE. The model is a vision-language model built on Qwen 3.5 architecture, Apache 2.0 licensed, with a native 262K context extendable to 1M via YaRN, and runs via Ollama or LM Studio. The author is weighing dropping Claude and Codex subscriptions in favor of this free local setup, while also flagging the dual-use risk of offline models capable of cracking license checks with no oversight. ## Content ## What is Qwen 3.8 27B? Qwen 3.8 27B is an open-weight vision-language model from Alibaba's Qwen team, released under Apache 2.0. It racked up millions of downloads in its first week, which is notable on its own, but the benchmark numbers are what's driving the attention. The architecture is nearly identical to the previous version. The gains come from a progressive training regimen that starts with simple tasks and scales up to complex, multi-step ones. Whatever they did, it worked: the model beats Opus 4.6 on SWEBench Pro, OSWorld, and IFBench, and sits at number one on the Image-to-WebDev Arena leaderboard among open models. It trails on pure reasoning benchmarks like HLE, so it's not a frontier model in every sense, but it's close enough to matter. Native context is 262K tokens, extensible to 1M via YaRN. Tool-calling is solid, which makes it genuinely useful as an agentic coding brain rather than just a chat model. ## Running it locally The 17GB Q4_K_M quantization fits on a 24GB RTX 4090 with room left for 64K context. On a 32GB MacBook Air, AD-Q5_K_M runs with 32K context and retains 97.3% next-token agreement with the full BF16 weights. You can load it in Ollama or LM Studio - just set the context length and sampler settings correctly or you'll leave performance on the table. For those who want to run it on something beefier, a Lenovo ThinkStation with an Nvidia GB10 Grace Blackwell chip handles it fully offline with no cloud involvement. ## Quantization: how low can you go? A detailed benchmark across GPQA Diamond, IFBench, and Terminal-Bench 2.1 (run on NVIDIA L40S, H100, and H200 cards via Modal, costing around $3,000 in GPU rental) gives a clear picture: - **BF16 and Q8**: essentially identical performance - **Q4_K_M**: matches BF16, fits in 17GB - **Q2**: still reasonable, noticeably degraded - **1-bit (IQ1_S)**: collapses to near-random performance, gets worse with longer reasoning chains A separate test on voxel tasks found Q8 sometimes matched or beat Q4 on specific tasks, which is a good reminder that higher precision doesn't automatically win. The practical recommendation from that test is AD-Q5_K_M for Mac users: 97.3% agreement with BF16, runs on 32GB RAM, 32K context. Weight sizes for reference: - Q8_0: 28.9GB - AD-Q4_K_M: 17.1GB (saves about 40% RAM vs Q8) ## Fine-tuning for free You can fine-tune Qwen 3.8 27B without paying for compute. Kaggle provides 30 hours of free GPU time with two Tesla T4s. Using QLoRA with Unsloth's kernels, the 27B model fits in 24GB VRAM with no accuracy loss. All you need is a Google account. People are already doing this with custom Q&A datasets and reporting meaningful before/after differences in model behavior on their specific tasks. ## What it can actually do: a reverse-engineering test One author ran it locally on a ThinkStation and gave it a task they assumed would need a frontier model: reverse-engineer the license verification of a commercial app they'd legitimately purchased. The model initially refused a direct jailbreak attempt, correctly identified the actual developer, then statically analyzed the arm64 disassembly, extracted the embedded public verification key, and built a working authentication bypass. It self-corrected after an initial key failed an integrity hash check. The whole thing took about 30 minutes, fully offline. This is the dual-use problem in concrete form. The same local privacy properties that make running a capable model offline appealing also mean there's no API to rate-limit, no logs, no oversight. A model that can do serious reverse-engineering work, running on hardware you own, is a different kind of tool than a cloud API. ## Using it as an agent For agentic coding workflows, pairing Qwen 3.8 27B with Hermes Agent gives you a fully local stack with no token costs. The feedback loop works better when you close it with real deployed-app checks rather than mocked tests - tools like TestSprite CLI do this without requiring cloud access. The combination of strong tool-calling, long context, and Apache 2.0 licensing makes this a reasonable replacement for paid coding assistants for a lot of workloads. Whether it's worth canceling Claude or Codex depends on what you're actually doing, but for local agentic coding it's a serious option now, not a compromise. ## Questions this post answers ### What quantization level should I use to run Qwen 3.8 27B on a 24GB GPU like an RTX 4090? Q4_K_M quantization fits in about 17GB and matches BF16 performance on benchmarks like GPQA Diamond, IFBench, and Terminal-Bench 2.1, leaving room for roughly 64K context on a 24GB RTX 4090. Q8 is essentially identical to BF16 but larger at 28.9GB, while 1-bit quantization (IQ1_S) collapses to near-random performance, especially on longer reasoning chains. _daily.dev surfaces practical quantization benchmarks for developers deciding how to run large models locally._ ### Can I fine-tune Qwen 3.8 27B without paying for GPU compute? Yes, using Kaggle's free tier which provides 30 hours of GPU time with two Tesla T4s. Applying QLoRA with Unsloth's kernels lets the 27B model fit in 24GB VRAM with no accuracy loss, and all that's required is a Google account to access the free compute. _Developers exploring low-cost fine-tuning workflows can track these setups through daily.dev._ ### Does Qwen 3.8 27B outperform Claude Opus 4.6 on coding benchmarks? Yes, on specific benchmarks: it beats Opus 4.6 on SWEBench Pro, OSWorld, and IFBench, and ranks first among open models on the Image-to-WebDev Arena leaderboard. It still trails on pure reasoning benchmarks like HLE, so it is not a frontier model across every category, but it's close enough to be a serious local alternative for agentic coding. _daily.dev helps developers weighing local models against paid coding assistants keep tabs on benchmark shifts._ ## Community take How the wider developer community reacted, aggregated from 5 discussions and 266 comments across lobsters, hackernews (as of 2026-09-13). **TL;DR:** Discussion mostly moves past the benchmark itself into a long tangent about whether the article was AI-written and whether that matters, plus a deep statistics argument over confidence intervals, while a separate practical thread shares real-world experience running Qwen3.8 27B at various quantizations on consumer GPUs. **Sentiment:** 30% positive · 45% mixed · 25% skeptical **The case for** - Several people report Q3/Q4 quantizations running well on consumer cards (5060ti, 9070XT, 3090, 5090) with usable speeds and long context. - Some find the small quantized model genuinely useful for coding/debugging tasks, even finding bugs frontier models missed. - One commenter's own theory suggests longer thinking at lower quant can compensate for precision loss, keeping success rates similar. **The pushback** - Multiple commenters criticize the writing as AI-assisted, calling it padded, poorly organized, or hard to retain information from. - A detailed statistical critique argues the confidence intervals used for run-to-run variation are conceptually invalid/misapplied. - Some argue quantized local models are still much slower or less capable than frontier cloud models for nuanced coding tasks. - One person notes the tested quants aren't uniformly quantized, so results may not generalize to other quantization schemes. - A request for KV-cache quantization benchmarks and sub-16GB (Q3) coverage suggests gaps in the original benchmark. **By community** - lobsters (mixed): No comments were available to gauge sentiment. - hackernews (mixed): Splits between a long meta-debate about AI-written prose/statistics rigor and practical, largely positive first-hand experiences running the quantized models locally. **Hottest debate:** Whether the article's presentation (and suspected AI assistance in writing it) undermines trust in it, versus whether that concern is irrelevant to judging the actual content. **Open questions** - How does performance hold up at 3-bit quantization, especially for sub-16GB consumer cards? - What is the effect of KV cache quantization combined with different model quantization levels, especially at long context? - Does longer 'thinking' at lower quantization actually compensate for precision loss, or just burn more tokens without matching quality? **Highlights** > hmm, assuming that this article is part written by claude and part human-written, can anyone help me find a rule of thumb for "how to know if the article is worth reading"? Because on the one hand, the prose and the presentation is painful (narrating irrelevant points, nonlinear X-axes, ambiguous chart labels, etc etc), But on the other hand, the result that I'm assuming the author means to communicate ("on these evals, generation quality seems fairly good") sounds worthwhile to share? Because I really struggle with this question at the moment. Am I allowed to draw an adverse inference that "if the writeup presents irrelevant text side by side with the data, then this may be a sign that the author does not understand the task that they are attempting to write up"? > — [Farmadupe on hackernews · 5 comments](https://news.ycombinator.com/item?id=49612561) > This confirms a theory I have to explain the minimal loss in quality when using lower quants (I use IQ3_XXS with an 8-bit KV cache) and the XHIGH (default) thinking level. It's well-known that while quantization affects the sampling probability distribution (given the same context, which next token is the most probable), Qwen 3.8 27b seems to offset that by just thinking more and as a result eventually finishing the task (benchmark or otherwise). So as long as the thinking (albeit longer) is sound, this leads to the same success rate (as shown in the article) but potentially at the cost of more tokens and hence more time. I think it'll be further useful to chart each quantization's used tokens as well, in addition to the success rate. Thanks for doing and sharing the research! > — [sharmajai on hackernews · 4 comments](https://news.ycombinator.com/item?id=49612992) > On many models that I tested in past context quantization had very bad effect on model performance. However qwen3.8 27b is different. I'm now running NVFP4 quantized both weight and cache on my RTX5090 and getting excellent results: 264k cache allocated for pool, 10k tok/s prompt processing, 200 tok/s generation for single stream, or 801 tok/s generation for 8 concurrent streams. Also have about 2Gb vram left for use of OS. my coding agents regularly reach 200k context used without noticeable degradation. P.S. I used setup from: https://github.com/seanyourhighness/vllm-sm12x-nvfp4-dflash2 > — [skolos on hackernews · 2 comments](https://news.ycombinator.com/item?id=49615427) > https://artificialanalysis.ai/?models=qwen3-8-27b%2Cclaude-o... Index methodologies here: https://artificialanalysis.ai/evaluations/artificial-analysi... Also see some specific benchmarks here: https://huggingface.co/Qwen/Qwen3.8-27B e.g. qwen scores 61.7 on swe bench pro, while opus 4.6 scores 53.4. If you want to argue with the benchmarks, go for it. Fwiw i am not saying qwen3.8-27b is better or as good as the frontier. But i am saying it has crossed the threshold and is now a useful tool for coding and debugging. From my experience, Qwen3.6-35b-a3b was what you describe - it could do surgical edits only. > — [kennywinker on hackernews](https://news.ycombinator.com/item?id=49614446) > I would be very interested in a similar benchmark for *KV cache* quantizations. I use Qwen3.8 27B Q4_K_M for coding sometimes and therefore need a relatively long context. I settled on q8_0 because it is the only way to fit the model + 100k tokens into 24GB VRAM, but still wonder what am I loosing in quality, and what other options are there. I also heard that KV cache quantization matters more with longer contexts. It may be interesting to benchmark this too: what the quality looks like on different combinations of model quantization × KV cache quantization × context size. > — [alentred on hackernews · 3 comments](https://news.ycombinator.com/item?id=49613840) **Source threads** - [lobsters](https://lobste.rs/s/lxiabq/benchmarking_qwen3_8_27b_quantizations_4) · 2 points · 0 comments - [hackernews](https://news.ycombinator.com/item?id=49611128) · 268 points · 256 comments - [hackernews](https://news.ycombinator.com/item?id=49452656) · 15 points · 4 comments - [hackernews](https://news.ycombinator.com/item?id=49487826) · 3 points · 4 comments - [hackernews](https://news.ycombinator.com/item?id=49562787) · 3 points · 2 comments ## Community discussion Top comments from developers on daily.dev. **@am1101** · 0 upvotes > "local" if you have the right $4k system. --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#local-ai](https://daily.dev/tags/local-ai), [#reverse-engineering](https://daily.dev/tags/reverse-engineering), [#ollama](https://daily.dev/tags/ollama), [#qwen](https://daily.dev/tags/qwen) [View this post on daily.dev](https://daily.dev/posts/qwen-3-8-27b-a-local-model-that-punches-way-above-its-weight-class-s1n5mlvep) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Qwen 3.8 27B: a local model that punches way above its weight class","url":"https://daily.dev/posts/qwen-3-8-27b-a-local-model-that-punches-way-above-its-weight-class-s1n5mlvep","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/qwen-3-8-27b-a-local-model-that-punches-way-above-its-weight-class-s1n5mlvep"},"datePublished":"2026-08-22T23:03:52.952Z","dateModified":"2026-09-13T19:29:03.970Z","description":"A hands-on account of running Qwen 3.8 27B locally on a Lenovo ThinkStation with an Nvidia GB10 Grace Blackwell chip and on a Mac with 48GB RAM. The model...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/225deae3fc4222b5651248fcdb3ce505?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/225deae3fc4222b5651248fcdb3ce505?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/qwen-3-8-27b-a-local-model-that-punches-way-above-its-weight-class-s1n5mlvep","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":13},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"ai-agents,local-ai,reverse-engineering,ollama,qwen","timeRequired":"PT4M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"Qwen 3.8 27B: a local model that punches way above its weight class"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/qwen-3-8-27b-a-local-model-that-punches-way-above-its-weight-class-s1n5mlvep","comment":[{"@type":"Comment","text":"“local” if you have the right $4k system.","datePublished":"2026-08-26T17:13:50.646Z","url":"https://daily.dev/posts/S1n5MLvep#c-GV6DWFiDT","author":{"@type":"Person","name":"AM","url":"https://daily.dev/am1101","image":"https://media.daily.dev/image/upload/s--DJulCBOC--/f_auto/v1745238461/avatars/avatar_2uCFcp3dLRlOuICIGtoeB"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/qwen-3-8-27b-a-local-model-that-punches-way-above-its-weight-class-s1n5mlvep#faq","mainEntity":[{"@type":"Question","name":"What quantization level should I use to run Qwen 3.8 27B on a 24GB GPU like an RTX 4090?","acceptedAnswer":{"@type":"Answer","text":"Q4_K_M quantization fits in about 17GB and matches BF16 performance on benchmarks like GPQA Diamond, IFBench, and Terminal-Bench 2.1, leaving room for roughly 64K context on a 24GB RTX 4090. Q8 is essentially identical to BF16 but larger at 28.9GB, while 1-bit quantization (IQ1_S) collapses to near-random performance, especially on longer reasoning chains. daily.dev surfaces practical quantization benchmarks for developers deciding how to run large models locally."}},{"@type":"Question","name":"Can I fine-tune Qwen 3.8 27B without paying for GPU compute?","acceptedAnswer":{"@type":"Answer","text":"Yes, using Kaggle's free tier which provides 30 hours of GPU time with two Tesla T4s. Applying QLoRA with Unsloth's kernels lets the 27B model fit in 24GB VRAM with no accuracy loss, and all that's required is a Google account to access the free compute. Developers exploring low-cost fine-tuning workflows can track these setups through daily.dev."}},{"@type":"Question","name":"Does Qwen 3.8 27B outperform Claude Opus 4.6 on coding benchmarks?","acceptedAnswer":{"@type":"Answer","text":"Yes, on specific benchmarks: it beats Opus 4.6 on SWEBench Pro, OSWorld, and IFBench, and ranks first among open models on the Image-to-WebDev Arena leaderboard. It still trails on pure reasoning benchmarks like HLE, so it is not a frontier model across every category, but it's close enough to be a serious local alternative for agentic coding. daily.dev helps developers weighing local models against paid coding assistants keep tabs on benchmark shifts."}}]} ```