--- title: I changed two settings in my local AI setup, and stopped... description: Two runtime settings let local LLMs run faster without more VRAM. For mixture-of-experts models, llama.cpp's --n-cpu-moe flag offloads expert layers to system... canonical: https://daily.dev/posts/i-changed-two-settings-in-my-local-ai-setup-and-stopped-needing-more-vram-qfmts6co8 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: I changed two settings in my local AI setup, and stopped needing more VRAM | daily.dev og:description: Two runtime settings let local LLMs run faster without more VRAM. For mixture-of-experts models, llama.cpp's --n-cpu-moe flag offloads expert layers to system... og:url: https://daily.dev/posts/i-changed-two-settings-in-my-local-ai-setup-and-stopped-needing-more-vram-qfmts6co8 og:image: https://api.daily.dev/og/posts/QFmtS6Co8.png og:image:alt: I changed two settings in my local AI setup, and stopped needing more VRAM og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # I changed two settings in my local AI setup, and stopped needing more VRAM **[XDA Developers](https://daily.dev/sources/xda-developers)** · 4 min read · 1 upvotes · 0 comments ## Summary Two runtime settings let local LLMs run faster without more VRAM. For mixture-of-experts models, llama.cpp's --n-cpu-moe flag offloads expert layers to system RAM, since only a few experts activate per token. On a Legion 5 Pro with 32GB DDR4-3200, setting it to 28 for Qwen3.6-35B-A3B raised speed from 4.02 to 32.82 tokens per second. For dense models, speculative decoding pairs a small draft model with the target model, lifting throughput from 75 to 179 tokens per second. LM Studio offers a slider for MoE offloading; Ollama currently does not. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do I speed up a mixture-of-experts model in llama.cpp without more VRAM? Use the --n-cpu-moe flag to offload a chosen number of expert layers to system RAM while keeping the rest of the model in VRAM, since only a few experts activate per token. On a Legion 5 Pro with 32GB DDR4-3200, setting --n-cpu-moe to 28 took Qwen3.6-35B-A3B from 4.02 to 32.82 tokens per second, an eightfold speedup. _Tuning local model configs like this is easier when daily.dev surfaces the latest llama.cpp tips as they appear._ ### What is speculative decoding and how much faster does it make dense LLM inference? Speculative decoding pairs a target model with a smaller draft model that writes upcoming tokens first, which the target model then verifies in a batch and keeps whatever matches. With the same sampler settings, this raised throughput from 75 to 179 tokens per second on a dense model, though the draft model needs its own slice of VRAM, typically a few hundred megabytes. _Developers chasing faster local inference can follow techniques like this through daily.dev._ ### Does Ollama support offloading mixture-of-experts layers to system RAM like llama.cpp's --n-cpu-moe flag? No, Ollama does not currently let users set MoE expert offloading at all. LM Studio, by contrast, includes a dedicated slider to manage this kind of offloading, giving it more control over splitting MoE models between GPU and system RAM than Ollama offers. _Comparing runtime capabilities like Ollama versus LM Studio is simpler when daily.dev tracks these feature gaps for you._ ## Similar posts on daily.dev - [I changed one llama.cpp flag, and my 8GB laptop GPU now runs a 35B model faster than I can read](https://daily.dev/posts/i-changed-one-llama-cpp-flag-and-my-8gb-laptop-gpu-now-runs-a-35b-model-faster-than-i-can-read-da13mrgf2) · XDA Developers · 118 upvotes · 5 comments - [MoE models changed what "fits on my GPU" means, and most hardware advice hasn't caught up](https://daily.dev/posts/moe-models-changed-what-fits-on-my-gpu-means-and-most-hardware-advice-hasn-t-caught-up-qcmkcp9b6) · XDA Developers · 1 upvotes · 1 comments - [Local LLMs Are Getting Easier: The Complete Guide \(2026\)](https://daily.dev/posts/local-llms-are-getting-easier-the-complete-guide-2026--gaggzz9j2) · SitePoint · 1 upvotes · 0 comments --- Tags: [#ai-inference](https://daily.dev/tags/ai-inference), [#local-ai](https://daily.dev/tags/local-ai), [#ollama](https://daily.dev/tags/ollama), [#llama-cpp](https://daily.dev/tags/llama-cpp) [View this post on daily.dev](https://daily.dev/posts/i-changed-two-settings-in-my-local-ai-setup-and-stopped-needing-more-vram-qfmts6co8) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"I changed two settings in my local AI setup, and stopped needing more VRAM","url":"https://daily.dev/posts/i-changed-two-settings-in-my-local-ai-setup-and-stopped-needing-more-vram-qfmts6co8","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/i-changed-two-settings-in-my-local-ai-setup-and-stopped-needing-more-vram-qfmts6co8"},"datePublished":"2026-10-31T18:30:58.589Z","dateModified":"2026-10-31T18:35:15.074Z","description":"Two runtime settings let local LLMs run faster without more VRAM. For mixture-of-experts models, llama.cpp's --n-cpu-moe flag offloads expert layers to system...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/560e1fd7ef68a3a2518019eef54de70f?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/560e1fd7ef68a3a2518019eef54de70f?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"XDA Developers","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"XDA Developers","logo":"https://media.daily.dev/image/upload/s--kCg6yyAP--/f_auto,q_auto/v1774964407/logos/xda-developers?_a=BAMAMiWQ0","url":"https://daily.dev/sources/xda-developers"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/i-changed-two-settings-in-my-local-ai-setup-and-stopped-needing-more-vram-qfmts6co8","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-inference,local-ai,ollama,llama-cpp","timeRequired":"PT4M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"XDA Developers","item":"https://daily.dev/sources/xda-developers"},{"@type":"ListItem","position":3,"name":"I changed two settings in my local AI setup, and stopped needing more VRAM"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/i-changed-two-settings-in-my-local-ai-setup-and-stopped-needing-more-vram-qfmts6co8#faq","mainEntity":[{"@type":"Question","name":"How do I speed up a mixture-of-experts model in llama.cpp without more VRAM?","acceptedAnswer":{"@type":"Answer","text":"Use the --n-cpu-moe flag to offload a chosen number of expert layers to system RAM while keeping the rest of the model in VRAM, since only a few experts activate per token. On a Legion 5 Pro with 32GB DDR4-3200, setting --n-cpu-moe to 28 took Qwen3.6-35B-A3B from 4.02 to 32.82 tokens per second, an eightfold speedup. Tuning local model configs like this is easier when daily.dev surfaces the latest llama.cpp tips as they appear."}},{"@type":"Question","name":"What is speculative decoding and how much faster does it make dense LLM inference?","acceptedAnswer":{"@type":"Answer","text":"Speculative decoding pairs a target model with a smaller draft model that writes upcoming tokens first, which the target model then verifies in a batch and keeps whatever matches. With the same sampler settings, this raised throughput from 75 to 179 tokens per second on a dense model, though the draft model needs its own slice of VRAM, typically a few hundred megabytes. Developers chasing faster local inference can follow techniques like this through daily.dev."}},{"@type":"Question","name":"Does Ollama support offloading mixture-of-experts layers to system RAM like llama.cpp's --n-cpu-moe flag?","acceptedAnswer":{"@type":"Answer","text":"No, Ollama does not currently let users set MoE expert offloading at all. LM Studio, by contrast, includes a dedicated slider to manage this kind of offloading, giving it more control over splitting MoE models between GPU and system RAM than Ollama offers. Comparing runtime capabilities like Ollama versus LM Studio is simpler when daily.dev tracks these feature gaps for you."}}]} ```