--- title: Unified Memory, Explained: Why Mini PCs Can Run 70B... description: Unified memory is the key concept behind mini PCs running 70B LLMs that consumer GPUs cannot. A 128GB unified-memory box like the AMD Ryzen AI Max+ 395 can... canonical: https://daily.dev/posts/unified-memory-explained-why-mini-pcs-can-run-70b-models-a-big-gpu-can-t-and-where-they-slow-down-rinvfgbab twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Unified Memory, Explained: Why Mini PCs Can Run 70B Models a Big GPU Can't (and Where They Slow Down) | daily.dev og:description: Unified memory is the key concept behind mini PCs running 70B LLMs that consumer GPUs cannot. A 128GB unified-memory box like the AMD Ryzen AI Max+ 395 can... og:url: https://daily.dev/posts/unified-memory-explained-why-mini-pcs-can-run-70b-models-a-big-gpu-can-t-and-where-they-slow-down-rinvfgbab og:image: https://api.daily.dev/og/posts/rinVFgbAB.png og:image:alt: Unified Memory, Explained: Why Mini PCs Can Run 70B Models a Big GPU Can't (and Where They Slow Down) og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Unified Memory, Explained: Why Mini PCs Can Run 70B Models a Big GPU Can't (and Where They Slow Down) **[Hacker News](https://daily.dev/sources/hn)** · 11 min read · 1 upvotes · 1 comments ## Summary Unified memory is the key concept behind mini PCs running 70B LLMs that consumer GPUs cannot. A 128GB unified-memory box like the AMD Ryzen AI Max+ 395 can load a 40GB 4-bit quantized 70B model that won't fit in a 32GB RTX 5090, but pays a steep speed penalty due to far lower memory bandwidth (~256 GB/s vs ~1,792 GB/s). Text generation speed is memory-bandwidth-bound (roofline model), yielding only 4–6 tokens/sec on dense 70B models. Prompt processing (prefill) is compute-bound and even slower on integrated GPUs, making long-context use painful. NPUs are largely irrelevant for local LLM chat due to shared bandwidth and lack of software support. Mixture-of-Experts models (e.g., Qwen3-30B-A3B) are the sweet spot, activating far fewer parameters per token and achieving 70+ tok/s on the same hardware. The article provides a decision table matching use cases to hardware and references owner-measured benchmarks from ServeTheHome, Level1Techs, and The Register. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How fast can a Strix Halo mini PC with 128GB unified memory run a dense 70B parameter model? A dense 70B model at 4-bit quantization runs at roughly 4 to 6 tokens per second on an AMD Ryzen AI Max+ 395 Strix Halo box with 256 GB/s memory bandwidth. Owner benchmarks measured Llama 3.3 70B at 3.7 to 3.8 tok/s and Shisa V2 70B at 5.0 tok/s, since decode speed is capped by memory bandwidth divided by the roughly 40GB read per token. _For anyone weighing local 70B inference speed against a GPU purchase, daily.dev surfaces benchmarks and hardware writeups like this._ ### Why do Mixture-of-Experts models run so much faster than dense models on unified memory mini PCs? MoE models like Qwen3-30B-A3B only activate about 3B of their 30B total parameters per token, so they read roughly 2GB per token instead of the 40GB a dense 70B model requires. On the same Strix Halo hardware that manages only about 5 tokens per second on a dense 70B model, the MoE model reaches around 72 tokens per second because decode speed is set by bytes read per token, not total parameter count. _Developers picking local models for constrained hardware can track MoE versus dense tradeoffs via daily.dev._ ### Can the NPU in chips like Strix Halo or Snapdragon X2 speed up local LLM inference? Not meaningfully for chat-style decode. The NPU shares the same memory pool as the CPU and GPU, so since decode is bandwidth-bound rather than compute-bound, routing work to the NPU doesn't raise the ceiling. Mainstream tools like llama.cpp and Ollama also don't drive these NPUs well; AMD's NPU offload measured only 4-5 tokens per second on a 7B model versus an expected ~40. _Anyone sizing up NPU marketing claims against real local-LLM performance can dig into hardware breakdowns on daily.dev._ ## Community discussion Top comments from developers on daily.dev. **@kartiknvj** · 0 upvotes > The roofline framing is the useful bit, because people fixate on fitting the weights and forget that dense 70B decode is bandwidth bound at 4 to 6 tokens per second no matter how much memory you have. I have found the mixture-of-experts point is where these boxes earn their place, since activating a fraction of the parameters per token turns a bandwidth wall into something usable. Did the benchmarks separate prefill from decode, since long prompts on integrated GPUs seem to be where it falls apart? ## Similar posts on daily.dev - [High-VRAM GPUs aren't the future of local AI — unified memory and Mixture of Experts models are](https://daily.dev/posts/high-vram-gpus-aren-t-the-future-of-local-ai-unified-memory-and-mixture-of-experts-models-are-jdokpgomz) · XDA Developers · 8 upvotes · 0 comments - [IEEE 500,000 Milestone: A Global Community's Power](https://daily.dev/posts/ieee-500-000-milestone-a-global-community-s-power-uujgjoumf) · IEEE Spectrum · 0 upvotes · 0 comments - [AI On Your Gaming PC](https://daily.dev/posts/ai-on-your-gaming-pc-a7iadecz7) · Hackaday · 2 upvotes · 2 comments - [How a GPU Actually Works](https://daily.dev/posts/how-a-gpu-actually-works-t655zbnjv) · Daily Dose of Data Science \| Avi Chawla \| Substack · 0 upvotes · 0 comments --- Tags: [#local-ai](https://daily.dev/tags/local-ai), [#ai-inference](https://daily.dev/tags/ai-inference), [#llama-cpp](https://daily.dev/tags/llama-cpp) [View this post on daily.dev](https://daily.dev/posts/unified-memory-explained-why-mini-pcs-can-run-70b-models-a-big-gpu-can-t-and-where-they-slow-down-rinvfgbab) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Unified Memory, Explained: Why Mini PCs Can Run 70B Models a Big GPU Can't (and Where They Slow Down)","url":"https://daily.dev/posts/unified-memory-explained-why-mini-pcs-can-run-70b-models-a-big-gpu-can-t-and-where-they-slow-down-rinvfgbab","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/unified-memory-explained-why-mini-pcs-can-run-70b-models-a-big-gpu-can-t-and-where-they-slow-down-rinvfgbab"},"datePublished":"2026-07-10T15:38:27.386Z","dateModified":"2026-09-13T21:53:13.249Z","description":"Unified memory is the key concept behind mini PCs running 70B LLMs that consumer GPUs cannot. A 128GB unified-memory box like the AMD Ryzen AI Max+ 395 can...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/3a4b2b5eeafdb60f5c1e2836d1739550?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/3a4b2b5eeafdb60f5c1e2836d1739550?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/unified-memory-explained-why-mini-pcs-can-run-70b-models-a-big-gpu-can-t-and-where-they-slow-down-rinvfgbab","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"local-ai,ai-inference,llama-cpp","timeRequired":"PT11M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"Unified Memory, Explained: Why Mini PCs Can Run 70B Models a Big GPU Can't (and Where They Slow Down)"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/unified-memory-explained-why-mini-pcs-can-run-70b-models-a-big-gpu-can-t-and-where-they-slow-down-rinvfgbab","comment":[{"@type":"Comment","text":"The roofline framing is the useful bit, because people fixate on fitting the weights and forget that dense 70B decode is bandwidth bound at 4 to 6 tokens per second no matter how much memory you have. I have found the mixture-of-experts point is where these boxes earn their place, since activating a fraction of the parameters per token turns a bandwidth wall into something usable. Did the benchmarks separate prefill from decode, since long prompts on integrated GPUs seem to be where it falls apart?","datePublished":"2026-07-10T16:31:07.988Z","url":"https://daily.dev/posts/rinVFgbAB#c-085yBtZoj","author":{"@type":"Person","name":"kartik-nvjk","url":"https://daily.dev/kartiknvj","image":"https://media.daily.dev/image/upload/s--3gGgsVCw--/f_auto/v1781456774/avatars/avatar_TvTVeiMdkRCqWUDullFmy?_a=BAMAMiWQ0"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/unified-memory-explained-why-mini-pcs-can-run-70b-models-a-big-gpu-can-t-and-where-they-slow-down-rinvfgbab#faq","mainEntity":[{"@type":"Question","name":"How fast can a Strix Halo mini PC with 128GB unified memory run a dense 70B parameter model?","acceptedAnswer":{"@type":"Answer","text":"A dense 70B model at 4-bit quantization runs at roughly 4 to 6 tokens per second on an AMD Ryzen AI Max+ 395 Strix Halo box with 256 GB/s memory bandwidth. Owner benchmarks measured Llama 3.3 70B at 3.7 to 3.8 tok/s and Shisa V2 70B at 5.0 tok/s, since decode speed is capped by memory bandwidth divided by the roughly 40GB read per token. For anyone weighing local 70B inference speed against a GPU purchase, daily.dev surfaces benchmarks and hardware writeups like this."}},{"@type":"Question","name":"Why do Mixture-of-Experts models run so much faster than dense models on unified memory mini PCs?","acceptedAnswer":{"@type":"Answer","text":"MoE models like Qwen3-30B-A3B only activate about 3B of their 30B total parameters per token, so they read roughly 2GB per token instead of the 40GB a dense 70B model requires. On the same Strix Halo hardware that manages only about 5 tokens per second on a dense 70B model, the MoE model reaches around 72 tokens per second because decode speed is set by bytes read per token, not total parameter count. Developers picking local models for constrained hardware can track MoE versus dense tradeoffs via daily.dev."}},{"@type":"Question","name":"Can the NPU in chips like Strix Halo or Snapdragon X2 speed up local LLM inference?","acceptedAnswer":{"@type":"Answer","text":"Not meaningfully for chat-style decode. The NPU shares the same memory pool as the CPU and GPU, so since decode is bandwidth-bound rather than compute-bound, routing work to the NPU doesn't raise the ceiling. Mainstream tools like llama.cpp and Ollama also don't drive these NPUs well; AMD's NPU offload measured only 4-5 tokens per second on a 7B model versus an expected ~40. Anyone sizing up NPU marketing claims against real local-LLM performance can dig into hardware breakdowns on daily.dev."}}]} ```