--- title: I need to rant about local models | daily.dev description: A strong opinion piece arguing that running large open-weight AI models locally on consumer hardware is impractical and overhyped. The author distinguishes... canonical: https://daily.dev/posts/i-need-to-rant-about-local-models-kwfonoahk twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: I need to rant about local models | daily.dev og:description: A strong opinion piece arguing that running large open-weight AI models locally on consumer hardware is impractical and overhyped. The author distinguishes... og:url: https://daily.dev/posts/i-need-to-rant-about-local-models-kwfonoahk og:image: https://api.daily.dev/og/posts/KwFoNoahK.png og:image:alt: I need to rant about local models og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # I need to rant about local models **[Theo - t3․gg](https://daily.dev/sources/t3dotgg)** · 28 min read · 1 upvotes · 0 comments ## Summary A strong opinion piece arguing that running large open-weight AI models locally on consumer hardware is impractical and overhyped. The author distinguishes between open-weight models (which he supports) and the idea of running them locally (which he considers delusional). Key arguments include: frontier open-weight models like GLM-52 require 400GB+ VRAM in full precision, consumer GPUs max out at 32GB VRAM making them unsuitable, unified memory Macs are the only viable consumer option but are expensive, parallelism requirements for agentic workflows make local inference even less feasible, electricity costs add up significantly, and open-weight models burn more tokens than frontier closed models making per-run costs less competitive than per-token prices suggest. The author's actual position is that open-weight models are valuable because they enable competitive cloud hosting markets with multiple providers offering cheaper inference — not because individuals can run them at home. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much VRAM do I need to run the full GLM-5.2 model locally? The full GLM-5.2 model is conservatively around 400GB, and the full precision BF16 version requires roughly 1.5TB, far beyond any consumer hardware. Even quantized and pruned versions still sit in the 200GB range, which barely runs on high-end consumer setups if at all, ruling out realistic local use for most developers. _Developers weighing local versus hosted inference for coding agents can track hardware realities on daily.dev._ ### Why can't a gaming PC with 128GB of RAM run large open-weight models like Deep Seek V4 well? What matters for inference is VRAM, not system RAM, and consumer GPUs like the RTX 5090 only have 32GB of VRAM regardless of how much RAM sits alongside it. A MacBook with 128GB works better because it uses unified memory that counts as usable VRAM, unlike a discrete gaming GPU where extra system RAM does nothing for inference once VRAM is saturated. _Anyone sizing hardware for coding agent workloads can compare unified memory versus GPU VRAM trade-offs on daily.dev._ ### Are open-weight models like GLM-5.2 actually cheaper to run than frontier models like Opus 4.8? Not by nearly as much as the per-token pricing suggests. Deep Infra prices GLM-5.2 at $3 per million output tokens versus $25 for Opus 4.8, a 10x difference, but because open-weight models burn far more tokens to complete the same task, a real coding run on Opus 4.8 costs about $8 while GLM-5.2 costs about $4, only a 2x gap in practice. _Teams comparing real-world costs of open-weight versus frontier coding models can dig into token efficiency data on daily.dev._ ## Similar posts on daily.dev - [Local AI is more accessible than ever, but with one major GPU-sized caveat](https://daily.dev/posts/local-ai-is-more-accessible-than-ever-but-with-one-major-gpu-sized-caveat-ozobd3gds) · XDA Developers · 0 upvotes · 0 comments - [No, local models will not win](https://daily.dev/posts/no-local-models-will-not-win-t3ebfknyr) · sean goedecke · 2 upvotes · 2 comments --- Tags: [#open-source](https://daily.dev/tags/open-source), [#llm](https://daily.dev/tags/llm), [#gpu](https://daily.dev/tags/gpu), [#ai-inference](https://daily.dev/tags/ai-inference), [#local-ai](https://daily.dev/tags/local-ai) [View this post on daily.dev](https://daily.dev/posts/i-need-to-rant-about-local-models-kwfonoahk) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"I need to rant about local models","url":"https://daily.dev/posts/i-need-to-rant-about-local-models-kwfonoahk","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/i-need-to-rant-about-local-models-kwfonoahk"},"datePublished":"2026-07-07T21:06:01.812Z","dateModified":"2026-10-15T02:22:53.384Z","description":"A strong opinion piece arguing that running large open-weight AI models locally on consumer hardware is impractical and overhyped. The author distinguishes...","image":"https://i.ytimg.com/vi/wnfxSxP8pGs/sddefault.jpg","thumbnailUrl":"https://i.ytimg.com/vi/wnfxSxP8pGs/sddefault.jpg","isAccessibleForFree":true,"articleSection":"Theo - t3․gg","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Theo - t3․gg","logo":"https://media.daily.dev/image/upload/s--UmX7IyU3--/f_auto/v1704628081/logos/t3dotgg.jpg","url":"https://daily.dev/sources/t3dotgg"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/i-need-to-rant-about-local-models-kwfonoahk","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"open-source,llm,gpu,ai-inference,local-ai","timeRequired":"PT28M","video":{"@type":"VideoObject","name":"I need to rant about local models","description":"A strong opinion piece arguing that running large open-weight AI models locally on consumer hardware is impractical and overhyped. The author distinguishes...","thumbnailUrl":"https://i.ytimg.com/vi/wnfxSxP8pGs/sddefault.jpg","uploadDate":"2026-07-07T21:06:01.812Z","duration":"PT28M","url":"https://api.daily.dev/r/KwFoNoahK","embedUrl":"https://www.youtube.com/embed/wnfxSxP8pGs"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Theo - t3․gg","item":"https://daily.dev/sources/t3dotgg"},{"@type":"ListItem","position":3,"name":"I need to rant about local models"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/i-need-to-rant-about-local-models-kwfonoahk#faq","mainEntity":[{"@type":"Question","name":"How much VRAM do I need to run the full GLM-5.2 model locally?","acceptedAnswer":{"@type":"Answer","text":"The full GLM-5.2 model is conservatively around 400GB, and the full precision BF16 version requires roughly 1.5TB, far beyond any consumer hardware. Even quantized and pruned versions still sit in the 200GB range, which barely runs on high-end consumer setups if at all, ruling out realistic local use for most developers. Developers weighing local versus hosted inference for coding agents can track hardware realities on daily.dev."}},{"@type":"Question","name":"Why can't a gaming PC with 128GB of RAM run large open-weight models like Deep Seek V4 well?","acceptedAnswer":{"@type":"Answer","text":"What matters for inference is VRAM, not system RAM, and consumer GPUs like the RTX 5090 only have 32GB of VRAM regardless of how much RAM sits alongside it. A MacBook with 128GB works better because it uses unified memory that counts as usable VRAM, unlike a discrete gaming GPU where extra system RAM does nothing for inference once VRAM is saturated. Anyone sizing hardware for coding agent workloads can compare unified memory versus GPU VRAM trade-offs on daily.dev."}},{"@type":"Question","name":"Are open-weight models like GLM-5.2 actually cheaper to run than frontier models like Opus 4.8?","acceptedAnswer":{"@type":"Answer","text":"Not by nearly as much as the per-token pricing suggests. Deep Infra prices GLM-5.2 at $3 per million output tokens versus $25 for Opus 4.8, a 10x difference, but because open-weight models burn far more tokens to complete the same task, a real coding run on Opus 4.8 costs about $8 while GLM-5.2 costs about $4, only a 2x gap in practice. Teams comparing real-world costs of open-weight versus frontier coding models can dig into token efficiency data on daily.dev."}}]} ```