--- title: DwarfStar: the dwarf star illuminating local frontier AI description: DwarfStar is a new open-source inference engine written in C by Salvatore Sanfilippo (creator of Redis), designed to run the DeepSeek V4 Flash frontier model... canonical: https://daily.dev/posts/dwarfstar-the-dwarf-star-illuminating-local-frontier-ai-qwk8wihpq twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: DwarfStar: the dwarf star illuminating local frontier AI | daily.dev og:description: DwarfStar is a new open-source inference engine written in C by Salvatore Sanfilippo (creator of Redis), designed to run the DeepSeek V4 Flash frontier model... og:url: https://daily.dev/posts/dwarfstar-the-dwarf-star-illuminating-local-frontier-ai-qwk8wihpq og:image: https://api.daily.dev/og/posts/qwK8wihpq.png og:image:alt: DwarfStar: the dwarf star illuminating local frontier AI og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # DwarfStar: the dwarf star illuminating local frontier AI **[Codemotion](https://daily.dev/sources/codemotion)** · 13 min read · 4 upvotes · 0 comments ## Summary DwarfStar is a new open-source inference engine written in C by Salvatore Sanfilippo (creator of Redis), designed to run the DeepSeek V4 Flash frontier model locally on consumer Apple Silicon hardware. It achieves this through aggressive asymmetric 2-bit quantization of MoE expert weights, SSD streaming for machines with insufficient RAM, persistent KV cache sessions on disk, and distributed inference across multiple machines via layer splitting. On a 128 GB M3 Max MacBook Pro, it achieves ~26 tokens/s generation with an 81 GB quantized model. The project is MIT-licensed, free, and already has 15,500 GitHub stars weeks after launch. It also explores ensemble inference combining logits from multiple model instances. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How does DwarfStar let a 64GB MacBook run a 284 billion parameter model like DeepSeek V4 Flash? DwarfStar uses 2-bit quantization that compresses the routed Mixture-of-Experts weights (via imatrix calibration) down to about 81GB, then streams infrequently-used experts from SSD instead of keeping them all in RAM. Frequently activated 'hot' experts stay resident in memory, while modern NVMe SSDs at 7-10 GB/s make on-demand loading of the rest fast enough to be usable. _daily.dev surfaces engineering deep dives like this for developers exploring local AI inference tradeoffs._ ### What tokens per second can a 512GB M3 Ultra Mac Studio achieve running DeepSeek V4 Flash with DwarfStar's 2-bit quantization? On short prompts, prefill reaches 84.43 tokens/s and generation 36.86 tokens/s. On long prompts of about 11,700 tokens, prefill rises to 468.03 tokens/s due to chunked prefill, while generation drops to 27.39 tokens/s as the context grows, according to published DwarfStar benchmarks. _developers comparing local inference hardware options can track benchmark reports like this via daily.dev._ ### How does DwarfStar avoid re-processing the entire conversation context every time a local AI agent session resumes? DwarfStar saves persistent sessions to disk containing the exact KV cache state, token checkpoint, and even the probability distribution of the last generated token, so resumption is nearly instantaneous instead of re-running the costly prefill step. This particularly helps code agents with long system prompts, such as Claude Code's roughly 25,000-token prompt, which then only pays the prefill cost once. _daily.dev helps developers building local AI agents keep up with techniques for cutting inference costs._ ## Similar posts on daily.dev - [DwarfStar — A Story](https://daily.dev/posts/dwarfstar-a-story-rzidwkopm) · Hacker News · 0 upvotes · 0 comments - [DeepSeek previews new AI model that ‘closes the gap’ with frontier models](https://daily.dev/posts/deepseek-previews-new-ai-model-that-closes-the-gap-with-frontier-models-7lqiiymer) · TechCrunch · 0 upvotes · 0 comments --- Tags: [#data-science](https://daily.dev/tags/data-science), [#deepseek](https://daily.dev/tags/deepseek), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts) [View this post on daily.dev](https://daily.dev/posts/dwarfstar-the-dwarf-star-illuminating-local-frontier-ai-qwk8wihpq) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"DwarfStar: the dwarf star illuminating local frontier AI","url":"https://daily.dev/posts/dwarfstar-the-dwarf-star-illuminating-local-frontier-ai-qwk8wihpq","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/dwarfstar-the-dwarf-star-illuminating-local-frontier-ai-qwk8wihpq"},"datePublished":"2026-07-14T21:49:22.778Z","dateModified":"2026-09-13T19:14:53.215Z","description":"DwarfStar is a new open-source inference engine written in C by Salvatore Sanfilippo (creator of Redis), designed to run the DeepSeek V4 Flash frontier model...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/cff0642ed405b49dc781a98ffc459edc?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/cff0642ed405b49dc781a98ffc459edc?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Codemotion","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Codemotion","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/008c4c0fe1ba4cd3b6a6740d442d242b","url":"https://daily.dev/sources/codemotion"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/dwarfstar-the-dwarf-star-illuminating-local-frontier-ai-qwk8wihpq","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":4},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-science,deepseek,mixture-of-experts","timeRequired":"PT13M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Codemotion","item":"https://daily.dev/sources/codemotion"},{"@type":"ListItem","position":3,"name":"DwarfStar: the dwarf star illuminating local frontier AI"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/dwarfstar-the-dwarf-star-illuminating-local-frontier-ai-qwk8wihpq#faq","mainEntity":[{"@type":"Question","name":"How does DwarfStar let a 64GB MacBook run a 284 billion parameter model like DeepSeek V4 Flash?","acceptedAnswer":{"@type":"Answer","text":"DwarfStar uses 2-bit quantization that compresses the routed Mixture-of-Experts weights (via imatrix calibration) down to about 81GB, then streams infrequently-used experts from SSD instead of keeping them all in RAM. Frequently activated 'hot' experts stay resident in memory, while modern NVMe SSDs at 7-10 GB/s make on-demand loading of the rest fast enough to be usable. daily.dev surfaces engineering deep dives like this for developers exploring local AI inference tradeoffs."}},{"@type":"Question","name":"What tokens per second can a 512GB M3 Ultra Mac Studio achieve running DeepSeek V4 Flash with DwarfStar's 2-bit quantization?","acceptedAnswer":{"@type":"Answer","text":"On short prompts, prefill reaches 84.43 tokens/s and generation 36.86 tokens/s. On long prompts of about 11,700 tokens, prefill rises to 468.03 tokens/s due to chunked prefill, while generation drops to 27.39 tokens/s as the context grows, according to published DwarfStar benchmarks. developers comparing local inference hardware options can track benchmark reports like this via daily.dev."}},{"@type":"Question","name":"How does DwarfStar avoid re-processing the entire conversation context every time a local AI agent session resumes?","acceptedAnswer":{"@type":"Answer","text":"DwarfStar saves persistent sessions to disk containing the exact KV cache state, token checkpoint, and even the probability distribution of the last generated token, so resumption is nearly instantaneous instead of re-running the costly prefill step. This particularly helps code agents with long system prompts, such as Claude Code's roughly 25,000-token prompt, which then only pays the prefill cost once. daily.dev helps developers building local AI agents keep up with techniques for cutting inference costs."}}]} ```