--- title: Redis creator's local LLM runner is quietly becoming a... description: Salvatore Sanfilippo, creator of Redis, has been building DwarfStar (ds4), a purpose-built C inference engine for running DeepSeek V4 Flash (284B MoE... canonical: https://daily.dev/posts/redis-creator-s-local-llm-runner-is-quietly-becoming-a-real-inference-server-pmfzrazge twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Redis creator's local LLM runner is quietly becoming a real inference server | daily.dev og:description: Salvatore Sanfilippo, creator of Redis, has been building DwarfStar (ds4), a purpose-built C inference engine for running DeepSeek V4 Flash (284B MoE... og:url: https://daily.dev/posts/redis-creator-s-local-llm-runner-is-quietly-becoming-a-real-inference-server-pmfzrazge og:image: https://api.daily.dev/og/posts/PMfzRAZGE.png og:image:alt: Redis creator's local LLM runner is quietly becoming a real inference server og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Redis creator's local LLM runner is quietly becoming a real inference server **[Trends](https://daily.dev/sources/trends)** · 2 min read · 2 upvotes · 0 comments ## Summary Salvatore Sanfilippo, creator of Redis, has been building DwarfStar (ds4), a purpose-built C inference engine for running DeepSeek V4 Flash (284B MoE parameters) on consumer hardware. Key engineering choices include asymmetric 2-bit quantization on MoE expert weights, SSD weight streaming for memory-constrained machines, persistent KV cache to disk for large context windows, and distributed inference via layer splitting. On a 128 GB M3 Max MacBook Pro it achieves ~26 tokens/sec with an 81 GB quantized model. The target use case is long, unattended agentic tasks prioritizing privacy and zero API cost. Recent microbatching experiments on an 8xL40S server yielded 120 tokens/sec aggregated generation and 2,000 tokens/sec prefill, suggesting the project is evolving from a personal experiment into a legitimate multi-user inference server. It has 15,500 GitHub stars and is MIT licensed. ## Content Salvatore Sanfilippo, the person who built Redis, has been quietly shipping an inference engine called DwarfStar that runs DeepSeek V4 Flash (284 billion parameters) on consumer hardware. The numbers are moving fast enough that it's worth paying attention. The setup: DwarfStar is written in C, purpose-built for DeepSeek V4, and uses aggressive 2-bit quantization on the MoE expert weights to squeeze the model into machines with 128GB unified memory. It also streams weights from SSD when RAM runs short and persists the KV cache to disk, which means large context windows become a storage question rather than a memory wall. Sessions survive restarts without reprocessing. When it launched a few weeks ago, a 128GB M3 Max MacBook Pro was hitting around 26 tokens/second. That's already usable for long agentic tasks — overnight code refactors, documentation runs — where you care more about privacy and zero API costs than raw speed. Since then, Sanfilippo has kept pushing. A newer branch with Q8 projections and smaller routed experts (68GB total) hit 60 t/s generation and 550 t/s prefill on Apple Silicon. He then implemented microbatching for a friend's 8xL40S server and got 120 t/s aggregated generation with 2000 t/s prefill — "basically a fast multi-user server," in his words. He didn't expect to get there. Distributed inference is also in the picture. An early test using tensor parallelism over RDMA across two M3 Ultras showed meaningful speedups over a single machine. Meanwhile, a separate team got DeepSeek V4 Flash running at 32 t/s fully local on a Ryzen AI MAX+ 395 (Strix Halo), showing this isn't just an Apple Silicon story. The project is MIT-licensed, already has 15,500 GitHub stars, and is moving fast. If you've been assuming frontier-scale local inference is still a few years out, DwarfStar is a decent reason to revisit that assumption. ## Questions this post answers ### What token-per-second speeds can DwarfStar achieve running DeepSeek V4 Flash on a MacBook Pro? A 128GB M3 Max MacBook Pro initially hit around 26 tokens/second at launch. A newer branch using Q8 projections and smaller routed experts (68GB total) reached 60 t/s generation and 550 t/s prefill on Apple Silicon, showing rapid iteration on the same hardware class. _Track fast-moving local inference benchmarks like this one on daily.dev before choosing your own hardware setup._ ### Can DeepSeek V4 run fully locally on AMD hardware instead of Apple Silicon? Yes, a separate team got DeepSeek V4 Flash running fully local on a Ryzen AI MAX+ 395 (Strix Halo) at 32 tokens per second, demonstrating that frontier-scale local inference isn't limited to Apple Silicon machines and works on AMD's unified-memory chips too. _Developers comparing local inference hardware options can follow cross-platform results like this on daily.dev._ ### How does DwarfStar fit a 284 billion parameter model into 128GB of memory? DwarfStar uses aggressive 2-bit quantization on the MoE expert weights of DeepSeek V4 (284B parameters) to shrink it enough for 128GB unified memory machines. It also streams weights from SSD when RAM runs short and persists the KV cache to disk, turning large context windows into a storage problem rather than a memory limit. _Engineers evaluating quantization tradeoffs for local LLM deployment can keep up with projects like DwarfStar on daily.dev._ ## Similar posts on daily.dev - [GitHub - antirez/ds4: DeepSeek 4 Flash local inference engine for Metal](https://daily.dev/posts/github---antirez-ds4-deepseek-4-flash-local-inference-engine-for-metal-sqcj9sya5) · Hacker News · 3 upvotes · 0 comments - [DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0](https://daily.dev/posts/deepseek-v4-1-flash-on-vllm-5x-agentic-throughput-since-day-0-js5lna7tm) · vLLM · 0 upvotes · 0 comments - [Serving DeepSeek-V4 on GB300 with SGLang: 5x Higher Throughput at the Same Interactivity Since Day-0 – PyTorch](https://daily.dev/posts/serving-deepseek-v4-on-gb300-with-sglang-5x-higher-throughput-at-the-same-interactivity-since-day-0-mmxalpv2n) · PyTorch · 1 upvotes · 0 comments --- Tags: [#data-science](https://daily.dev/tags/data-science), [#c](https://daily.dev/tags/c), [#ai-inference](https://daily.dev/tags/ai-inference), [#local-ai](https://daily.dev/tags/local-ai), [#deepseek](https://daily.dev/tags/deepseek) [View this post on daily.dev](https://daily.dev/posts/redis-creator-s-local-llm-runner-is-quietly-becoming-a-real-inference-server-pmfzrazge) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Redis creator's local LLM runner is quietly becoming a real inference server","url":"https://daily.dev/posts/redis-creator-s-local-llm-runner-is-quietly-becoming-a-real-inference-server-pmfzrazge","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/redis-creator-s-local-llm-runner-is-quietly-becoming-a-real-inference-server-pmfzrazge"},"datePublished":"2026-07-18T18:09:21.091Z","dateModified":"2026-10-15T02:22:19.344Z","description":"Salvatore Sanfilippo, creator of Redis, has been building DwarfStar (ds4), a purpose-built C inference engine for running DeepSeek V4 Flash (284B MoE...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f422d8151f489792e6668d05bee4047a?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f422d8151f489792e6668d05bee4047a?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Trends","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Trends","logo":"https://media.daily.dev/image/upload/s--ZfSp3asX--/f_auto,q_auto/v1780996004/logos/trends?_a=BAMAMiWQ0","url":"https://daily.dev/sources/trends"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/redis-creator-s-local-llm-runner-is-quietly-becoming-a-real-inference-server-pmfzrazge","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-science,c,ai-inference,local-ai,deepseek","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Trends","item":"https://daily.dev/sources/trends"},{"@type":"ListItem","position":3,"name":"Redis creator's local LLM runner is quietly becoming a real inference server"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/redis-creator-s-local-llm-runner-is-quietly-becoming-a-real-inference-server-pmfzrazge#faq","mainEntity":[{"@type":"Question","name":"What token-per-second speeds can DwarfStar achieve running DeepSeek V4 Flash on a MacBook Pro?","acceptedAnswer":{"@type":"Answer","text":"A 128GB M3 Max MacBook Pro initially hit around 26 tokens/second at launch. A newer branch using Q8 projections and smaller routed experts (68GB total) reached 60 t/s generation and 550 t/s prefill on Apple Silicon, showing rapid iteration on the same hardware class. Track fast-moving local inference benchmarks like this one on daily.dev before choosing your own hardware setup."}},{"@type":"Question","name":"Can DeepSeek V4 run fully locally on AMD hardware instead of Apple Silicon?","acceptedAnswer":{"@type":"Answer","text":"Yes, a separate team got DeepSeek V4 Flash running fully local on a Ryzen AI MAX+ 395 (Strix Halo) at 32 tokens per second, demonstrating that frontier-scale local inference isn't limited to Apple Silicon machines and works on AMD's unified-memory chips too. Developers comparing local inference hardware options can follow cross-platform results like this on daily.dev."}},{"@type":"Question","name":"How does DwarfStar fit a 284 billion parameter model into 128GB of memory?","acceptedAnswer":{"@type":"Answer","text":"DwarfStar uses aggressive 2-bit quantization on the MoE expert weights of DeepSeek V4 (284B parameters) to shrink it enough for 128GB unified memory machines. It also streams weights from SSD when RAM runs short and persists the KV cache to disk, turning large context windows into a storage problem rather than a memory limit. Engineers evaluating quantization tradeoffs for local LLM deployment can keep up with projects like DwarfStar on daily.dev."}}]} ```