DwarfStar is a new open-source inference engine written in C by Salvatore Sanfilippo (creator of Redis), designed to run the DeepSeek V4 Flash frontier model locally on consumer Apple Silicon hardware. It achieves this through aggressive asymmetric 2-bit quantization of MoE expert weights, SSD streaming for machines with insufficient RAM, persistent KV cache sessions on disk, and distributed inference across multiple machines via layer splitting. On a 128 GB M3 Max MacBook Pro, it achieves ~26 tokens/s generation with an 81 GB quantized model. The project is MIT-licensed, free, and already has 15,500 GitHub stars weeks after launch. It also explores ensemble inference combining logits from multiple model instances.
Table of contents
Antirez: the programmer who does one thing, and does it wellThe problem: frontier models live on SaturnMixture of Experts: when specialization becomes an advantageWhen RAM is not enough: the disk as an extension of memoryContext, sessions, and memory that survives rebootTwo machines are worth more than oneThe numbers: what to expect in practiceGLM 5.2 and the project’s directionThe frontier, open to everyone (almost)1.9K Impressions