--- title: Deltafin runs Kimi K3 (2.8T parameter MoE) on a single... description: Deltafin is a research project enabling local inference of Kimi K3, a 2.8 trillion parameter Mixture-of-Experts model, on a single Apple Silicon Mac. It... canonical: https://daily.dev/posts/deltafin-runs-kimi-k3-2-8t-parameter-moe-on-a-single-apple-silicon-mac-by-streaming-expert-weights-dsyu9p93x twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Deltafin runs Kimi K3 (2.8T parameter MoE) on a single Apple Silicon Mac by streaming expert weights on demand | daily.dev og:description: Deltafin is a research project enabling local inference of Kimi K3, a 2.8 trillion parameter Mixture-of-Experts model, on a single Apple Silicon Mac. It... og:url: https://daily.dev/posts/deltafin-runs-kimi-k3-2-8t-parameter-moe-on-a-single-apple-silicon-mac-by-streaming-expert-weights-dsyu9p93x og:image: https://api.daily.dev/og/posts/Dsyu9p93X.png og:image:alt: Deltafin runs Kimi K3 (2.8T parameter MoE) on a single Apple Silicon Mac by streaming expert weights on demand og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Deltafin runs Kimi K3 (2.8T parameter MoE) on a single Apple Silicon Mac by streaming expert weights on demand **[Collections](https://daily.dev/sources/collections)** · 2 min read · 1 upvotes · 0 comments ## Summary Deltafin is a research project enabling local inference of Kimi K3, a 2.8 trillion parameter Mixture-of-Experts model, on a single Apple Silicon Mac. It exploits MoE sparsity — only 16 of 896 experts activate per token — by streaming MXFP4-quantized expert weights on demand via HTTP range requests and caching them to local NVMe. A ~114 GB 'resident spine' stays on disk while other weights are fetched as needed. Optimizations include fused NEON MXFP4 dequant+GEMV kernels, double-buffered layer loading, parallel expert reads, int8 spine quantization, custom Metal shaders, and n-gram speculation. On an M1 Max with 64 GB RAM, performance is ~15 seconds per token locally or ~3 minutes per token in streaming mode. An OpenAI-compatible API server is included. The project is explicitly a research artifact and existence proof rather than a practical chat solution. ## Content Someone built a C inference engine that runs the full 2.78-trillion-parameter Kimi K3 model on a consumer MacBook Pro. The catch: it does 0.49–0.54 tokens per second, which antirez clocked as "a bit slow" while watching it stream 1.6TB of weights in mxfp4 on an M5 Max with 128GB RAM. The project is called WASTE (Weight-Aware Streaming Tensor Engine), and the core trick is refusing to load the full 982 GB model into RAM. Instead, it keeps the model trunk resident at around 27 GB and streams only the expert weights that actually get activated per token, pulling them directly from NVMe and bypassing the OS page cache entirely. Remaining RAM acts as a bounded expert cache. On a 64 GB M5 Pro MacBook, that gets you just under half a token per second. The engineering is genuinely interesting. WASTE uses 3-bit residual vector quantization for the experts, implements an absorbed KV cache for K3's hybrid attention architecture, and supports multimodal inputs through a 27-layer ViT. It also ships an OpenAI-compatible HTTP server and runs on macOS, Linux, and Windows. No dependencies. Pure C. The repo is unusually honest about what didn't work: it documents every optimization that was tried and rejected, with performance tables and rationale. That's rare. The obvious question is whether sub-1 token/second is useful for anything beyond proving the concept. For interactive use, no. For batch inference on a machine you already own, maybe. The more interesting framing is what this says about MoE architectures: because K3 only activates a fraction of its experts per token, you can stream the rest from disk without the model completely falling apart. That's the architectural property WASTE is exploiting. Antirez's reaction was dry but not dismissive. Half a token per second on a laptop running a nearly 3-trillion-parameter model is either impressive or useless depending on what you were hoping to do with it. ## Similar posts on daily.dev - [GitHub - sqliteai/waste: Run the full 2.78-trillion-parameter Kimi K3 model beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine.](https://daily.dev/posts/github---sqliteai-waste-run-the-full-2-78-trillion-parameter-kimi-k3-model-beyond-available-ram-by--o7y68bmgf) · Hacker News · 0 upvotes · 0 comments - [Kimi K3 runs on 8GB CPU, No GPU](https://daily.dev/posts/kimi-k3-runs-on-8gb-cpu-no-gpu-khbh5h19l) · Medium · 0 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#llm](https://daily.dev/tags/llm), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts) [View this post on daily.dev](https://daily.dev/posts/deltafin-runs-kimi-k3-2-8t-parameter-moe-on-a-single-apple-silicon-mac-by-streaming-expert-weights-dsyu9p93x) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Deltafin runs Kimi K3 (2.8T parameter MoE) on a single Apple Silicon Mac by streaming expert weights on demand","url":"https://daily.dev/posts/deltafin-runs-kimi-k3-2-8t-parameter-moe-on-a-single-apple-silicon-mac-by-streaming-expert-weights-dsyu9p93x","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/deltafin-runs-kimi-k3-2-8t-parameter-moe-on-a-single-apple-silicon-mac-by-streaming-expert-weights-dsyu9p93x"},"datePublished":"2026-07-28T23:52:56.909Z","dateModified":"2026-07-31T18:47:25.325Z","description":"Deltafin is a research project enabling local inference of Kimi K3, a 2.8 trillion parameter Mixture-of-Experts model, on a single Apple Silicon Mac. It...","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/deltafin-runs-kimi-k3-2-8t-parameter-moe-on-a-single-apple-silicon-mac-by-streaming-expert-weights-dsyu9p93x","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,llm,mixture-of-experts","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"Deltafin runs Kimi K3 (2.8T parameter MoE) on a single Apple Silicon Mac by streaming expert weights on demand"}]} ```