--- title: GitHub - leonickson1/Swiftlet | daily.dev description: Swiftlet is an open-source Swift + Metal runtime that enables running large 35B and 80B Qwen3 Mixture-of-Experts models on ordinary Apple devices, including... canonical: https://daily.dev/posts/github---leonickson1-swiftlet-phmsh7j3c twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: GitHub - leonickson1/Swiftlet | daily.dev og:description: Swiftlet is an open-source Swift + Metal runtime that enables running large 35B and 80B Qwen3 Mixture-of-Experts models on ordinary Apple devices, including... og:url: https://daily.dev/posts/github---leonickson1-swiftlet-phmsh7j3c og:image: https://api.daily.dev/og/posts/pHmsh7j3C.png og:image:alt: GitHub - leonickson1/Swiftlet og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # GitHub - leonickson1/Swiftlet **[Hacker News](https://daily.dev/sources/hn)** · 6 min read · 0 upvotes · 0 comments ## Summary Swiftlet is an open-source Swift + Metal runtime that enables running large 35B and 80B Qwen3 Mixture-of-Experts models on ordinary Apple devices, including iPhones. It achieves this by keeping only the small dense core (~1.3–2.5 GB) resident in RAM while streaming routed expert weights from SSD on demand. The 35B model runs in just 2.6 GB of RAM at 7–11 tok/s on an M5 Mac, and even on an iPhone 17 at ~1 tok/s — claimed to be the first model of this class to run natively on a phone. It provides a Swift package, CLI tools, an OpenAI-compatible local server, and an iOS app (Priv AI). The runtime uses LFU+recency expert caching, fixed-stride .qpack containers for efficient pread fetches, runtime-compiled Metal shaders, and Gated DeltaNet linear attention to avoid a growing KV cache. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community take How the wider developer community reacted, aggregated from 1 discussion and 89 comments across hackernews (as of 2026-08-04). **TL;DR:** The community is broadly enthusiastic about Swiftlet as a meaningful step toward running large models on consumer hardware, though many debate whether SSD-streaming is truly practical given prefill bottlenecks, drive wear concerns, and the economics of centralized inference versus local compute. **Sentiment:** 55% positive · 30% mixed · 15% skeptical **The case for** - Streaming expert weights from SSD is seen as a clever architectural approach that enables models far too large for RAM to run at all on consumer devices. - The project is praised as foundational work — iterative improvements like this historically accelerate progress once motivated amateurs can access the problem. - Privacy and network-independence are cited as compelling advantages of on-device inference that pure economics comparisons to cloud servers miss. - Memory bandwidth on Apple Silicon has been steadily increasing, making the trajectory toward practical local inference credible. - The LFU+recency expert caching strategy is noted as a smart way to keep the most-used experts resident and reduce SSD reads. **The pushback** - Prefill is the real bottleneck — processing 10k tokens could take roughly half an hour on an M5, making it impractical for interactive or coding use cases. - The economics of centralized inference (parallelism, cost per token) heavily favor cloud servers for most users. - Some commenters are skeptical that SSD-offload projects represent genuine progress toward truly capable local inference rather than an illusion of it. - Read disturb on NAND flash under heavy sequential read workloads may be a real concern, with one analysis suggesting months-scale SSD lifespan under 24/7 streaming. - The 'first model of this class on a phone' claim is questioned, with a link to a prior 400B model running on iPhone cited as a counterexample. **By community** - hackernews (mixed): Broadly optimistic about the long-term trajectory of on-device inference, but divided on whether SSD-streaming is practical today given prefill speed, drive longevity, and cloud economics. **Hottest debate:** Whether SSD-offload inference represents meaningful progress toward practical local LLMs or is fundamentally limited by prefill bottlenecks and drive wear. **Open questions** - Can expert caching be made tunable so users with more RAM (e.g. 32 GB) can trade storage reads for speed? - How serious is NAND read disturb under sustained weight-streaming workloads in practice? - Is the 'first model of this class on a phone' claim accurate given prior projects like ANEMLL's 400B iPhone demo? - Will memory bandwidth improvements on Apple Silicon eventually make SSD offloading unnecessary for models of this size? **Highlights** > I know everyone wants to crap all over these setups that are impractical, but this is how progress happens. People will keep plugging away at this and figure out how to avoid wearing the hard drive, how to make it run faster, custom hardware buses etc. Keep going! I personally can't wait for the day when a 1t param model runs off a $200 SSD instead of a $50k rack of Nvidia chips. > — [dghlsakjg on hackernews · 8 comments](https://news.ycombinator.com/item?id=49164268) > this is cool but like, are we just vibe coding NAND burners at this point? these decode times don't really tell the whole story, because prefill becomes the bottleneck. half an hour to process 10k tokens on an M5 seems... not great > — [brrrrrm on hackernews · 2 comments](https://news.ycombinator.com/item?id=49164178) > We'd bought 4 x $11K Mac Studios at my college and via exo, we had Kimi K2.5 at 30 TPS. Not too wild an idea! > — [sh3rl0ck on hackernews](https://news.ycombinator.com/item?id=49165647) > > but this is how progress happens. This is progress in the same way that a man climbing a tree is making progress toward reaching the moon. This project is essentially the MoE-of-the day, with some platform-related optimizations. > Keep going! I personally can't wait for the day when a 1t param model runs off a $200 SSD instead of a $50k rack of Nvidia chips. They won't. Projects like this give the illusion that that will be possible. > — [pizza234 on hackernews · 3 comments](https://news.ycombinator.com/item?id=49166284) > "As far as we know, that is the first time a model of this class has run natively on a phone.". I feel like I've seen a similar statement on a lot of these streaming weight projects. 400b model on an iPhone: https://x.com/anemll/status/2035901335984611412 > — [sallymander on hackernews](https://news.ycombinator.com/item?id=49165648) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49158333) · 191 points · 89 comments ## Similar posts on daily.dev - [Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac](https://daily.dev/posts/show-hn-open-source-engine-running-gemma-4-26b-in-2-gb-ram-on-any-m-series-mac-nwy9umvdc) · Hacker News · 0 upvotes · 0 comments - [GitHub - danveloper/flash-moe: Running a big model on a small laptop](https://daily.dev/posts/github---danveloper-flash-moe-running-a-big-model-on-a-small-laptop-xsgaeefuf) · Hacker News · 3 upvotes · 0 comments - [\[Hands-on\] Deploy and Run LLMs on your Phone\!](https://daily.dev/posts/hands-on-deploy-and-run-llms-on-your-phone--eh6og7s71) · Daily Dose of Data Science \| Avi Chawla \| Substack · 37 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#swift](https://daily.dev/tags/swift), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts) [View this post on daily.dev](https://daily.dev/posts/github---leonickson1-swiftlet-phmsh7j3c) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"GitHub - leonickson1/Swiftlet","url":"https://daily.dev/posts/github---leonickson1-swiftlet-phmsh7j3c","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/github---leonickson1-swiftlet-phmsh7j3c"},"datePublished":"2026-08-04T04:40:56.424Z","dateModified":"2026-08-04T11:17:11.769Z","description":"Swiftlet is an open-source Swift + Metal runtime that enables running large 35B and 80B Qwen3 Mixture-of-Experts models on ordinary Apple devices, including...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/adf21e90dbfcc08e983cb284acbd02bf?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/adf21e90dbfcc08e983cb284acbd02bf?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/github---leonickson1-swiftlet-phmsh7j3c","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,swift,mixture-of-experts","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"GitHub - leonickson1/Swiftlet"}]} ```