--- title: Running a 28.9M parameter LLM on an $8 microcontroller description: A 28.9 million parameter language model runs entirely on an ESP32-S3 microcontroller costing about $8, achieving roughly 9.5 tokens per second with no server... canonical: https://daily.dev/posts/running-a-28-9m-parameter-llm-on-an-8-microcontroller-zoblyzaer twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Running a 28.9M parameter LLM on an $8 microcontroller | daily.dev og:description: A 28.9 million parameter language model runs entirely on an ESP32-S3 microcontroller costing about $8, achieving roughly 9.5 tokens per second with no server... og:url: https://daily.dev/posts/running-a-28-9m-parameter-llm-on-an-8-microcontroller-zoblyzaer og:image: https://api.daily.dev/og/posts/ZOblyZaer.png og:image:alt: Running a 28.9M parameter LLM on an $8 microcontroller og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Running a 28.9M parameter LLM on an $8 microcontroller **[Hacker News](https://daily.dev/sources/hn)** · 4 min read · 3 upvotes · 0 comments ## Summary A 28.9 million parameter language model runs entirely on an ESP32-S3 microcontroller costing about $8, achieving roughly 9.5 tokens per second with no server connectivity. The key innovation is using Google's Per-Layer Embeddings technique (from Gemma 3n/4) to store 25 million parameters in slow flash memory rather than the chip's 512KB SRAM, reading only ~450 bytes per token. The model is quantized to 4-bit and occupies 14.9MB of flash. Trained on TinyStories, it generates simple coherent stories but cannot answer questions or follow instructions. This represents roughly 100x more parameters than the previous record for this class of chip. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community take How the wider developer community reacted, aggregated from 1 discussion and 78 comments across hackernews (as of 2026-07-30). **TL;DR:** The community is genuinely excited about running a small LLM on an ESP32-S3 microcontroller, praising the impressive token rate and the clever per-layer embedding trick, with side discussions about cheap MCU options and broader implications for edge AI. **Sentiment:** 65% positive · 25% mixed · 10% skeptical **The case for** - 9.7 tokens/sec on such constrained hardware is seen as surprisingly impressive performance. - The per-layer embedding trick is highlighted as a clever and novel approach enabling this feat. - Commenters note potential real-world applications like offline TTS/voice assistants and computer vision on edge devices. - The ESP32-S3 is praised as exceptional value (~$8 for a board with PSRAM, flash, and dual USB-C). - Flash memory reads are effectively unlimited, so wear is not a concern for inference workloads. **The pushback** - Skeptics question the practical utility of running models on such constrained devices for most use cases. - The approach may not scale easily to larger models due to bandwidth constraints between flash and SRAM. - Model accuracy of the heavily quantized weights is left unaddressed. **By community** - hackernews (positive): Commenters are enthusiastic about the technical achievement and spin off into lively discussions about cheap MCUs, edge AI applications, and RISC-V boards like Milk-V, with only mild skepticism about practical utility. **Hottest debate:** Whether running ML models on MCUs is a pointless exercise or a legitimate and valuable engineering challenge worth pursuing. **Open questions** - How accurate is the quantized model compared to full-precision versions? - What is the best option for running a local LLM on a Raspberry Pi 4 with acceptable latency? - Could a layer-per-MCU cluster (e.g., RP2350s connected via PIO) be a viable inference architecture? **Highlights** > This is a really neat use of the per-layer embedding trick. It's also worth noting that there viable TTS models that are ~20-30M param, so it might mean you can have a ESP32 with no network access read stuff out to you in near real time! > — [rao-v on hackernews · 1 comments](https://news.ycombinator.com/item?id=49053141) > Many artificial environments are useless to tasks they were not designed for until somebody experiments, tests, redesigns, and iterates. I'll give a specific example apropos of TFA. Computer vision models were never run on MCUs because they were constrained to the point of being useless for this task, but then someone tried the impractical, and now it's trivial[1] [1]: https://github.com/espressif/esp-vision > — [thomascountz on hackernews](https://news.ycombinator.com/item?id=49055589) > One of the things I have been wanting to try for a while now is something like this with a layer per MCU. I have some crazy ideas with RP2350's talking to each other with dedicated lines fed by PIO going through a combination of interpolators and dual multiply instructions. PSRAM, Flash, and even SD cards may not have the best bandwidth individually, but they can reach quite impressive rates when you have a shitton of them running all at the same time. The large scale dedicated hardware systems will still have the edge for performance per watt, but the low entry level and slow incline does make these things quite appealing. > — [Lerc on hackernews · 2 comments](https://news.ycombinator.com/item?id=49053364) > Someone did this exact thing recently, but running GLM-5.2 with something like 16GB of DRAM, a standard desktop CPU and nVME SSD. I think they got something like 10 _seconds per token_ (not tokens per second). EDIT: it was 25GB of ram and up to 20 seconds per token! https://github.com/JustVugg/colibri > — [AussieWog93 on hackernews · 4 comments](https://news.ycombinator.com/item?id=49053782) > Why can't this scale to run much larger models on CPU backed by flash with good access patterns? > — [chrishynes on hackernews · 2 comments](https://news.ycombinator.com/item?id=49053161) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49050512) · 93 points · 78 comments ## Similar posts on daily.dev - [28.9M-parameter LLM runs locally on ESP32-S3 at 9 tokens/s](https://daily.dev/posts/28-9m-parameter-llm-runs-locally-on-esp32-s3-at-9-tokens-s-xmajabusu) · CNX Software · 0 upvotes · 0 comments - [Someone squeezed a 28.9M LLM onto an ESP32-S3, and so can you](https://daily.dev/posts/someone-squeezed-a-28-9m-llm-onto-an-esp32-s3-and-so-can-you-a5f9lzpva) · XDA Developers · 2 upvotes · 0 comments - [GitHub - Carloscodix/qapla: A char-level transformer trained from scratch on an $8 ESP32-S3. Not inference: the chip runs the full training loop, with backprop written by hand in C.](https://daily.dev/posts/github---carloscodix-qapla-a-char-level-transformer-trained-from-scratch-on-an-8-esp32-s3-not-inf-6nizth4qa) · Hacker News · 0 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#python](https://daily.dev/tags/python), [#llm](https://daily.dev/tags/llm), [#iot](https://daily.dev/tags/iot), [#embedded](https://daily.dev/tags/embedded) [View this post on daily.dev](https://daily.dev/posts/running-a-28-9m-parameter-llm-on-an-8-microcontroller-zoblyzaer) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Running a 28.9M parameter LLM on an $8 microcontroller","url":"https://daily.dev/posts/running-a-28-9m-parameter-llm-on-an-8-microcontroller-zoblyzaer","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/running-a-28-9m-parameter-llm-on-an-8-microcontroller-zoblyzaer"},"datePublished":"2026-07-26T00:11:10.760Z","dateModified":"2026-07-30T19:09:19.878Z","description":"A 28.9 million parameter language model runs entirely on an ESP32-S3 microcontroller costing about $8, achieving roughly 9.5 tokens per second with no server...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/ec814cb03bb19a7210106d5101e1d2b5?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/ec814cb03bb19a7210106d5101e1d2b5?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/running-a-28-9m-parameter-llm-on-an-8-microcontroller-zoblyzaer","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":3},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,python,llm,iot,embedded","timeRequired":"PT4M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"Running a 28.9M parameter LLM on an $8 microcontroller"}]} ```