--- title: GitHub - Carloscodix/qapla: A char-level transformer... description: A developer trained a character-level transformer from scratch entirely on an $8 ESP32-S3 microcontroller — no GPU, no pre-trained weights, no inference... canonical: https://daily.dev/posts/github---carloscodix-qapla-a-char-level-transformer-trained-from-scratch-on-an-8-esp32-s3-not-inf-6nizth4qa twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: GitHub - Carloscodix/qapla: A char-level transformer trained from scratch on an $8 ESP32-S3. Not inference: the chip runs the full training loop, with backprop written by hand in C. | daily.dev og:description: A developer trained a character-level transformer from scratch entirely on an $8 ESP32-S3 microcontroller — no GPU, no pre-trained weights, no inference... og:url: https://daily.dev/posts/github---carloscodix-qapla-a-char-level-transformer-trained-from-scratch-on-an-8-esp32-s3-not-inf-6nizth4qa og:image: https://api.daily.dev/og/posts/6niZth4Qa.png og:image:alt: GitHub - Carloscodix/qapla: A char-level transformer trained from scratch on an $8 ESP32-S3. Not inference: the chip runs the full training loop, with backprop written by hand in C. og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # GitHub - Carloscodix/qapla: A char-level transformer trained from scratch on an $8 ESP32-S3. Not inference: the chip runs the full training loop, with backprop written by hand in C. **[Hacker News](https://daily.dev/sources/hn)** · 13 min read · 0 upvotes · 0 comments ## Summary A developer trained a character-level transformer from scratch entirely on an $8 ESP32-S3 microcontroller — no GPU, no pre-trained weights, no inference tricks. The model (~319K parameters) learns the Klingon language character by character, with the full training loop including forward pass, hand-written backpropagation in C, and weight updates (SGD with momentum + cosine LR) running on-chip. Memory constraints are the core engineering challenge: training requires weights, gradients, momentum buffers, activations, and corpus all simultaneously in a few MB of PSRAM. Gradients are verified against finite differences (worst relative error 1.07e-08). Training ran for ~2 days on a phone charger, reaching a moving-average loss of ~1.87. The generated text shows real morphological patterns from Klingon despite no explicit grammar rules being provided. The project is open source (Apache 2.0) and fully reproducible with PlatformIO. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community take How the wider developer community reacted, aggregated from 1 discussion and 32 comments across hackernews (as of 2026-08-05). **TL;DR:** The community finds the project genuinely impressive as an engineering feat — training a transformer with hand-written backprop on a $8 microcontroller — but there's notable skepticism about whether the README was AI-generated and mild debate about the practical usefulness of the Klingon demo. **Sentiment:** 45% positive · 35% mixed · 20% skeptical **The case for** - Running the full training loop including hand-written backpropagation in C on a $8 chip is a legitimately impressive engineering challenge. - Gradient correctness is rigorously verified against finite differences (worst relative error 1.07e-08), lending credibility to the implementation. - The project opens up interesting possibilities like autonomous edge-training on sensor data powered by solar panels. **The pushback** - The README is widely suspected to be LLM-generated, undermining trust in the project's authenticity. - The Klingon demo is seen as a novelty with little practical value; a sensor-data training example would better illustrate real-world applicability. - At least one commenter suspects the author account itself may be an LLM, raising deeper authenticity concerns. **By community** - hackernews (mixed): Commenters appreciate the engineering ambition but are distracted by suspicions of AI-generated documentation and debate the practical relevance of the Klingon use case. **Hottest debate:** Whether the README (and possibly the author's responses) are AI-generated rather than written by a human developer. **Open questions** - Could clusters of ESP32-S3s be interconnected for distributed edge training, and what would the bottleneck be? - What would a practical, non-novelty application of on-device transformer training look like (e.g., training on live sensor data)? - Is maximum training-state memory rather than parameter count the true limiting variable for scaling this approach? **Highlights** > You're talking to an LLM, unfortunately https://news.ycombinator.com/threads?id=runtime_lens (enable showdead in your HN settings) > — [Tiberium on hackernews](https://news.ycombinator.com/item?id=49182649) > I think the readme is clearly LLM generated. The tone, the short sentences, the em dashes, lists, sections.. It all feels like LLM. > — [4gotunameagain on hackernews](https://news.ycombinator.com/item?id=49180510) > This small Klingon speaking language model was trained completely on ESP32. The training took 2 days Number of parameters: 319K (Disclaimer) The models is tiny and is not a pocket chatbot. It mistakes and is not capable to support conversation, but that's not the goal of the project The goals of the project is to bring training to edge devices and it worked out How can one use it? By using solar panels such device could be turned into autonomous meteorological station > — [pavelai on hackernews](https://news.ycombinator.com/item?id=49178465) > Would probably be better to demonstrate by exsmple how this approach is used to train on sensor data and then use it (as is hinted by the author) instead of acknowledging that the klingon poc is useless. > — [chicken-stew on hackernews · 1 comments](https://news.ycombinator.com/item?id=49179260) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49178464) · 51 points · 32 comments ## Similar posts on daily.dev - [28.9M-parameter LLM runs locally on ESP32-S3 at 9 tokens/s](https://daily.dev/posts/28-9m-parameter-llm-runs-locally-on-esp32-s3-at-9-tokens-s-xmajabusu) · CNX Software · 0 upvotes · 0 comments - [GitHub - gizmo64k/soulplayer-c64: A real 25k-parameter transformer running on a Commodore 64\!](https://daily.dev/posts/github---gizmo64k-soulplayer-c64-a-real-25k-parameter-transformer-running-on-a-commodore-64--fbf9hfz8d) · Hacker News · 0 upvotes · 0 comments - [Someone squeezed a 28.9M LLM onto an ESP32-S3, and so can you](https://daily.dev/posts/someone-squeezed-a-28-9m-llm-onto-an-esp32-s3-and-so-can-you-a5f9lzpva) · XDA Developers · 2 upvotes · 0 comments - [Bytes Speak All Languages: Cross-Script Name Retrieval via Contrastive Learning](https://daily.dev/posts/bytes-speak-all-languages-cross-script-name-retrieval-via-contrastive-learning-nexqauuco) · Towards Data Science · 2 upvotes · 0 comments - [GitHub - angelos-p/llm-from-scratch](https://daily.dev/posts/github---angelos-p-llm-from-scratch-5sw4itutx) · Hacker News · 1 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#c](https://daily.dev/tags/c), [#iot](https://daily.dev/tags/iot), [#embedded](https://daily.dev/tags/embedded) [View this post on daily.dev](https://daily.dev/posts/github---carloscodix-qapla-a-char-level-transformer-trained-from-scratch-on-an-8-esp32-s3-not-inf-6nizth4qa) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"GitHub - Carloscodix/qapla: A char-level transformer trained from scratch on an $8 ESP32-S3. Not inference: the chip runs the full training loop, with backprop written by hand in C.","url":"https://daily.dev/posts/github---carloscodix-qapla-a-char-level-transformer-trained-from-scratch-on-an-8-esp32-s3-not-inf-6nizth4qa","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/github---carloscodix-qapla-a-char-level-transformer-trained-from-scratch-on-an-8-esp32-s3-not-inf-6nizth4qa"},"datePublished":"2026-08-05T10:34:03.660Z","dateModified":"2026-08-05T13:36:30.088Z","description":"A developer trained a character-level transformer from scratch entirely on an $8 ESP32-S3 microcontroller — no GPU, no pre-trained weights, no inference...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7906dba39329ce3733e9d11425096f19?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7906dba39329ce3733e9d11425096f19?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/github---carloscodix-qapla-a-char-level-transformer-trained-from-scratch-on-an-8-esp32-s3-not-inf-6nizth4qa","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,c,iot,embedded","timeRequired":"PT13M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"GitHub - Carloscodix/qapla: A char-level transformer trained from scratch on an $8 ESP32-S3. Not inference: the chip runs the full training loop, with backprop written by hand in C."}]} ```