--- title: GitHub - gizmo64k/soulplayer-c64: A real 25k-parameter... description: A fully functional 2-layer decoder-only transformer with ~25,000 int8 parameters has been implemented in hand-written 6502/6510 assembly and runs on an... canonical: https://daily.dev/posts/github---gizmo64k-soulplayer-c64-a-real-25k-parameter-transformer-running-on-a-commodore-64--fbf9hfz8d twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: GitHub - gizmo64k/soulplayer-c64: A real 25k-parameter transformer running on a Commodore 64! | daily.dev og:description: A fully functional 2-layer decoder-only transformer with ~25,000 int8 parameters has been implemented in hand-written 6502/6510 assembly and runs on an... og:url: https://daily.dev/posts/github---gizmo64k-soulplayer-c64-a-real-25k-parameter-transformer-running-on-a-commodore-64--fbf9hfz8d og:image: https://api.daily.dev/og/posts/FbF9HfZ8d.png og:image:alt: GitHub - gizmo64k/soulplayer-c64: A real 25k-parameter transformer running on a Commodore 64! og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # GitHub - gizmo64k/soulplayer-c64: A real 25k-parameter transformer running on a Commodore 64! **[Hacker News](https://daily.dev/sources/hn)** · 6 min read · 0 upvotes · 0 comments ## Summary A fully functional 2-layer decoder-only transformer with ~25,000 int8 parameters has been implemented in hand-written 6502/6510 assembly and runs on an unmodified Commodore 64 at 1 MHz. The architecture mirrors modern LLMs (multi-head causal self-attention, RMSNorm, ReLU MLP) but fits in 25 KB on a floppy disk, generating one token every ~60 seconds. Key engineering challenges included implementing softmax via a 128-entry lookup table, performing all matrix math with shift-and-add (no multiply instruction), and using quantization-aware training (QAT) with power-of-2 scaling to bridge float training and int8 inference. The repo includes training scripts (Python/PyTorch), a 6502 assembler, a memory-faithful Python shadow for testing, and a ready-to-run disk image. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [A Working Nintendo in ~1GB of Qwen Weights](https://daily.dev/posts/a-working-nintendo-in-1gb-of-qwen-weights-sbrwwn5e8) · Hacker News · 0 upvotes · 0 comments - [Autoregressive Language Model on the 6502 Processor](https://daily.dev/posts/autoregressive-language-model-on-the-6502-processor-ufuvsz7ze) · Hacker News · 0 upvotes · 0 comments - [Someone squeezed a 28.9M LLM onto an ESP32-S3, and so can you](https://daily.dev/posts/someone-squeezed-a-28-9m-llm-onto-an-esp32-s3-and-so-can-you-a5f9lzpva) · XDA Developers · 2 upvotes · 0 comments - [I Built a Tiny Computer Inside a Transformer](https://daily.dev/posts/i-built-a-tiny-computer-inside-a-transformer-onwjlebem) · Towards Data Science · 0 upvotes · 0 comments --- Tags: [#python](https://daily.dev/tags/python), [#data-science](https://daily.dev/tags/data-science) [View this post on daily.dev](https://daily.dev/posts/github---gizmo64k-soulplayer-c64-a-real-25k-parameter-transformer-running-on-a-commodore-64--fbf9hfz8d) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"GitHub - gizmo64k/soulplayer-c64: A real 25k-parameter transformer running on a Commodore 64!","url":"https://daily.dev/posts/github---gizmo64k-soulplayer-c64-a-real-25k-parameter-transformer-running-on-a-commodore-64--fbf9hfz8d","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/github---gizmo64k-soulplayer-c64-a-real-25k-parameter-transformer-running-on-a-commodore-64--fbf9hfz8d"},"datePublished":"2026-04-21T02:53:35.963Z","dateModified":"2026-04-21T02:53:59.024Z","description":"A fully functional 2-layer decoder-only transformer with ~25,000 int8 parameters has been implemented in hand-written 6502/6510 assembly and runs on an...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/5a15be74d1eaff0266015131bf2876d5?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/5a15be74d1eaff0266015131bf2876d5?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/github---gizmo64k-soulplayer-c64-a-real-25k-parameter-transformer-running-on-a-commodore-64--fbf9hfz8d","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"python,data-science","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"GitHub - gizmo64k/soulplayer-c64: A real 25k-parameter transformer running on a Commodore 64!"}]} ```