--- title: LLMs From Scratch Reaches 100,000 GitHub Stars | daily.dev description: The LLMs-from-scratch GitHub repository by Sebastian Raschka has surpassed 100,000 stars. The repo covers the full LLM development pipeline from scratch —... canonical: https://daily.dev/posts/llms-from-scratch-reaches-100-000-github-stars-fzt5tx6te twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: LLMs From Scratch Reaches 100,000 GitHub Stars | daily.dev og:description: The LLMs-from-scratch GitHub repository by Sebastian Raschka has surpassed 100,000 stars. The repo covers the full LLM development pipeline from scratch —... og:url: https://daily.dev/posts/llms-from-scratch-reaches-100-000-github-stars-fzt5tx6te og:image: https://api.daily.dev/og/posts/fzt5tx6Te.png og:image:alt: LLMs From Scratch Reaches 100,000 GitHub Stars og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # LLMs From Scratch Reaches 100,000 GitHub Stars **[Sebastian Raschka](https://daily.dev/sources/sebastianraschka)** · 2 min read · 40 upvotes · 1 comments ## Summary The LLMs-from-scratch GitHub repository by Sebastian Raschka has surpassed 100,000 stars. The repo covers the full LLM development pipeline from scratch — tokenization, attention mechanisms, pretraining, classification, and instruction fine-tuning — along with from-scratch implementations of Llama, Qwen, Gemma, and Olmo. It also includes advanced topics like GQA, MLA, sliding-window attention, Gated DeltaNet, DeepSeek Sparse Attention, mixture-of-experts, KV caching, DPO, LoRA, and memory-efficient weight loading. The author plans to continue adding new material and is working on a larger applied small LLM project. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What topics does the LLMs-from-scratch GitHub repository cover? It covers the complete path of building large language models from scratch, including tokenization, attention, pretraining, classification, and instruction fine-tuning. It also includes from-scratch implementations of Llama, Qwen, Gemma, and Olmo, attention variants like GQA, MLA, sliding-window attention, Gated DeltaNet, DeepSeek Sparse Attention, and mixture-of-experts, plus KV caching, DPO, evaluation, and LoRA materials. _daily.dev surfaces resources like this for developers learning to build LLMs from first principles._ ## Community discussion Top comments from developers on daily.dev. **@agustinbarrientos** · 0 upvotes > Thanks for keeping the RL and reasoning material in separate repos ## Similar posts on daily.dev - [LLM From Scratch is a hands-on workshop where you write every piece of an AI from nothing](https://daily.dev/posts/llm-from-scratch-is-a-hands-on-workshop-where-you-write-every-piece-of-an-ai-from-nothing-ntyfh6hef) · XDA Developers · 0 upvotes · 0 comments - [An LLM From “Scratch”](https://daily.dev/posts/an-llm-from-scratch--liyi2mnpd) · Hackaday · 1 upvotes · 0 comments - [The Architecture Behind Open-Source LLMs](https://daily.dev/posts/the-architecture-behind-open-source-llms-vqh5xymdw) · ByteByteGo · 2 upvotes · 0 comments - [GitHub - angelos-p/llm-from-scratch](https://daily.dev/posts/github---angelos-p-llm-from-scratch-5sw4itutx) · Hacker News · 1 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#deep-learning](https://daily.dev/tags/deep-learning) [View this post on daily.dev](https://daily.dev/posts/llms-from-scratch-reaches-100-000-github-stars-fzt5tx6te) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"LLMs From Scratch Reaches 100,000 GitHub Stars","url":"https://daily.dev/posts/llms-from-scratch-reaches-100-000-github-stars-fzt5tx6te","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/llms-from-scratch-reaches-100-000-github-stars-fzt5tx6te"},"datePublished":"2026-08-07T15:42:47.978Z","dateModified":"2026-09-13T21:45:27.409Z","description":"The LLMs-from-scratch GitHub repository by Sebastian Raschka has surpassed 100,000 stars. The repo covers the full LLM development pipeline from scratch —...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/3e113579f3146ef69fb5f53ff35adf88?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/3e113579f3146ef69fb5f53ff35adf88?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Sebastian Raschka","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Sebastian Raschka","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/eb22d2e73d074c5598d31588ad16a51c","url":"https://daily.dev/sources/sebastianraschka"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/llms-from-scratch-reaches-100-000-github-stars-fzt5tx6te","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":40},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"llm,deep-learning","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Sebastian Raschka","item":"https://daily.dev/sources/sebastianraschka"},{"@type":"ListItem","position":3,"name":"LLMs From Scratch Reaches 100,000 GitHub Stars"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/llms-from-scratch-reaches-100-000-github-stars-fzt5tx6te","comment":[{"@type":"Comment","text":"Thanks for keeping the RL and reasoning material in separate repos","datePublished":"2026-08-15T22:26:52.888Z","url":"https://daily.dev/posts/fzt5tx6Te#c-G5YGVBiJq","author":{"@type":"Person","name":"Agustin Barrientos","url":"https://daily.dev/agustinbarrientos","image":"https://media.daily.dev/image/upload/s--5ayxQnqn--/f_auto/v1788281802/avatars/avatar_wQYYVe5Tbj0NJ7C7qPoa8?_a=BAMAMicg0"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/llms-from-scratch-reaches-100-000-github-stars-fzt5tx6te#faq","mainEntity":[{"@type":"Question","name":"What topics does the LLMs-from-scratch GitHub repository cover?","acceptedAnswer":{"@type":"Answer","text":"It covers the complete path of building large language models from scratch, including tokenization, attention, pretraining, classification, and instruction fine-tuning. It also includes from-scratch implementations of Llama, Qwen, Gemma, and Olmo, attention variants like GQA, MLA, sliding-window attention, Gated DeltaNet, DeepSeek Sparse Attention, and mixture-of-experts, plus KV caching, DPO, evaluation, and LoRA materials. daily.dev surfaces resources like this for developers learning to build LLMs from first principles."}}]} ```