<!-- mobian-agent-page publisher="dailydev" canonical="https://daily.dev/posts/WMB5ng5r8" -->

---
title: I Built a Complete LLM From Scratch in Pure Rust —...
description: A developer built a complete LLM pipeline from scratch in pure Rust — no Python, no PyTorch — called aarambh-ai. The 14-crate workspace covers pretraining, SFT...
canonical: https://daily.dev/posts/wmb5ng5r8
twitter:card: summary_large_image
twitter:site: @dailydotdev
og:type: website
og:site_name: daily.dev
og:title: I Built a Complete LLM From Scratch in Pure Rust — Pretraining, GRPO, Flash Attention, and Self-Learning Without a Single Line of Python | daily.dev
og:description: A developer built a complete LLM pipeline from scratch in pure Rust — no Python, no PyTorch — called aarambh-ai. The 14-crate workspace covers pretraining, SFT...
og:url: https://daily.dev/posts/wmb5ng5r8
og:image: https://api.daily.dev/og/posts/WMB5ng5r8.png
og:image:alt: Post cover image
og:image:width: 1200
og:image:height: 630
og:locale: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# I Built a Complete LLM From Scratch in Pure Rust — Pretraining, GRPO, Flash Attention, and Self-Learning Without a Single Line of Python

**[Rustverse](https://daily.dev/sources/rustverse)** · [@khauma](https://daily.dev/khauma) · 42 upvotes · 4 comments

## Summary

A developer built a complete LLM pipeline from scratch in pure Rust — no Python, no PyTorch — called aarambh-ai. The 14-crate workspace covers pretraining, SFT fine-tuning, LoRA/QLoRA, GRPO reinforcement learning, INT4/GGUF quantization, Flash Attention v2 CUDA kernels, a thinking engine, a safety layer, and an online self-learning loop. Four model scales (25M to 1.3B parameters) share identical code. Key technical details include: AdamW with β₂=0.95 (not 0.999), Cholesky decomposition for stable GPTQ Hessian inversion, tiled Flash Attention achieving 3.5× training speedup on A100, and QLoRA fine-tuning a 117M model in ~400MB RAM. The quantized Tiny model weighs 13MB and runs on CPU with no Python runtime required.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://aarambhdevhub.medium.com/i-built-a-complete-llm-from-scratch-in-pure-rust-pretraining-grpo-flash-attention-and-c585123c14b4>

## Community discussion

Top comments from developers on daily.dev.

**@lilmas\_t** · 0 upvotes

> Is it open source?

**@agustinbarrientos** · 0 upvotes

> A 13 MB Tiny model with no Python runtime fits neatly inside a Rust CLI

**@soapbravo** · 0 upvotes

> Great work!
>
> Though llama.cpp is already no python thing I believe.

## Similar posts on daily.dev

- [6 Things I Learned Building LLMs From Scratch That No Tutorial Teaches You](https://daily.dev/posts/6-things-i-learned-building-llms-from-scratch-that-no-tutorial-teaches-you-iyjs6lyzo) · Towards Data Science · 1 upvotes · 0 comments
- [GitHub - angelos-p/llm-from-scratch](https://daily.dev/posts/github---angelos-p-llm-from-scratch-5sw4itutx) · Hacker News · 1 upvotes · 0 comments
- [How to Fine-Tune LLMs in 2026](https://daily.dev/posts/how-to-fine-tune-llms-in-2026-q6hsrxnzr) · Daily Dose of Data Science \| Avi Chawla \| Substack · 0 upvotes · 0 comments

---

Tags: [#data-science](https://daily.dev/tags/data-science), [#llm](https://daily.dev/tags/llm), [#rust](https://daily.dev/tags/rust)

[View this post on daily.dev](https://daily.dev/posts/wmb5ng5r8)

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]}
{"@context":"https://schema.org","@type":"DiscussionForumPosting","mainEntityOfPage":"https://daily.dev/posts/wmb5ng5r8","headline":"I Built a Complete LLM From Scratch in Pure Rust — Pretraining, GRPO, Flash Attention, and Self-Learning Without a Single Line of Python","text":"Shared: I Built a Complete LLM From Scratch in Pure Rust — Pretraining, GRPO, Flash Attention, and Self-Learning Without a Single Line of Python","url":"https://daily.dev/posts/wmb5ng5r8","datePublished":"2026-07-21T08:22:28.934Z","dateModified":"2026-07-21T08:22:28.936Z","author":{"@type":"Person","name":"Kahlil Wallace","url":"https://daily.dev/khauma","image":"https://media.daily.dev/image/upload/s--QvppE9Ip--/f_auto/v1767378558/avatars/avatar_7eQFeSQRDx0muV3n92gxT?_a=BAMAK+ZW0","description":"A nerd who loves software development but is down for longboarding, bouldering or martial arts!","worksFor":{"@type":"Organization","name":"Yellow Duck Technologies","logo":"https://media.daily.dev/image/upload/s--sJ49BN2f--/f_auto,q_auto/v1782805767/logos/yellow-duck-technologies?_a=BAMAMicg0"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"EndorseAction"},"userInteractionCount":16140}},"image":"https://media.daily.dev/image/upload/s--CxzD6vbw--/f_auto/v1722860399/public/Placeholder%2005","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":42},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":4}],"sharedContent":{"@type":"WebPage","url":"https://api.daily.dev/r/d869ptK5U"},"comment":[{"@type":"Comment","text":"Is it open source?","datePublished":"2026-07-27T08:57:13.616Z","url":"https://daily.dev/posts/WMB5ng5r8#c-J5Z1iKOw9","author":{"@type":"Person","name":"Taz","url":"https://daily.dev/lilmas_t","image":"https://media.daily.dev/image/upload/s--VHSP-SxJ--/f_auto/v1743166090/avatars/avatar_SYE1wuv27HcJLPobmWWlB"}},{"@type":"Comment","text":"A 13 MB Tiny model with no Python runtime fits neatly inside a Rust CLI","datePublished":"2026-08-07T03:59:55.695Z","url":"https://daily.dev/posts/WMB5ng5r8#c-cVxBfEJA8","author":{"@type":"Person","name":"Agustin Barrientos","url":"https://daily.dev/agustinbarrientos","image":"https://media.daily.dev/image/upload/s--5ayxQnqn--/f_auto/v1788281802/avatars/avatar_wQYYVe5Tbj0NJ7C7qPoa8?_a=BAMAMicg0"}},{"@type":"Comment","text":"Great work!\nThough llama.cpp is already no python thing I believe.","datePublished":"2026-07-28T14:06:14.790Z","url":"https://daily.dev/posts/WMB5ng5r8#c-y43li5GV5","author":{"@type":"Person","name":"Soap Bravo","url":"https://daily.dev/soapbravo","image":"https://media.daily.dev/image/upload/s--rkKDWshV--/f_auto/v1783071551/avatars/avatar_1D30kcJ92FZKAXwAPCvAl?_a=BAMAMicg0"}}],"isPartOf":{"@type":"WebPage","url":"https://daily.dev/squads/rustverse","name":"Rustverse"}}
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Rustverse","item":"https://daily.dev/squads/rustverse"},{"@type":"ListItem","position":3,"name":"I Built a Complete LLM From Scratch in Pure Rust — Pretraining, GRPO, Flash Attention, and Self-Learning Without a Single Line of Python"}]}
```

