--- title: NanoGPT Slowrun | daily.dev description: NanoGPT Slowrun is an open benchmarking effort by Q Labs focused on data-efficient learning algorithms for language models. Unlike speedrun benchmarks that... canonical: https://daily.dev/posts/nanogpt-slowrun-e0itxta2i twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: NanoGPT Slowrun | daily.dev og:description: NanoGPT Slowrun is an open benchmarking effort by Q Labs focused on data-efficient learning algorithms for language models. Unlike speedrun benchmarks that... og:url: https://daily.dev/posts/nanogpt-slowrun-e0itxta2i og:image: https://api.daily.dev/og/posts/E0ITxTA2i.png og:image:alt: NanoGPT Slowrun og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # NanoGPT Slowrun **[Hacker News](https://daily.dev/sources/hn)** · 3 min read · 0 upvotes · 0 comments ## Summary NanoGPT Slowrun is an open benchmarking effort by Q Labs focused on data-efficient learning algorithms for language models. Unlike speedrun benchmarks that optimize wall-clock time, Slowrun trains on a fixed 100M token dataset (FineWeb) with unlimited compute, rewarding algorithms that achieve the lowest validation loss. Community contributions have already pushed data efficiency from 2.4x to 5.5x versus modded-nanogpt in just days. Key findings include Muon optimizer outperforming AdamW/SOAP/MAGMA, the importance of multi-epoch training with aggressive regularization (weight decay up to 16x standard plus dropout), shuffling at epoch start, learned value embedding projections, SwiGLU activations, and model ensembling. Open research directions include second-order optimizers, diffusion models, curriculum learning, and gradient descent alternatives. The project aims for 10x data efficiency short-term and potentially 100x by year-end. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [10x Data Efficiency](https://daily.dev/posts/10x-data-efficiency-6xohm5dkd) · Hacker News · 1 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#llm](https://daily.dev/tags/llm) [View this post on daily.dev](https://daily.dev/posts/nanogpt-slowrun-e0itxta2i) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"NanoGPT Slowrun","url":"https://daily.dev/posts/nanogpt-slowrun-e0itxta2i","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/nanogpt-slowrun-e0itxta2i"},"datePublished":"2026-03-04T21:48:35.871Z","dateModified":"2026-03-04T21:48:54.299Z","description":"NanoGPT Slowrun is an open benchmarking effort by Q Labs focused on data-efficient learning algorithms for language models. Unlike speedrun benchmarks that...","image":"https://media.daily.dev/image/upload/s--58gMhC4P--/f_auto/v1722860399/public/Placeholder%2012","thumbnailUrl":"https://media.daily.dev/image/upload/s--58gMhC4P--/f_auto/v1722860399/public/Placeholder%2012","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/nanogpt-slowrun-e0itxta2i","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,llm","timeRequired":"PT3M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"NanoGPT Slowrun"}]} ```