--- title: PyTorch: A Reference Language | daily.dev description: PyTorch is increasingly playing a dual role: as both a reference language for deep learning and a production implementation language. As kernel DSLs have made... canonical: https://daily.dev/posts/pytorch-a-reference-language-p0gqyo2ve twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: PyTorch: A Reference Language | daily.dev og:description: PyTorch is increasingly playing a dual role: as both a reference language for deep learning and a production implementation language. As kernel DSLs have made... og:url: https://daily.dev/posts/pytorch-a-reference-language-p0gqyo2ve og:image: https://api.daily.dev/og/posts/p0gqYo2ve.png og:image:alt: PyTorch: A Reference Language og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # PyTorch: A Reference Language **[Hacker News](https://daily.dev/sources/hn)** · 5 min read · 0 upvotes · 0 comments ## Summary PyTorch is increasingly playing a dual role: as both a reference language for deep learning and a production implementation language. As kernel DSLs have made it easier to write optimized operators, LLMs are now enabling a similar pattern for training steps — generating explicit forward-backward code that can be optimized independently from the PyTorch reference implementation. The proposed workflow keeps traditional autograd-based PyTorch code as a reference spec, uses LLMs to generate optimized explicit backward passes, and employs a verifier (bitwise equivalence or translation validation) to ensure correctness. This approach sidesteps the limitations of implicit autograd graphs at scale while keeping PyTorch central to frontier training research. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community take How the wider developer community reacted, aggregated from 2 discussions and 11 comments across hackernews, lobsters (as of 2026-08-03). **TL;DR:** The community largely agrees that PyTorch has become the de facto lingua franca for deep learning research, and finds the idea of using it as a reference spec while generating optimized backward passes via LLMs plausible — though some question whether this is just sidestepping compiler improvements. **Sentiment:** 55% positive · 30% mixed · 15% skeptical **The case for** - Researchers already think and write in PyTorch, so tooling that adapts to them rather than the reverse is a natural fit. - PyTorch (and Python) have genuinely become a lingua franca for ML research, with papers already writing minimal PyTorch training loops as pseudocode. - Compilers can still play a useful role as verifiers in an LLM-assisted workflow, preserving their value rather than eliminating it. - Not every team has kernel engineers, so LLM-generated optimized code with compiler-backed verification offers a practical baseline. **The pushback** - The approach could be seen as giving up on making compilers better rather than solving the underlying problem. - The workflow may only be relevant for large-scale distributed training; for most users, standard PyTorch autograd is sufficient. **By community** - hackernews (mixed): Commenters are broadly receptive to PyTorch as a reference language but debate whether LLM-based code generation sidesteps compiler progress or legitimately complements it, with the post's author (ezyang) engaging directly to defend compilers' continued relevance. - lobsters (mixed): No comments were posted, so no community signal is available. **Hottest debate:** Whether using LLMs to generate optimized backward passes represents a pragmatic evolution or an admission that compiler technology has hit its limits. **Open questions** - How reliable is bitwise-equivalence or translation-validation verification in practice for complex, large-scale training steps? - Will this workflow remain relevant as compiler technology (e.g., torch.compile) continues to improve? - How does this relate to or differ from existing projects like tinygrad that already use a reference-implementation approach? **Highlights** > One of the reasons why I have been thinking and writing this topic is because there is indeed a malaise among the team, which is along the lines of "AI agents are blowing up coding everywhere, what place does a compiler have in this brave new world?" If compilers truly are pointless, then I would like to update on this and save people time and grief! But I also don't think compilers are pointless. In this post, which looks at the maximalist AI situation, the compiler resurrects itself as the verifier: the two types of systems do very similar things! You can also defend compilers on more banal grounds: not everyone has a ton of kernel engineers to hand optimize your kernels; token efficient ways of getting good baselines is useful; there's more to ML than transformer models and compiler leverage is really useful there. There's also a historical point which is that torch.compile... by all objective metrics, has really been quite successful! I do agree it's hard to tell the future. These days, I ask myself, "Do I think this is likely to happen in six months." I don't think compilers die in six months. Ask me again in six months :) > — [ezyang on hackernews · 1 comments](https://news.ycombinator.com/item?id=49084779) > I think this idea is exactly right. Many ML papers that include pseudocode basically just write out a minimal PyTorch training loop - down to the loss.backward(). It has, along with Python, become something of a linga franca for research in the field. > — [ainch on hackernews](https://news.ycombinator.com/item?id=49085999) > The point is there are limits on how much better the compilers can get, without modifying the language to be closer to the target environments. And there are multiple target environments, so a one-size-fits-all solution is not really possible. You have to compromise somewhere - you can’t have a single high level language that automatically produces maximally efficient code for every target, because the targets are not general-purpose computing environments, they have significant constraints. If what you’re looking for is a way to avoid having to get into the operational details of the targets, and just provide a high level description that somehow gets figured out, then an LLM-based solution is almost certainly going to be your best bet. > — [antonvs on hackernews](https://news.ycombinator.com/item?id=49081160) > This kind of sounds like just giving up on making the compiler better. > — [ansk on hackernews · 2 comments](https://news.ycombinator.com/item?id=49080246) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49079454) · 69 points · 11 comments - [lobsters](https://lobste.rs/s/qzomrk/pytorch_reference_language) · 1 points · 0 comments ## Similar posts on daily.dev - [Why PyTorch Won](https://daily.dev/posts/why-pytorch-won-bkjepea9h) · The New Stack · 0 upvotes · 0 comments - [Automating Inference Optimizations with NVIDIA TensorRT LLM AutoDeploy](https://daily.dev/posts/automating-inference-optimizations-with-nvidia-tensorrt-llm-autodeploy-qklxaunml) · NVIDIA Developer · 1 upvotes · 0 comments - [LLMs could be, but shouldn't be compilers](https://daily.dev/posts/llms-could-be-but-shouldn-t-be-compilers-cwutlxyzi) · Hacker News · 1 upvotes · 1 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#deep-learning](https://daily.dev/tags/deep-learning), [#pytorch](https://daily.dev/tags/pytorch) [View this post on daily.dev](https://daily.dev/posts/pytorch-a-reference-language-p0gqyo2ve) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"PyTorch: A Reference Language","url":"https://daily.dev/posts/pytorch-a-reference-language-p0gqyo2ve","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/pytorch-a-reference-language-p0gqyo2ve"},"datePublished":"2026-07-28T13:01:21.625Z","dateModified":"2026-08-03T10:46:37.029Z","description":"PyTorch is increasingly playing a dual role: as both a reference language for deep learning and a production implementation language. As kernel DSLs have made...","image":"https://media.daily.dev/image/upload/s--OHB84bZF--/f_auto/v1722860399/public/Placeholder%2010","thumbnailUrl":"https://media.daily.dev/image/upload/s--OHB84bZF--/f_auto/v1722860399/public/Placeholder%2010","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/pytorch-a-reference-language-p0gqyo2ve","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,deep-learning,pytorch","timeRequired":"PT5M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"PyTorch: A Reference Language"}]} ```