--- title: TorchTPU is open source: native PyTorch on Google TPUs description: Google has open sourced TorchTPU, a PyTorch backend for TPUs built with Meta's PyTorch team, letting existing PyTorch code run on TPUs with minimal edits,... canonical: https://daily.dev/posts/torchtpu-is-open-source-native-pytorch-on-google-tpus-8c1x9zqzd twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: TorchTPU is open source: native PyTorch on Google TPUs | daily.dev og:description: Google has open sourced TorchTPU, a PyTorch backend for TPUs built with Meta's PyTorch team, letting existing PyTorch code run on TPUs with minimal edits,... og:url: https://daily.dev/posts/torchtpu-is-open-source-native-pytorch-on-google-tpus-8c1x9zqzd og:image: https://api.daily.dev/og/posts/8C1x9Zqzd.png og:image:alt: TorchTPU is open source: native PyTorch on Google TPUs og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # TorchTPU is open source: native PyTorch on Google TPUs **[Collections](https://daily.dev/sources/collections)** · 3 min read · 1 upvotes · 0 comments ## Summary Google has open sourced TorchTPU, a PyTorch backend for TPUs built with Meta's PyTorch team, letting existing PyTorch code run on TPUs with minimal edits, often just swapping '.to("cuda")' for '.to("tpu")'. It supports eager execution and torch.compile across roughly 450 Aten ops, with a DeferAndFuse eager mode running 50% or more faster than strict eager. 4,821 of 4,917 tested Hugging Face models passed (98%), and numerics matched GPU on 638 of 657 architecture families. A 64-chip TPU v7 pod hit 1,828 tokens per second in training, about 6% behind JAX/MaxText's 1,937. ## Content Google has open-sourced TorchTPU, a native PyTorch backend for TPUs. It replaces PyTorch/XLA, the older route for running PyTorch on TPUs. Google built it with Meta's PyTorch team, starting in mid 2025. The code is on GitHub. ## What changes for developers PyTorch/XLA queued operations for a compiler, so debugging on a TPU felt different from debugging on a GPU. TorchTPU runs each operation as written, which allows step-by-step debugging. For production speed you switch to torch.compile. Existing code needs only small edits, such as swapping `.to("cuda")` for `.to("tpu")`. It supports eager execution, torch.compile, and roughly 450 Aten ops. The repository also includes custom ATen kernels, a compilation cache for XLA execution, and data and tensor parallelism. Examples cover Llama, Qwen, ResNet, DLRM, minGPT and UNet. The latest public release on PyPI is 0.2.0. Builds use Bazel and need Python 3.11 or newer. ## Numbers - 4,821 of 4,917 tested Hugging Face models passed (98%). - Numerics matched GPU results on 638 of 657 architecture families (97%). - The DeferAndFuse eager mode can run 50% or more faster than strict eager execution. - Training on a 64-chip TPU v7 pod reached 1,828 tokens per second, against 1,937 for JAX/MaxText. - TorchTitan has been validated on up to 8,192 Ironwood (v7x) TPUs. It reached 52% Model Flops Utilization on Qwen 3 8B. That last figure is the peak compute number being quoted for Qwen 3 8B training. ## Ecosystem support TorchTPU works with vLLM, SGLang, llm-d, TorchTitan, Ray, TorchRec, Hugging Face, and LiteRT. - Hugging Face Transformers v5 supports TPU deployment through `device_map="tpu"`. - SGLang runs with RadixAttention. - llm-d 1.0, built with Red Hat and IBM, handles fleet-wide routing. - VeRL gets a TPU backend for reinforcement learning, and Ray adds native TPU scheduling. - LiteRT, formerly TensorFlow Lite, joins the PyTorch ecosystem for on-device inference on phones, browsers, and embedded hardware. - A TPU agent suite for automated onboarding and performance tuning is in early preview. ## vLLM TorchTPU vLLM has a new backend built on the TorchTPU stack, made with Google and @inferact. It keeps vLLM's model code, scheduler, KV cache manager, and API server unchanged. Models compile with `torch.compile(backend='tpu')` down to XLA. It ships custom Pallas kernels for attention, Mixture-of-Experts, and quantized matmul (FP8, INT8, MXFP4, NVFP4). All six vLLM parallelism dimensions work across single and multi-host TPU slices. It also supports prefix caching, speculative decoding, KV cache offloading, and disaggregated prefill/decode serving through TPU Sync. TPU v7x (Ironwood) is the recommended, CI-tested hardware. The v6e, v5e, and v5p generations remain experimental. ## Why it matters Amazon already offers a similar native PyTorch backend for its own chips. With both now available, PyTorch code has a documented path off Nvidia GPUs. The numbers above are Google's own, so independent benchmarks will say more about how close TPUs get to GPU-level convenience. ## Questions this post answers ### How does TorchTPU's training performance compare to JAX/MaxText on TPU v7? TorchTPU reached 1,828 tokens per second on a 64-chip TPU v7 pod during training, compared to 1,937 tokens per second for JAX/MaxText on the same hardware. That's roughly a 6% gap, considered small given TorchTPU is a new backend built jointly by Google and Meta's PyTorch team. _Developers evaluating TPUs for PyTorch training can track benchmarks like this on daily.dev._ ### What percentage of Hugging Face models work correctly with TorchTPU? 4,821 of 4,917 tested Hugging Face models passed on TorchTPU, a 98% success rate. Numerics also matched GPU results on 638 of 657 architecture families tested, roughly 97%, suggesting broad compatibility for existing PyTorch model code running on TPUs with minimal changes. _Anyone deciding whether to port models to TPU hardware can follow compatibility data like this on daily.dev._ ### How do I run existing PyTorch code on Google TPUs instead of GPUs? With TorchTPU, you typically just swap '.to("cuda")' for '.to("tpu")' in existing PyTorch code. The backend supports eager execution and torch.compile, covers about 450 Aten ops, and includes a DeferAndFuse eager mode that runs 50% or more faster than strict eager execution. _Developers porting PyTorch workloads to new hardware can follow backend updates like this on daily.dev._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 64 comments across hackernews (as of 2026-10-21). **TL;DR:** The release is seen mainly as Google's push to make TPUs more accessible via first-party PyTorch support (replacing the older pytorch/xla path) and to court customers outside Google, with timing attributed to the PyTorch conference rather than reacting to competitor news. **Sentiment:** 30% positive · 45% mixed · 25% skeptical **The case for** - Seen as finally delivering proper first-party PyTorch support on TPUs after years of fighting XLA integration. - Viewed as a smart move to familiarize more developers with TPU hardware that Google wants to sell externally. - Some see it as chipping away at CUDA's moat by making it easier to target alternative accelerator stacks. **The pushback** - Several note that actual TPU hardware remains hard to obtain ('unobtanium') outside of Google Cloud or select partners, limiting practical usefulness. - Skepticism that the timing is coincidental with other AI news rather than a deliberate strategic release. - Concern that hardware access, not software support, is still the real moat/bottleneck in AI infrastructure. **By community** - hackernews (mixed): Discussion ranges from practical excitement about first-party PyTorch/TPU support to skepticism about hardware availability, release timing, and broader AI market/bubble tangents. **Hottest debate:** Whether the release timing was a deliberate reaction to competitor news or just a coincidence tied to the PyTorch conference. **Open questions** - Who outside Google can actually obtain and use TPU hardware for training or inference? - Does this release reduce CUDA's competitive advantage, or is hardware access still the dominant moat? - What does this mean for JAX's role relative to PyTorch on TPUs going forward? **Highlights** > lol no this is a replacement for https://github.com/pytorch/xla > — [mathisfun123 on hackernews](https://news.ycombinator.com/item?id=50170003) > Yeah a lot of people bought into the idea of CUDA being a moat but we’re well past the point where it’s pretty trivial to jump to another equivalent stack on the software side. > — [AnotherGoodName on hackernews · 2 comments](https://news.ycombinator.com/item?id=50171275) > Those are the two main interfaces: the inference hardware and the endpoints. Endpoints are pretty much standardized to the point that you can switch them out in seconds. CUDA still has an edge but every release like this chips away at that. Even so, TPUs are unobtanium so while I appreciate Google's effort here I also recognize that it is not something that will be useful to me. In that sense AMD is still better positioned to give NVidia a run for their money but given the family connection and AMD's underdog status I am not very hopeful that we'll see meaningful competition in inference hardware any time soon. NVidia is a much harder nut to crack than OpenAI or Anthropic. They are easily a decade ahead of viable competition. Check out the GB300 if you don't believe that. > — [jacquesm on hackernews](https://news.ycombinator.com/item?id=50171463) > For people interested in this: I've been working on an open-source libtpu implementation for https://tenstorrent.com/ accelerators as a side project, it can already run most of torchtpu, see https://github.com/pcmoritz/libtt (based on the tenstorrent software stack). It shows all the things involved in lowering torch to tile based accelerators. > — [pcmoritz on hackernews · 2 comments](https://news.ycombinator.com/item?id=50170447) > it doesn't seem like there's an abundance of shovels coming anytime soon > — [airza on hackernews · 2 comments](https://news.ycombinator.com/item?id=50169820) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=50169053) · 254 points · 64 comments ## Similar posts on daily.dev - [Announcing New vLLM and SGLang Inference Engines on TPU](https://daily.dev/posts/announcing-new-vllm-and-sglang-inference-engines-on-tpu-s4veggelf) · Google Developers · 2 upvotes · 0 comments - [TorchTPU: Running PyTorch Natively on TPUs at Google Scale](https://daily.dev/posts/torchtpu-running-pytorch-natively-on-tpus-at-google-scale-wnrth1jyb) · Google Developers · 0 upvotes · 0 comments - [Google splits its TPU line in two for the agentic era](https://daily.dev/posts/google-splits-its-tpu-line-in-two-for-the-agentic-era-zbjc2yyc7) · The New Stack · 0 upvotes · 0 comments - [Training large models on Ironwood TPUs](https://daily.dev/posts/training-large-models-on-ironwood-tpus-zrglcgqla) · Google Cloud · 1 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#google](https://daily.dev/tags/google), [#pytorch](https://daily.dev/tags/pytorch), [#huggingface](https://daily.dev/tags/huggingface) [View this post on daily.dev](https://daily.dev/posts/torchtpu-is-open-source-native-pytorch-on-google-tpus-8c1x9zqzd) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"TorchTPU is open source: native PyTorch on Google TPUs","url":"https://daily.dev/posts/torchtpu-is-open-source-native-pytorch-on-google-tpus-8c1x9zqzd","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/torchtpu-is-open-source-native-pytorch-on-google-tpus-8c1x9zqzd"},"datePublished":"2026-10-20T16:21:51.875Z","dateModified":"2026-10-21T18:50:49.190Z","description":"Google has open sourced TorchTPU, a PyTorch backend for TPUs built with Meta's PyTorch team, letting existing PyTorch code run on TPUs with minimal edits,...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/26f2656f04d3eb32bb50046a3c314352?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/26f2656f04d3eb32bb50046a3c314352?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/torchtpu-is-open-source-native-pytorch-on-google-tpus-8c1x9zqzd","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,google,pytorch,huggingface","timeRequired":"PT3M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"TorchTPU is open source: native PyTorch on Google TPUs"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/torchtpu-is-open-source-native-pytorch-on-google-tpus-8c1x9zqzd#faq","mainEntity":[{"@type":"Question","name":"How does TorchTPU's training performance compare to JAX/MaxText on TPU v7?","acceptedAnswer":{"@type":"Answer","text":"TorchTPU reached 1,828 tokens per second on a 64-chip TPU v7 pod during training, compared to 1,937 tokens per second for JAX/MaxText on the same hardware. That's roughly a 6% gap, considered small given TorchTPU is a new backend built jointly by Google and Meta's PyTorch team. Developers evaluating TPUs for PyTorch training can track benchmarks like this on daily.dev."}},{"@type":"Question","name":"What percentage of Hugging Face models work correctly with TorchTPU?","acceptedAnswer":{"@type":"Answer","text":"4,821 of 4,917 tested Hugging Face models passed on TorchTPU, a 98% success rate. Numerics also matched GPU results on 638 of 657 architecture families tested, roughly 97%, suggesting broad compatibility for existing PyTorch model code running on TPUs with minimal changes. Anyone deciding whether to port models to TPU hardware can follow compatibility data like this on daily.dev."}},{"@type":"Question","name":"How do I run existing PyTorch code on Google TPUs instead of GPUs?","acceptedAnswer":{"@type":"Answer","text":"With TorchTPU, you typically just swap '.to(\"cuda\")' for '.to(\"tpu\")' in existing PyTorch code. The backend supports eager execution and torch.compile, covers about 450 Aten ops, and includes a DeferAndFuse eager mode that runs 50% or more faster than strict eager execution. Developers porting PyTorch workloads to new hardware can follow backend updates like this on daily.dev."}}]} ```