--- title: TorchNeuron brings native PyTorch support to AWS... description: Amazon has launched TorchNeuron, an Apache 2.0-licensed open-source PyTorch backend enabling existing PyTorch training code to run on Trainium2 and Trainium3... canonical: https://daily.dev/posts/torchneuron-brings-native-pytorch-support-to-aws-trainium-in-public-beta-ic3uksqdm copyright: © 2026 Daily Dev Ltd. terms: https://daily.dev/tos/ twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: TorchNeuron brings native PyTorch support to AWS Trainium in public beta | daily.dev og:description: Amazon has launched TorchNeuron, an Apache 2.0-licensed open-source PyTorch backend enabling existing PyTorch training code to run on Trainium2 and Trainium3... og:url: https://daily.dev/posts/torchneuron-brings-native-pytorch-support-to-aws-trainium-in-public-beta-ic3uksqdm og:image: https://api.daily.dev/og/posts/IC3uksqDM.png og:image:alt: TorchNeuron brings native PyTorch support to AWS Trainium in public beta og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # TorchNeuron brings native PyTorch support to AWS Trainium in public beta **[Collections](https://daily.dev/sources/collections)** · 3 min read · 0 upvotes · 0 comments ## Summary Amazon has launched TorchNeuron, an Apache 2.0-licensed open-source PyTorch backend enabling existing PyTorch training code to run on Trainium2 and Trainium3 chips with minimal code changes, often just a device setting switch. It integrates via PyTorch's PrivateUse1 extension point, supporting Hugging Face Transformers v5, TorchTitan, vLLM, and distributed training APIs like DDP and FSDP2. The beta has gaps: no sparse tensor or dynamic shape support, incomplete operator coverage, and it skips roughly 22,000 PyTorch tests. Early adopters include Cartesia, Deepgram, the Lieber Institute, and Hugging Face, with Deepgram reporting a 60% cost-efficiency gain over its GPU baseline. The move targets CUDA lock-in as the real barrier to challenging Nvidia, as Amazon's custom chip business reportedly exceeds a $25B annual revenue run rate. ## Content Amazon has released TorchNeuron, an open-source PyTorch backend for its Trainium2 and Trainium3 chips. It goes into public beta on October 20 at the PyTorch Conference, under the Apache 2.0 license. Install it with `pip install torch-neuronx`. The pitch is simple: existing PyTorch training code should run on Trainium by changing a device setting, not by rewriting the code. ## How it works TorchNeuron registers Trainium through PyTorch's PrivateUse1 extension point, so `neuron` shows up as a native device. That's a different route from the earlier optimum-neuron approach, which was built on PyTorch XLA with lazy tensors and static shapes. What you get now: - Eager execution, including a new adaptive eager mode - `torch.compile` for production - Native distributed APIs: DDP, FSDP2, DTensor, and tensor and pipeline parallelism - The Neuron Kernel Interface (NKI) for custom kernels - Neuron Explorer for profiling Because it plugs into PyTorch's own hardware extension system, existing testing and scaling tooling keeps working. Libraries such as Hugging Face Transformers v5, TorchTitan, and vLLM run with minimal changes. ## Hugging Face on Trainium The Hugging Face side is where this gets practical. Transformers, Diffusers, Accelerate, PEFT, and TRL run on Trainium using the same classes and scripts as on any other accelerator. No Neuron-specific trainer or model fork is needed. Text generation support covers 89% of the top 100 model architectures, up from 46% with Optimum Neuron v0.4.5. For many models, getting there is just `device_map=neuron`. Examples cover tensor-parallel inference with Transformers, SFT training with TRL on 8 Trainium2 chips, and tensor-parallel image generation with Diffusers pipelines (FLUX.1-dev, FLUX.2, Qwen-Image). Neuron kernels on the Hub, written in NKI, can give up to 2.4x speedups on models like vision transformers. ## Early results and gaps Cartesia, Deepgram, the Lieber Institute for Brain Development, and Hugging Face are early adopters. Deepgram reports about 60% more audio seconds processed per dollar than its GPU baseline. Testing covered 100k Hugging Face models and PyTorch's upstream test suites. The first release has holes. Operator coverage is incomplete, and sparse tensors and dynamic input shapes are explicitly unsupported for now. Roughly 22,000 PyTorch tests are skipped. Closing those gaps is planned for later releases. ## The Nvidia angle This is part of Amazon's push against Nvidia's hold on AI chips, and on CUDA in particular. Software has always been the weak spot for alternatives, and a native PyTorch device addresses exactly that. Amazon's custom chip business has reportedly passed a $25B annual revenue run rate, and Uber has also signed an AWS chip deal. Whether the beta's missing features matter will depend on the workload. For standard Hugging Face training and inference, the barrier to trying Trainium just got much lower. ## Questions this post answers ### Can I run my existing PyTorch training code on AWS Trainium chips without rewriting it? Yes, for most code the switch is just changing the device setting rather than rewriting the code. TorchNeuron, an Apache 2.0-licensed open-source backend, registers Trainium through PyTorch's PrivateUse1 extension point, so existing testing and scaling tooling keeps working, and it supports Hugging Face Transformers v5, TorchTitan, vLLM, and distributed APIs like DDP and FSDP2. _Teams weighing a move off Nvidia GPUs track PyTorch hardware backend changes like this on daily.dev._ ### What are the current limitations of AWS TorchNeuron's PyTorch support on Trainium? Sparse tensors and dynamic input shapes are explicitly unsupported, operator coverage is incomplete, and the beta release skips roughly 22,000 PyTorch tests. Amazon says it tested against 100,000 Hugging Face models and PyTorch's upstream test suites, and plans to close remaining gaps in later releases after the public beta begins 20 October at the PyTorch Conference. _Anyone evaluating a beta ML framework before committing production workloads can follow gaps like these on daily.dev._ ### How much cost savings has Deepgram reported from switching to AWS Trainium versus GPUs? Deepgram reports about 60% more audio seconds processed per dollar compared with its GPU baseline. This is a customer's own self-reported number measured against its own baseline rather than an independent benchmark, so it should be treated as a single data point rather than a general performance guarantee for other workloads. _Engineers comparing GPU and Trainium costs for their own workloads follow real adopter numbers like this on daily.dev._ ## Similar posts on daily.dev - [An exclusive tour of Amazon’s Trainium lab, the chip that’s won over Anthropic, OpenAI, even Apple](https://daily.dev/posts/an-exclusive-tour-of-amazon-s-trainium-lab-the-chip-that-s-won-over-anthropic-openai-even-apple-lkzx6swqj) · TechCrunch · 2 upvotes · 0 comments - [Run cost-effective AI workloads on OpenShift with AWS Neuron Operator](https://daily.dev/posts/run-cost-effective-ai-workloads-on-openshift-with-aws-neuron-operator-gdgyktewi) · Red Hat Developer · 0 upvotes · 0 comments - [Uber joins Amazon’s Trainium roster with AWS expansion deal](https://daily.dev/posts/uber-joins-amazon-s-trainium-roster-with-aws-expansion-deal-vtnhjpvfx) · The Next Web · 1 upvotes · 0 comments - [Transformers v5: Simple model definitions powering the AI ecosystem](https://daily.dev/posts/transformers-v5-simple-model-definitions-powering-the-ai-ecosystem-curmejncx) · Hugging Face · 31 upvotes · 0 comments --- Tags: [#aws](https://daily.dev/tags/aws), [#nvidia](https://daily.dev/tags/nvidia), [#pytorch](https://daily.dev/tags/pytorch) [View this post on daily.dev](https://daily.dev/posts/torchneuron-brings-native-pytorch-support-to-aws-trainium-in-public-beta-ic3uksqdm) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"TorchNeuron brings native PyTorch support to AWS Trainium in public beta","url":"https://daily.dev/posts/torchneuron-brings-native-pytorch-support-to-aws-trainium-in-public-beta-ic3uksqdm","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/torchneuron-brings-native-pytorch-support-to-aws-trainium-in-public-beta-ic3uksqdm"},"datePublished":"2026-10-19T16:32:17.692Z","dateModified":"2026-10-19T19:09:21.269Z","description":"Amazon has launched TorchNeuron, an Apache 2.0-licensed open-source PyTorch backend enabling existing PyTorch training code to run on Trainium2 and Trainium3...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/2749bdee6222fdb1d0d3dc0fd48f1c56?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/2749bdee6222fdb1d0d3dc0fd48f1c56?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/torchneuron-brings-native-pytorch-support-to-aws-trainium-in-public-beta-ic3uksqdm","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"aws,nvidia,pytorch","timeRequired":"PT3M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"TorchNeuron brings native PyTorch support to AWS Trainium in public beta"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/torchneuron-brings-native-pytorch-support-to-aws-trainium-in-public-beta-ic3uksqdm#faq","mainEntity":[{"@type":"Question","name":"Can I run my existing PyTorch training code on AWS Trainium chips without rewriting it?","acceptedAnswer":{"@type":"Answer","text":"Yes, for most code the switch is just changing the device setting rather than rewriting the code. TorchNeuron, an Apache 2.0-licensed open-source backend, registers Trainium through PyTorch's PrivateUse1 extension point, so existing testing and scaling tooling keeps working, and it supports Hugging Face Transformers v5, TorchTitan, vLLM, and distributed APIs like DDP and FSDP2. Teams weighing a move off Nvidia GPUs track PyTorch hardware backend changes like this on daily.dev."}},{"@type":"Question","name":"What are the current limitations of AWS TorchNeuron's PyTorch support on Trainium?","acceptedAnswer":{"@type":"Answer","text":"Sparse tensors and dynamic input shapes are explicitly unsupported, operator coverage is incomplete, and the beta release skips roughly 22,000 PyTorch tests. Amazon says it tested against 100,000 Hugging Face models and PyTorch's upstream test suites, and plans to close remaining gaps in later releases after the public beta begins 20 October at the PyTorch Conference. Anyone evaluating a beta ML framework before committing production workloads can follow gaps like these on daily.dev."}},{"@type":"Question","name":"How much cost savings has Deepgram reported from switching to AWS Trainium versus GPUs?","acceptedAnswer":{"@type":"Answer","text":"Deepgram reports about 60% more audio seconds processed per dollar compared with its GPU baseline. This is a customer's own self-reported number measured against its own baseline rather than an independent benchmark, so it should be treated as a single data point rather than a general performance guarantee for other workloads. Engineers comparing GPU and Trainium costs for their own workloads follow real adopter numbers like this on daily.dev."}}]} ```