--- title: How to Make LLM Training Faster with Unsloth and NVIDIA description: Unsloth collaborated with NVIDIA to achieve ~25% faster LLM training through three targeted optimizations. First, packed-sequence metadata caching eliminates... canonical: https://daily.dev/posts/how-to-make-llm-training-faster-with-unsloth-and-nvidia-elnnmej8y twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How to Make LLM Training Faster with Unsloth and NVIDIA | daily.dev og:description: Unsloth collaborated with NVIDIA to achieve ~25% faster LLM training through three targeted optimizations. First, packed-sequence metadata caching eliminates... og:url: https://daily.dev/posts/how-to-make-llm-training-faster-with-unsloth-and-nvidia-elnnmej8y og:image: https://api.daily.dev/og/posts/ELnNMeJ8y.png og:image:alt: How to Make LLM Training Faster with Unsloth and NVIDIA og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How to Make LLM Training Faster with Unsloth and NVIDIA **[Hacker News](https://daily.dev/sources/hn)** · 11 min read · 1 upvotes · 0 comments ## Summary Unsloth collaborated with NVIDIA to achieve ~25% faster LLM training through three targeted optimizations. First, packed-sequence metadata caching eliminates repeated reconstruction of attention masks and sequence metadata across transformer layers, yielding up to +43.3% forward pass speedup on Qwen3-14B QLoRA SFT. Second, double-buffered activation checkpoint reloading overlaps CPU-to-GPU memory copies with backward compute, delivering 4.6–8.4% step-time improvements on 8B–32B dense models with minimal VRAM overhead. Third, MoE routing is improved by replacing per-expert dynamic torch.where queries with a single bincount pass, reducing CPU-GPU synchronization overhead proportional to num_experts down to a constant, yielding 10–15% speedups on GPT-OSS configurations. All three optimizations target overhead in the 'glue code' around main kernels rather than the kernels themselves, and are automatically enabled on RTX laptops, data center GPUs, and DGX Spark machines after updating Unsloth. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [How to Fine-Tune an LLM on NVIDIA GPUs With Unsloth](https://daily.dev/posts/how-to-fine-tune-an-llm-on-nvidia-gpus-with-unsloth-lfqgsiuoc) · NVIDIA · 2 upvotes · 0 comments - [Train an LLM on an NVIDIA Blackwell Desktop with Unsloth—and Scale It](https://daily.dev/posts/train-an-llm-on-an-nvidia-blackwell-desktop-with-unsloth-and-scale-it-9ojbp127t) · NVIDIA Developer · 1 upvotes · 0 comments - [Accelerate LLM model loading and increase context windows with GPUDirect on Amazon FSx for Lustre and TurboQuant](https://daily.dev/posts/accelerate-llm-model-loading-and-increase-context-windows-with-gpudirect-on-amazon-fsx-for-lustre-an-u0pcfjh9q) · AWS · 0 upvotes · 0 comments - [5 Techniques to Optimize LLMs in Production](https://daily.dev/posts/5-techniques-to-optimize-llms-in-production-lmz4pypvm) · Daily Dose of Data Science \| Avi Chawla \| Substack · 0 upvotes · 0 comments --- Tags: [#deep-learning](https://daily.dev/tags/deep-learning), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts) [View this post on daily.dev](https://daily.dev/posts/how-to-make-llm-training-faster-with-unsloth-and-nvidia-elnnmej8y) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How to Make LLM Training Faster with Unsloth and NVIDIA","url":"https://daily.dev/posts/how-to-make-llm-training-faster-with-unsloth-and-nvidia-elnnmej8y","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-to-make-llm-training-faster-with-unsloth-and-nvidia-elnnmej8y"},"datePublished":"2026-05-07T14:10:54.682Z","dateModified":"2026-05-07T14:11:26.076Z","description":"Unsloth collaborated with NVIDIA to achieve ~25% faster LLM training through three targeted optimizations. First, packed-sequence metadata caching eliminates...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/62c4ab361d85bd2a0a1710f16efb3b63?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/62c4ab361d85bd2a0a1710f16efb3b63?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-to-make-llm-training-faster-with-unsloth-and-nvidia-elnnmej8y","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"deep-learning,mixture-of-experts","timeRequired":"PT11M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"How to Make LLM Training Faster with Unsloth and NVIDIA"}]} ```