--- title: Bringing Nunchaku 4-bit Diffusion Inference to Diffusers description: Nunchaku's SVDQuant 4-bit quantization method (W4A4) is now natively integrated into Hugging Face Diffusers via a new 'Nunchaku Lite' path. Pre-quantized... canonical: https://daily.dev/posts/bringing-nunchaku-4-bit-diffusion-inference-to-diffusers-52bez8xcq twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Bringing Nunchaku 4-bit Diffusion Inference to Diffusers | daily.dev og:description: Nunchaku's SVDQuant 4-bit quantization method (W4A4) is now natively integrated into Hugging Face Diffusers via a new 'Nunchaku Lite' path. Pre-quantized... og:url: https://daily.dev/posts/bringing-nunchaku-4-bit-diffusion-inference-to-diffusers-52bez8xcq og:image: https://api.daily.dev/og/posts/52bEz8xcq.png og:image:alt: Bringing Nunchaku 4-bit Diffusion Inference to Diffusers og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Bringing Nunchaku 4-bit Diffusion Inference to Diffusers **[Hugging Face](https://daily.dev/sources/huggingface)** · 11 min read · 0 upvotes · 0 comments ## Summary Nunchaku's SVDQuant 4-bit quantization method (W4A4) is now natively integrated into Hugging Face Diffusers via a new 'Nunchaku Lite' path. Pre-quantized checkpoints load with a standard `from_pretrained()` call — no local CUDA compilation needed, as kernels are fetched from the Hub via the `kernels` package. Nunchaku Lite patches `nn.Linear` modules with SVDQ W4A4 and AWQ W4A16 runtime layers, delivering ~30% speedup and up to 50% VRAM reduction vs BF16 baselines. Combined with `torch.compile`, speedup reaches 1.8x. NVFP4 precision requires Blackwell GPUs (RTX 50 series), while INT4 works on Turing/Ampere/Ada. The companion `diffuse-compressor` toolkit lets users quantize new architectures, package them as standard Diffusers repos, and publish to the Hub. A detailed walkthrough covers inspecting, calibrating, quantizing, packaging, and verifying a model (FLUX.2 Klein 4B as example), including handling structural rewrites like fused QKV projections. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do I load a Nunchaku 4-bit quantized diffusion model directly in Diffusers without a separate inference engine? Install a recent Diffusers version along with transformers, accelerate, and kernels packages, then call from_pretrained() on a pre-quantized checkpoint repository like any normal Diffusers pipeline, for example ErnieImagePipeline.from_pretrained("lite-infer/ERNIE-Image-Turbo-nunchaku-lite-nvfp4_r32-bnb4-text-encoder"). CUDA kernels download automatically from the Hub via the kernels package, with no local compilation needed. _Track integrations like this Nunchaku-Diffusers bridge on daily.dev when picking diffusion inference stacks._ ### What GPU hardware is required to run Nunchaku Lite NVFP4 quantized diffusion checkpoints? NVFP4 checkpoints require an NVIDIA Blackwell GPU such as the RTX 50 series, RTX PRO 6000, or B200. For earlier hardware generations like Turing, Ampere, or Ada (RTX 30 and 40 series, A100, L40S), INT4 variants of the svdq_w4a4 and awq_w4a16 schemes should be used instead; Volta and Hopper GPUs are not supported by the 4-bit kernels at all. _Developers picking quantization formats for diffusion workloads can follow hardware compatibility notes like these on daily.dev._ ### How much speedup and memory savings does Nunchaku Lite provide compared to a BF16 diffusion pipeline? On an NVIDIA RTX PRO 6000 at 1024x1024 resolution, Nunchaku Lite NVFP4 cuts peak VRAM from 31.1 GB to 20.6 GB and speeds up the full pipeline from 3.00s to 2.27s (1.35x). Adding torch.compile brings the pipeline down to 1.68s (1.8x speedup), and further quantizing the text encoder with bitsandbytes NF4 reduces peak VRAM to 16.0 GB. _Anyone benchmarking quantization trade-offs for diffusion models can keep up with results like these via daily.dev._ ## Similar posts on daily.dev - [Faster Diffusion on Blackwell: MXFP8 and NVFP4 with Diffusers and TorchAO – PyTorch](https://daily.dev/posts/faster-diffusion-on-blackwell-mxfp8-and-nvfp4-with-diffusers-and-torchao-pytorch-i2hn4mzkr) · PyTorch · 0 upvotes · 0 comments --- Tags: [#data-science](https://daily.dev/tags/data-science), [#diffusion-models](https://daily.dev/tags/diffusion-models) [View this post on daily.dev](https://daily.dev/posts/bringing-nunchaku-4-bit-diffusion-inference-to-diffusers-52bez8xcq) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Bringing Nunchaku 4-bit Diffusion Inference to Diffusers","url":"https://daily.dev/posts/bringing-nunchaku-4-bit-diffusion-inference-to-diffusers-52bez8xcq","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/bringing-nunchaku-4-bit-diffusion-inference-to-diffusers-52bez8xcq"},"datePublished":"2026-07-23T07:33:08.327Z","dateModified":"2026-09-13T21:43:06.326Z","description":"Nunchaku's SVDQuant 4-bit quantization method (W4A4) is now natively integrated into Hugging Face Diffusers via a new 'Nunchaku Lite' path. Pre-quantized...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/46c83d5c412938daa351e9b3e1a65573?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/46c83d5c412938daa351e9b3e1a65573?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hugging Face","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hugging Face","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/f1f55c67d81a4330acf5b90b26b0c8e1","url":"https://daily.dev/sources/huggingface"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/bringing-nunchaku-4-bit-diffusion-inference-to-diffusers-52bez8xcq","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-science,diffusion-models","timeRequired":"PT11M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hugging Face","item":"https://daily.dev/sources/huggingface"},{"@type":"ListItem","position":3,"name":"Bringing Nunchaku 4-bit Diffusion Inference to Diffusers"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/bringing-nunchaku-4-bit-diffusion-inference-to-diffusers-52bez8xcq#faq","mainEntity":[{"@type":"Question","name":"How do I load a Nunchaku 4-bit quantized diffusion model directly in Diffusers without a separate inference engine?","acceptedAnswer":{"@type":"Answer","text":"Install a recent Diffusers version along with transformers, accelerate, and kernels packages, then call from_pretrained() on a pre-quantized checkpoint repository like any normal Diffusers pipeline, for example ErnieImagePipeline.from_pretrained(\"lite-infer/ERNIE-Image-Turbo-nunchaku-lite-nvfp4_r32-bnb4-text-encoder\"). CUDA kernels download automatically from the Hub via the kernels package, with no local compilation needed. Track integrations like this Nunchaku-Diffusers bridge on daily.dev when picking diffusion inference stacks."}},{"@type":"Question","name":"What GPU hardware is required to run Nunchaku Lite NVFP4 quantized diffusion checkpoints?","acceptedAnswer":{"@type":"Answer","text":"NVFP4 checkpoints require an NVIDIA Blackwell GPU such as the RTX 50 series, RTX PRO 6000, or B200. For earlier hardware generations like Turing, Ampere, or Ada (RTX 30 and 40 series, A100, L40S), INT4 variants of the svdq_w4a4 and awq_w4a16 schemes should be used instead; Volta and Hopper GPUs are not supported by the 4-bit kernels at all. Developers picking quantization formats for diffusion workloads can follow hardware compatibility notes like these on daily.dev."}},{"@type":"Question","name":"How much speedup and memory savings does Nunchaku Lite provide compared to a BF16 diffusion pipeline?","acceptedAnswer":{"@type":"Answer","text":"On an NVIDIA RTX PRO 6000 at 1024x1024 resolution, Nunchaku Lite NVFP4 cuts peak VRAM from 31.1 GB to 20.6 GB and speeds up the full pipeline from 3.00s to 2.27s (1.35x). Adding torch.compile brings the pipeline down to 1.68s (1.8x speedup), and further quantizing the text encoder with bitsandbytes NF4 reduces peak VRAM to 16.0 GB. Anyone benchmarking quantization trade-offs for diffusion models can keep up with results like these via daily.dev."}}]} ```