Chipmunk is a training-free method to accelerate Diffusion Transformers (DiTs) by exploiting two properties: activations change slowly across diffusion steps and are naturally sparse. It caches attention weights and MLP activations from previous steps, then dynamically computes sparse 'deltas' to update only the most changed values. A hardware-aware column-sparse sparsity pattern allows packing non-contiguous columns into dense shared memory tiles, enabling efficient GPU execution. Custom CUDA kernels handle fast sparsity identification, cache writeback overlapping with GEMM, and warp-specialized persistent execution. Results include up to 3.7x faster video generation on HunyuanVideo and 1.6x faster image generation on FLUX.1-dev. The code and kernels are open-sourced.
