CODA is a GPU kernel abstraction that fuses memory-bound Transformer operators (normalization, activations, residual updates, reductions) into GEMM-plus-epilogue programs. The key insight is that many operators can be algebraically reparameterized to execute while a GEMM output tile is still on-chip, avoiding costly global memory round-trips. CODA exposes composable epilogue primitives for scaling, reductions, pairwise transformations, and accumulation, covering nearly all non-attention computation in Transformer forward and backward passes. Both human- and LLM-authored CODA kernels achieve high performance across representative workloads, offering a path to combining framework productivity with hardware efficiency.
159 Impressions