Multi-token Residual Prediction

Multi-Token Residual Prediction (MRP) is a lightweight transformer module (3 layers) that accelerates diffusion language model (DLM) inference by predicting inter-step logit residuals rather than full distributions. Naive multi-token prediction collapses on DLMs beyond one step, but predicting the small correction between adjacent denoising steps is a low-complexity target a tiny module can handle. MRP serves two inference regimes: in static denoising it enables speculative decoding (up to 1.56× throughput in SGLang with lossless quality) or direct decoding (up to 1.9× with minor quality cost); in dynamic denoising it remasks over-eagerly revealed tokens using the residual signal, recovering up to +22.6 accuracy points on benchmarks like GSM8K, MATH500, HumanEval, and MBPP across SDAR 1.7B/4B/8B models. The module attaches to a frozen backbone, requires no backbone retraining, and composes with existing DLM inference methods.