DeepSeek's New AI Speed Hack Is Amazing

DeepSeek's new research, called DSpark, introduces three improvements to speculative decoding — a technique that speeds up LLM inference by using a small 'draft' model to predict multiple tokens at once, then having the larger model verify them. The three tricks are: adding minimal memory to the draft model so tokens stay coherent, early rejection of obviously wrong draft tokens to save compute, and dynamic prediction of how many draft tokens are worth verifying based on task type (structured tasks like code/math benefit more than open-ended prompts). The result is a reported 60–85% speed-up over DeepSeek's previous multi-token prediction baseline on their Flash and Pro models. The technique is open science but requires a matching draft model and access to the target model's probabilities — it can't be bolted onto closed APIs.