Diffusion Models Just Beat Large Language Models?

Diffusion-based models are gaining traction as an alternative to autoregressive large language models, particularly due to their superior data efficiency. Unlike LLMs that generate tokens sequentially left-to-right, diffusion models iteratively refine their entire output, allowing corrections and improvements throughout generation. This makes them especially effective for image, video, and code generation. A key advantage is that diffusion models can reuse training data far more times (up to 100 epochs vs ~4 for LLMs) without degradation, making them valuable as training data becomes scarcer. Internally, they use variational autoencoders to map inputs into a high-dimensional vector space, then navigate toward high-value regions representing meaningful outputs. The post also includes a broader opinion that current AI systems, including diffusion models, are not approaching human-level intelligence and AGI concerns are overblown.