Apple researchers propose Internalized Visual Thinking (IVT), a post-training framework for multimodal LLMs that predicts latent representations of future video frames alongside textual answers, instead of generating explicit intermediate images as Visual Chain-of-Thought does. This lets models reason proactively about video without the inference overhead of synthesizing or re-encoding frames. Across six evaluation settings, IVT outperforms text-only post-training and matches or beats Visual CoT while cutting end-to-end latency by more than 5x, suggesting pixel-space generation may be unnecessary for efficient video reasoning.
Questions this post answers
What is Internalized Visual Thinking (IVT) and how does it improve video reasoning in multimodal models?
Internalized Visual Thinking is a post-training framework that jointly optimizes textual prediction and next-embedding prediction over unlabeled videos, so a model learns to predict latent representations of future frames alongside its textual answer. At inference it skips generating or re-encoding future frames, cutting end-to-end latency by more than 5x compared to Visual Chain-of-Thought while matching or exceeding its accuracy across six evaluation settings. Track efficiency breakthroughs like this in multimodal reasoning research on daily.dev.