Richard Sutton's 'Bitter Lesson' argues that scalable general methods beat hand-crafted AI systems. Modern pre-training follows this for the learner but not for task construction — the self-supervised objective is still chosen by hand outside the training loop. This CMU research proposes V-pretraining (value-based pre-training with downstream feedback), which separates the learner from a lightweight task designer. The learner updates only on unlabeled self-supervised data, while the designer uses a small set of downstream examples to learn which self-supervised tasks produce gradients aligned with downstream goals. Technically, alignment is estimated via the inner product of pretraining and downstream gradients, avoiding backpropagation through the full training run. In language experiments on Qwen models continued-pretrained on NuminaMath-CoT with GSM8K feedback, V-pretraining improves math reasoning (e.g., Qwen2.5-0.5B: 22.20→29.60 Pass@1). In vision, DINO-style backbones trained on ImageNet with ADE20K/NYUv2 feedback show improved segmentation and depth estimation without collapsing general representations. Controls rule out contamination, label smoothing, and self-distillation as explanations.
