MIT researchers have developed a training technique called 'reinforcement learning with calibration rewards' (or 'reinforcement learning with calibration rewards') that teaches AI reasoning models to express calibrated confidence alongside their answers. Standard reinforcement learning training rewards correct answers but ignores uncertainty, causing models to become overconfident even when guessing. By adding a 'brier score' term to the reward function, the new approach penalizes the gap between stated confidence and actual accuracy. In experiments on a 7-billion-parameter model, the technique reduced calibration error by up to 90% without sacrificing accuracy, and generalized to unseen benchmarks. The method also outperformed post-hoc confidence classifiers and proved useful at inference time for selecting or weighting candidate answers.
