Training on the best of K guesses is a third scaling axis alongside parameters and data
A paper from UIUC and Harvard shows that generating several candidate outputs per training example and learning only from the closest match improves sample efficiency 6.2-fold, and that the benefit grows rather than shrinks as models and datasets get


