No-decay pre-training recipe gains traction across new models
Removing the decay phase turns training length into a flexible decision rather than a fixed plan, which matters for labs managing compute budgets mid-run. Teams planning large pre-training runs can weigh checkpoint averaging against cosine and WSD schedules using this compact comparison of named labs' choices. It is also a useful reference point when evaluating which labs disclose training details at all.