#learning rate scheduling

3 stories · 1 in 7 days · 2 in 30 days
Stories, products and papers tagged "learning rate scheduling", newest first.
Nov 18
PapersBuildersSelected00:12
No-decay pre-training recipe gains traction across new models

Removing the decay phase turns training length into a flexible decision rather than a fixed plan, which matters for labs managing compute budgets mid-run. Teams planning large pre-training runs can weigh checkpoint averaging against cosine and WSD schedules using this compact comparison of named labs' choices. It is also a useful reference point when evaluating which labs disclose training details at all.

Official accountSebastian Raschka@rasbtSource
Oct 27
Jul 27