Spectral Theory Explains Grokking via Weight Decay and Feature Learning
The finding that grokking time depends on the joint product of learning rate and weight decay gives practitioners a concrete dial for tuning generalization delays, not just a curiosity explanation. Researchers studying training dynamics and anyone tuning small models for algorithmic tasks can use the predicted phase structure to choose hyperparameters that avoid overfitting or failed fitting. The Transformer grid result suggests the theory holds beyond the idealized homogeneous setting.