#NTK

5 stories · 0 in 7 days · 2 in 30 days
Stories, products and papers tagged "NTK", newest first.
Oct 28
PapersMulti-sourceSelected10:47
Study Explains Why DPO Updates Stay Sparse via NTK Theory

The practical payoff is a theoretical licence to merge DPO-aligned checkpoints: sparser, disentangled updates mean less interference when combining model variants. Researchers working on preference alignment or model merging can use the support-pair bounds to reason about how many samples actually shape a fine-tune. Teams choosing between DPO and SFT for multi-task pipelines get a concrete 35× interference reduction figure to weigh.

Event
arXiv: Anthropic@Kensuke Sasaki, Issei SatoSource
Oct 27
Oct 8
Sep 23
PapersSelected10:13
Spectral Theory Explains Grokking via Weight Decay and Feature Learning

The finding that grokking time depends on the joint product of learning rate and weight decay gives practitioners a concrete dial for tuning generalization delays, not just a curiosity explanation. Researchers studying training dynamics and anyone tuning small models for algorithmic tasks can use the predicted phase structure to choose hyperparameters that avoid overfitting or failed fitting. The Transformer grid result suggests the theory holds beyond the idealized homogeneous setting.

arXiv cs.LG@Lenz Pracher et al. (5)Source
Aug 13