The practical payoff is a theoretical licence to merge DPO-aligned checkpoints: sparser, disentangled updates mean less interference when combining model variants. Researchers working on preference alignment or model merging can use the support-pair bounds to reason about how many samples actually shape a fine-tune. Teams choosing between DPO and SFT for multi-task pipelines get a concrete 35× interference reduction figure to weigh.
#NTK
5 stories · 0 in 7 days · 2 in 30 days
Stories, products and papers tagged "NTK", newest first.
Oct 28
Study Explains Why DPO Updates Stay Sparse via NTK Theory
Oct 27
Papers09:24
Chinese content研究揭示正交群下微分神经正切核的对称性结构用 NTK 理论分析物理信息神经网络的训练动力学,这篇把正交群对称性和谱性质证清楚了,做理论的朋友可以看看。
Oct 8
Papers10:17
Chinese contentHessian 特征值聚类之谜:对称性破缺理论给出统一解释训练好的模型 Hessian 谱总长那样?这篇论文用对称性破缺把聚簇和离群值一次讲清楚,还覆盖 transformer 和 NTK。
Sep 23
Spectral Theory Explains Grokking via Weight Decay and Feature Learning
The finding that grokking time depends on the joint product of learning rate and weight decay gives practitioners a concrete dial for tuning generalization delays, not just a curiosity explanation. Researchers studying training dynamics and anyone tuning small models for algorithmic tasks can use the predicted phase structure to choose hyperparameters that avoid overfitting or failed fitting. The Transformer grid result suggests the theory holds beyond the idealized homogeneous setting.
Aug 13
Models17:44
Chinese contentLIGHTYEAR:基于NTK的联邦学习个性化聚合新框架联邦学习遇到非独立同分布数据时,LIGHTYEAR用NTK判断哪些更新对你有用,比单纯比参数靠谱,五个数据集上都赢了。