这篇论文严格验证了Berthier等人的分层学习时间尺度猜想，对理解深度网络训练动态很有帮助。
本文研究无限宽两层神经网络联合优化学习错误指定的单指标模型，通过摄动参数调节第一层和第二层的相对训练速度。作者证明常数和线性分量在预测的时间尺度内以精确显式阈值恢复。进一步分析二次分量的学习起始，并表明早期学习的分量持续影响后续动态。证明基于奇异摄动流的定量近似，现象上权重测度在达到二次分量时出现奇异行为。
Singular perturbations and hierarchical learning in two-layer neural networks
We study the population gradient flow of an infinitely wide two-layer neural network learning a misspecified single-index model in high dimension. The two layers are optimized jointly, with a perturbative parameter tuning the relative training speed between the first and second layer. This setting was considered by Berthier, Montanari and Zhou in \cite{berthier2024learning}, who conjectured a hierarchical learning scenario with explicit timescales as the second layer is trained faster than the first. In this paper, we prove that the constant and linear components of the hidden link function are indeed recovered within the predicted timescales, at sharp explicit thresholds. We then analyze the onset of learning of the quadratic component and show that the components learned at earlier stages continue to influence the dynamics in an essential way. Our proof is based on quantitative approximation results for singularly perturbed flows evolving near a manifold defined by integral constraints. At a phenomenological level, we also show that the empirical measure of the weights displays singular behaviour when reaching the quadratic component of the hidden link, with a small fraction of neurons growing significantly while the remaining ones rearrange to preserve the components already learned.