研究揭示正交群下微分神经正切核的对称性结构
The Symmetry of Differential Neural Tangent Kernel under the Orthogonal Group
用 NTK 理论分析物理信息神经网络的训练动力学，这篇把正交群对称性和谱性质证清楚了，做理论的朋友可以看看。
论文研究 infinite DNTK 在正交群作用下的对称性，分别对两层网络和深层网络基于 DNTK 显式表达式与递推公式展开分析，结论覆盖带偏置与不带偏置的各类激活函数。作者证明同一群轨道上的算子诱导的球面积分算子共享谱，其 RKHS 仅相差一个正交变换。论文还给出 RePU 激活、零偏置初始化的浅层网络积分算子特征值衰减率的推导。
The Symmetry of Differential Neural Tangent Kernel under the Orthogonal Group
The differential neural tangent kernel (DNTK) is a tool for analyzing the training dynamics of overparameterized physics informed neural networks. In this work, we investigate the symmetry of the infinite DNTK under an orthogonal group, for two layer and deep neural networks separately, based on the explicit expression of the DNTK and recursive formula, respectively. The results are for all activations with or without a bias term, and shed insight into the structure of the infinite width DNTK. Then we prove that the integral operators over the unit sphere induced by operators from the same group orbit share the spectrum, and their reproducing kernel Hilbert spaces differ only by an orthogonal transformation, and that the operator fixed DNTK exhibits zonal symmetry on the unit sphere for symmetric operators. Thus the eigenvalue decay rate of the associated integral operator can be derived for shallow networks with RePU activation and zero bias initialization.