这篇论文数学证明了在组合结构任务上，神经正切核比真实神经网络差指数级样本数，实验显示两层网络在稀疏奇偶问题上测试误差低4-6个数量级。
论文在单位圆上分析神经网络优于神经正切核（NTK）的条件与幅度，提出傅里叶复杂度和结构复杂度两个度量。刻画深度L、宽度w、权重范数R的ReLU网络类的极小极大率，介于Ω(Lw^2R^2/n)和Õ(L^2w^2R^2/n)之间。当复杂度解耦时，NTK回归在深度-L迭代锯齿波上需要Ω(4^L)样本，而极小极大下界是L的多项式。数值实验表明，在超立方体稀疏奇偶模型上，两层网络比NTK测试误差低4到6个数量级。
A Function-Space Dichotomy for Compositional Learning: Exponential Sub-Optimality of the Neural Tangent Kernel
A persistent empirical observation is that trained neural networks outperform their neural tangent kernel (NTK) limit on tasks with compositional structure, yet a quantitative account of $\textbf{when}$ and $\textbf{by how much}$ has been lacking. Working on the unit circle, we give such an account through a dichotomy between two complexity measures of the target: its $\textbf{Fourier complexity}$, which controls NTK kernel regression, and its $\textbf{architectural complexity}$, which controls learning over depth-$L$, width-$w$ ReLU networks with the variation norm of the weights bounded by $R$. We first characterize the minimax rate of the architecture class $\mathcal{C}_{L,w,R}$, pinning it down up to a single factor of $L$: between $Ω(Lw^2R^2/n)$ and $\tilde{O}(L^2w^2R^2/n)$. We then show the NTK estimator sits $\textbf{exponentially}$ above this floor whenever the two complexities decouple: for the depth-$L$ iterated sawtooth, NTK regression needs $Ω(4^L)$ samples while the minimax floor is polynomial in $L$. Numerical experiments confirm the theoretical claims: on bandlimited smooth targets, the NTK is competitive or better, while on the hypercube sparse-parity model, a standard two-layer network beats the NTK by four to six orders of magnitude in test error. The gap is thus a function-space property, a mismatch between the kernel's smoothness bias and the target's compositional structure, rather than a generic kernel-versus-network phenomenon.