研究证明单隐藏层 ReLU 网络集合在 W^{1,p} 中弱闭
On the (non-)closedness of the set of ReLU networks
一条关于 ReLU 网络函数空间性质的数学证明：单隐藏层弱闭、两层不闭，还推翻了 Petersen 等人的猜想，做神经网络理论分析的可以看看。
arXiv 论文 2611.09887 证明：具有一个隐藏层、至多 N 个神经元的 ReLU 网络集合在 W^{1,p}(Ω) 中是弱闭的，因此强制且弱下半连续的能量在该集合上必能取得极小值。结论覆盖 Deep Ritz 方法（凸被积函数）、Sobolev 训练和 Lipschitz learning，并给出权重衰减趋于零时正则化极小者收敛到最小参数范数极小者的结果。论文同时证明两个隐藏层的有界宽度 ReLU 网络集合在 W^{1,p}(Ω) 与 C(Ω̄) 中不闭，推翻了 Petersen、Raslan 和 Voigtlaender 的一个猜想。
On the (non-)closedness of the set of ReLU networks
We prove that the set $\mathcal{N}_N$ of ReLU networks with one hidden layer and at most $N\in\mathbb N$ neurons is weakly closed in $W^{1,p}(Ω)$, for a bounded connected open set $Ω\subset\mathbb R^d$. Consequently, every coercive, weakly lower semicontinuous energy on this space attains its minimum over $\mathcal{N}_N$. This covers the Deep Ritz method with convex integrands, Sobolev training and Lipschitz learning. Moreover, as the weight decay parameter tends to zero, minimizers of the regularized problem converge, up to subsequences, to minimizers with minimal parameter norm. We also prove that the set of ReLU networks with two hidden layers of bounded widths is not closed in $W^{1,p}(Ω)$ or in $C(\overlineΩ)$. This disproves a conjecture of Petersen, Raslan and Voigtlaender.