研究发现神经 PDE 求解器参数无界增长导致极限解缺失问题
Singular parameters and missing limits in neural PDE solvers
搞 PDE 求解或科学计算的朋友可以看看，讲清楚了神经网络参数无界时为什么会丢极限，还给了加核导数的补全办法。
一篇 arXiv 论文分析了神经 PDE 求解器在参数无界增长时的行为，指出此时极限解在所选模型中可能没有有限表示，最优损失无法达到。论文将 tanh 深层网络的极限缺失与隐藏参数无界或神经元冗余联系起来。对于基于平移核构建的一类模型，作者通过在模型中加入核导数来补全缺失函数，使最优逼近在标准假设下可达到。数值实验跟踪了参数增长过程，并检验补全对 PDE 优化的影响。
Singular parameters and missing limits in neural PDE solvers
Neural solvers for partial differential equations (PDEs) can approach an accurate solution while their parameters grow without bound. In such cases, the limiting solution may have no finite representation in the chosen model, leaving the best loss unattained. Our analysis connects missing limits in deep neural tanh- networks to unbounded hidden parameters or increasingly redundant neurons. For a class of models built from translated kernels, we describe the missing functions and recover them by adding kernel derivatives to the model. This completion makes the best approximation attainable under standard assumptions. Numerical studies follow the associated parameter growth and explore how completion affects PDE optimization.
