1-Lipschitz Neural Networks on Hadamard Manifolds

精选理由

这篇论文给出了一种在非欧空间(如双曲流形、SPD流形)上构建Lipschitz可控神经网络的具体方法,想处理流形数据或者关心模型鲁棒性的朋友可以看看。

AI 摘要

论文提出一种在Hadamard流形上构造1-Lipschitz神经网络的方法,使用Busemann函数作为核心构建块,设计出梯度下降类型的1-Lipschitz层。在庞加莱圆盘上进行鲁棒分类实验,面对超几何扰动时分类器保持稳定。在对称正定(SPD)矩阵流形上训练无扩张去噪器,用于掩码Wishart协方差重建问题,结果优于静态、仅数据和对数欧几里得去噪基线。实验还验证了该去噪器的收敛性质。

原文 · arXiv cs.LG

Controlling the Lipschitz constant of a neural network is a standard way to promote robustness and stability. Most existing constraining strategies are designed for Euclidean spaces. In this work, we construct and analyze a class of 1-Lipschitz neural networks on Hadamard manifolds. Our layers are of gradient-descent type, $1$-Lipschitz, and quasi-$α$-firmly nonexpansive. The core building blocks of the proposed architecture are Busemann functions, and we exploit the properties of Busemann gradient flows to design $1$-Lipschitz geometry-preserving layers. We provide explicit constructions and examples for hyperbolic manifolds and the manifold of symmetric positive definite (SPD) matrices. We test the proposed architecture in two numerical experiments: robust classification on the Poincaré disk and masked-Wishart covariance reconstruction. On the Poincaré disk, the proposed networks yield robust classifiers under hyperbolic perturbations. On the SPD manifold, we train SPD-valued denoisers and adopt them as a Plug-and-Play prior for a masked-Wishart covariance reconstruction problem. We show improved results from the nonexpansive denoiser over static, data-only, and Log-Euclidean denoising baselines, and empirically test its convergence properties.