这篇论文提出的EISAM优化器比SAM更鲁棒，调参更简单，在多个架构上准确率更高，适合追求更好泛化性能的深度学习实践者。
EISAM（Extragradient-Inspired Sharpness-Aware Minimization）是一种新型优化器，通过两步更新过程（预测步骤探索损失景观几何、扰动步骤用基础优化器细化更新）来提升泛化性能。在基准数据集上的实验表明，EISAM在测试准确率和训练效率上持续优于SGD、Adam和SAM。理论分析显示EISAM通过引导参数向曲率更小的平坦极小值收紧泛化界。此外，EISAM降低了对扰动半径的敏感性，简化了超参数调优。
Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning
Generalization remains a pivotal challenge in deep learning, where traditional optimizers like Stochastic Gradient Descent (SGD) often converge to sharp minima, leading to overfitting and reduced performance on unseen data. Building on Sharpness-Aware Minimization (SAM), for seeking flat minima associated with improved generalization, we propose the Extragradient-Inspired Sharpness-Aware Minimization (EISAM), a novel optimizer that enhances generalization via the extragradient technique. EISAM uses a two-step update process: a prediction step investigating the geometry of the loss landscape and a perturbation step that refines updates with a base optimizer. This approach achieves better generalization performance than SAM. Crucially, EISAM reduces sensitivity to the perturbation radius, enhancing robustness, and simplifying the tuning across diverse settings. Extensive experiments on benchmark datasets demonstrate that EISAM consistently outperforms SGD, Adaptive Moment Estimation (Adam), and SAM in test accuracy and training efficiency across various architectures. Theoretical analysis further confirms that EISAM tightens the generalization bound by steering parameters toward flatter minima with reduced curvature. Accompanied by a thorough hyperparameter analysis, EISAM offers practical tuning guidance, establishing it as a robust, scalable, and broadly applicable optimization solution that advances both the theory and practice in deep learning.