研究：大步长梯度下降在对抗训练中的隐式偏差与损失锐度行为
Adversarial Training with Large Step Sizes: Implicit Bias, Optimization, and Sharpness
搞对抗训练的人可以看看这篇理论分析，它证明了用大步长训练也照样成立，不用非得用慢吞吞的小步长。
一篇 arXiv 论文（编号 2611.08457）分析确定性梯度下降（GD）在对抗训练（AT）中使用大恒定步长时的动力学。作者在对抗线性逻辑回归中研究参数收敛位置（隐式偏差）和损失下降速度，并考察神经网络训练中损失锐度（sharpness）的演化。结论有三点：AT 与标准训练（ST）之间的隐式偏差差距在任意大的恒定步长下依然存在；AT 中优化误差不存在一致的收敛速率；损失锐度呈现与 ST 中 Edge-of-Stability 行为不同的两阶段动态。
Adversarial Training with Large Step Sizes: Implicit Bias, Optimization, and Sharpness
Adversarial Training (AT) is an important method for improving model robustness. We study the dynamics of deterministic Gradient Descent (GD) in AT. Most existing works in this direction focus on the small or vanishing step sizes regime, where GD resembles a continuous-time flow. However, such step sizes can make training inefficient and are rarely used in practice. In this paper, we study the AT dynamics of GD with large constant step sizes. We first study GD in adversarial linear logistic regression, focusing on two aspects: where the model parameters converge (implicit bias) and how quickly the losses decrease (optimization). We then turn to neural networks training and study how the sharpness of losses evolve during AT. Our results advance the understanding of the behaviors of GD with large step sizes in AT and highlight three key differences from its behavior in Standard Training (ST): (i) the implicit bias gap between AT and ST, previously established only for small step sizes, persists for arbitrarily large constant step sizes, (ii) different from ST, there is no uniform rate of convergence for the optimization error in AT, and (iii) sharpness can exhibit two-stage dynamics that differ from the Edge-of-Stability behavior observed in ST.