不用训练多次就能拿到类似深度集成的对称性提升，做数据增强的可以看看。
该论文提出用随机权重平均(SWA)替代深度集成，在增强数据上获得对称性等变性提升。作者用奥恩斯坦-乌伦贝克过程近似训练末期轨迹，证明在无限宽度极限下SWA能带来超出单独SWA性能增益的等变性提升。实验覆盖计算机视觉与图分类任务，涉及离散和连续对称性。SWA无需多轮重复训练，计算成本低于深度集成。
Boosting Data Augmentation with Stochastic Weight Averaging
The symmetries of a learning task have become an important factor in designing modern deep learning solutions. Data augmentation is a straightforward and effective way of incorporating symmetries into a generic neural network. Recent results show that infinitely large deep ensembles show perfect symmetry when trained on augmented data. However, since training ensembles requires repeating the training process many times, this method is costly. In this work, we study stochastic weight averaging (SWA) as an alternative ensembling technique that does not require repeated training runs. We analyze SWA by approximating the stochastic training trajectory at the end of training with an Ornstein--Uhlenbeck process. We show that in the infinite-width limit, SWA on augmented data provides an equiviariance boost that goes beyond what could be expected from the performance increase due to SWA alone. We verify our results with extensive numerical experiments on numerous models spanning computer vision and graph classification with both discrete and continuous symmetries.