线性扩散模型为何学不会对称性：初始化与有限样本效应的定量分析
Quantifying the failure of symmetry learning in linear diffusion models due to initialization and finite-sample effects
如果你搞扩散模型，这篇用随机矩阵理论算清了对称性学不会的三个来源，结论是目标噪声最难消除，挺反直觉的。
一篇 arXiv 论文在线性扩散模型这一理论可解的设定下，研究生成模型学不到数据分布对称性的原因。作者识别了三种对称性破缺来源：非对称初始权重矩阵、带噪声的学习目标、有限样本噪声。结果显示目标噪声造成的破缺最顽固，即使在样本充足区间也持续存在。借助随机矩阵理论的 local laws，作者推导出一般情形下平均对称误差的公式，并证明越小的对称性越难学。
Quantifying the failure of symmetry learning in linear diffusion models due to initialization and finite-sample effects
Data distributions often possess one or more symmetries: for example, all orientations of any specific dog are included in the distribution of all possible dog pictures. Despite the omnipresence of such symmetries, modern generative modeling approaches may fail to incorporate or learn them, which can lead to lower sample efficiency and poorer generalization. To what extent do diffusion models fail to learn data distribution symmetries, and what sources of symmetry breaking are most responsible? Here, we answer this question in the theoretically tractable setting of linear diffusion models. We show that of three potential sources of symmetry breaking---a non-symmetric initial weight matrix, a noisy learning target, and finite-sample noise---target noise produces the most robust symmetry breaking, since it persists even in the sample-rich regime. Using local laws from random matrix theory, we derive formulas for average symmetry error in fairly general cases involving symmetry within a subspace and show that smaller symmetries are harder to learn.