去噪生成的新优化框架:收敛性与防记忆化保证
An Optimization Framework for Denoising Generation
把扩散生成看成对固定密度做梯度上升,论文给了收敛距离和流形维度的具体证明,还指出避免记忆训练样本的正带宽区间。做生成模型理论或可控生成的人值得看看这套分析工具。
一篇 arXiv 论文将噪声无关的去噪生成解释为在固定 log-density 景观上的梯度上升,并回答了轨迹是否接近有意义的数据、不同初始化是否保留多样性这两个问题。
论文在显式 posterior 与景观条件下证明:上升几乎必然收敛到 reach 至少为 τ 的 m 维参考流形附近,误差在 O(mσ_min²/τ) 内;并通过 source–sink 恒等式证明有限分辨率下的区域覆盖。在仿射构造中,识别出一个正带宽区间,使经验 score 优化在训练分辨率下能避免记忆训练样本,同时保留多样性。
该固定目标天然支持 penalty 引导的可控生成,实验验证了预测机制,显示大步长引导下的鲁棒性提升,以及与有限数据生成泛化一致的下游增益。
An Optimization Framework for Denoising Generation
Noise-unconditioned denoising learns a single score, so generation can be viewed as gradient ascent on a fixed log-density landscape. This optimization interpretation leaves two questions: do trajectories approach meaningful data, and do different initializations retain diverse outcomes? We establish both phenomena for empirical and population multi-scale Gaussian-mixture objectives. Under explicit posterior and landscape conditions, ascent converges almost surely within $O(mσ_{\min}^2/τ)$ of an $m$-dimensional reference manifold with reach at least $τ$. We derive a source--sink identity for the same fixed field and use it to prove finite-resolution regional coverage. In an affine construction, we further identify a positive-bandwidth regime where even empirical-score optimization can avoid training-sample memorization while preserving diversity at the training resolution. The same fixed objective naturally supports penalty-guided controllable generation. Experiments support our predicted mechanisms, show improved robustness under large-step guidance, and demonstrate downstream gains consistent with finite-data generative generalization.
This report is categorized under AITOP's "Papers" desk. Key entities and technical topics under continuous tracking include diffusion models, score-based, controllable generation.
Source attribution: curated from arXiv cs.LG (@Xinyu Zhou, Jiawei Zhang, Stephen J. Wright).