扩散模型置信度限制研究
Limits of Confidence in Diffusion
Apple团队揭示了扩散模型在处理依赖标记时的根本限制，对生成模型设计有重要启示。
Apple研究团队发表关于离散扩散模型的论文，揭示了在像素、音素或单词等通用领域中，标记间存在固有依赖关系。研究指出，只有当写入位置在已固定标记条件下条件独立时，单步才匹配训练分布。任何单位置分布的乘积都无法匹配依赖组。
Limits of Confidence in Diffusion
Discrete diffusion, including remasking and uniform-state samplers, generate a sequence by writing multiple token positions per step, drawing each from a per-position distribution and choosing which positions to write from those same distributions. For domains of general interest (pixels, phonemes, or words) there are inherent dependencies between tokens. We show that a step matches the training distribution only when the positions it writes are conditionally independent given the tokens already fixed, that no product of per-position distributions can match a dependent group, and that…
