离散扩散模型终于有了加速方案——FLDD 用可学习加噪替代固定链,做生成模型的研究者可以直接在少步采样场景下替换传统方法,效果提升明显。
离散扩散模型在多个领域表现优异,但传统方法使用固定马尔可夫加噪链,导致生成过程需要大量采样步骤,计算成本高。本文提出 Forward-Learned Discrete Diffusion (FLDD),引入可学习的非马尔可夫加噪过程,使生成过程在保持因子化分布的同时,能更高效地匹配目标分布。通过端到端训练所有参数,FLDD 在相同采样步数下生成质量显著优于传统离散扩散模型。实验表明,该方法在图像、文本等基准上均能实现少步高质量生成,有望加速离散扩散模型的实际应用。
Forward-Learned Discrete Diffusion: Learning how to noise to denoise faster
Discrete diffusion models are a powerful class of generative models with strong performance across many domains. For efficiency, however, discrete diffusion typically parameterizes the generative (reverse) process with factorized distributions, which makes it difficult for the model to learn the target process in a small number of steps and necessitates a long, computationally expensive sampling procedure. To reduce the gap between the target and model distributions and enable few-step generation, we propose Forward-Learned Discrete Diffusion (FLDD), which introduces discrete diffusion with a learnable forward (noising) process. Rather than fixing a Markovian forward chain, we adopt a non-Markovian formulation with learnable marginal and posterior distributions. This allows the generative process to remain factorized while matching the target defined by the noising process. We train all parameters end-to-end under the standard variational objective. Experiments on various benchmarks show that, for a given number of sampling steps, our approach produces a higher quality samples than conventional discrete diffusion models using the same reverse parameterization.