指数残差流：为离散扩散模型构建确定性 ODE 采样器
An ODE-based Sampler for Discrete Diffusion Models
给离散扩散模型做了个 ODE 采样器，像连续扩散的概率流 ODE 一样，还能做一致性蒸馏加速采样，做生成方向的研究者可以看看。
论文提出指数残差流（ERFs），回答了离散扩散模型是否存在类似连续扩散概率流 ODE 的确定性采样形式。方法为每个 token 和位置引入连续残差变量，残差按分段定义的 ODE 演化，触及零时离散状态发生变化，并证明该构造能保持原始扩散的边际分布。作者给出了实用离散化方案，并针对 masked 扩散和 uniform 扩散做了特化。确定性轨迹支持一致性蒸馏，在代码和文本生成等多项基准上取得经验性提升。
An ODE-based Sampler for Discrete Diffusion Models
The marginals of a continuous diffusion model can be reproduced by a deterministic ordinary differential equation (ODE), known as the probability flow ODE, which provides a foundation for accelerated sampling and consistency distillation. In this paper we address the question whether an analogous formulation exists for discrete diffusion models and answer it affirmatively by introducing exponential residual flows (ERFs). Our framework augments the discrete state with a continuous residual variable for each token and position. These residuals evolve deterministically according to piecewise-defined ODEs whose vector fields depend on the current discrete state and the forward and backward rate matrices of the prescribed diffusion model. The discrete state changes whenever a residual reaches zero. We prove that this construction preserves the marginals of the original diffusion. We derive a practical discretization of our sampler and specialize it to masked and uniform diffusion. The deterministic trajectories of ERFs support consistency distillation and yield empirical improvements over previous approaches across a variety of benchmarks including code and text generation tasks.
