这篇论文把离散扩散模型的理论基础讲透了，用Oracle Distance定理统一了多种损失函数，还揭示了去噪器和桥接预测器在初始化时的差异。想搞懂扩散模型原理的人别错过。
论文严格推导了连续时间马尔可夫链(CTMC)的ELBO，证明负ELBO恰好等于数据熵加上从Oracle逆过程到学习过程的路径KL散度。Oracle Distance定理指出，唯一最优解是给定当前噪声状态下真实逆跳转率的条件期望，而不可约成本是前向过程Z_t破坏关于干净数据Z_0信息的速率-d/dt I(Z_0; Z_t)。对于token-factorizing噪声，优化器有去噪器、桥接预测器和分数三种精确坐标，且存在封闭转换公式。该框架统一解释了MDM、UDM、SEDD和GIDD等方法的损失函数含义，并证明了去噪器参数化会在初始化时使均匀扩散ELBO发散，而桥接预测器保持有限。所有恒等式在精确可解模型上得到数值验证。
What Does a Discrete Diffusion Model Learn?
What does a discrete diffusion model learn: a denoiser, a score ratio, or a bridge plug-in predictor? At the level of jump rates, these are one object in different coordinates, and reading a neural network in the wrong coordinate changes the process being trained and sampled. Starting with a rigorous derivation of the continuous-time Markov chain (CTMC) ELBO for any noising process, boundary terms included, we prove the \emph{Oracle Distance} theorem: the negative ELBO is exactly equal to the data entropy plus the path KL from the oracle reverse process to the learned one, not merely a bound. Its unique optimizer is therefore the conditional expectation of the true reverse jump rate given the current noisy state, and its irreducible cost is the rate at which the forward process $Z_t$ destroys information about the clean data $Z_0$, $-\tfrac{d}{dt}I(Z_0; Z_t)$, so every noising process shares the same best achievable negative ELBO: the data entropy. For sequences with token-factorizing noise, the oracle projection yields three exact coordinates for the optimizer: denoiser, cavity (bridge plug-in), and score, with closed-form conversions among them. This framework identifies which law each loss in the literature actually optimizes, recovering MDM, UDM, SEDD, and GIDD as special cases; explains why denoiser and cavity coincide for masked diffusion but not for uniform diffusion; proves that a denoiser parameterization makes the uniform ELBO diverge at initialization while the bridge plug-in stays finite; and calibrates ELBO implementations exactly at initialization. Every identity is verified numerically, without approximation, on an exactly solvable model.