BCSM：离散扩散蒸馏新目标函数，单步推理下 HumanEval 提升 16.9%
Bidirectional Centered Score Matching for Discrete Diffusion Distillation
离散扩散蒸馏有了新目标函数 BCSM，把 CSM 的平方级开销降到线性，单步推理还能在 HumanEval 上多拿 16.9%，做生成模型蒸馏的可以看.
论文提出 Bidirectional Centered Score Matching（BCSM），用于离散扩散模型的少步蒸馏。相比 token-wise KL 与完整成对形式的 Concrete Score Matching（CSM），BCSM 将成对比较复杂度从 O(|V|²) 降至 O(K)。实验显示 NFE 最高减少 6 倍，单步推理下 HumanEval 相对提升 16.9%，MMMU 相对提升 15.1%。
Bidirectional Centered Score Matching for Discrete Diffusion Distillation
Discrete diffusion models with parallel decoding are often distilled using token-wise KL objectives. Although token-wise KL exactly matches the corresponding product-form block distributions, its probability-space optimization geometry and teacher-only candidate truncation may provide insufficient supervision under aggressive few-step distillation. Concrete Score Matching (CSM) provides an alternative within-position objective by aligning relative logit geometry, but its complete pairwise form incurs $\mathcal{O}(|\mathcal{V}|^2)$ complexity. We propose Bidirectional Centered Score Matching (BCSM), an efficient within-position distillation objective that constructs an $\mathcal{O}(K)$ bidirectional active set, reduces pairwise comparisons to $\mathcal{O}(K^2)$, and factorizes them into an $\mathcal{O}(K)$ objective. We further introduce cross-timestep transition alignment. Experiments demonstrate strong quality--efficiency trade-offs, including up to $6\times$ NFE reduction with a 16.9\% relative HumanEval gain and a 15.1\% relative MMMU gain at single-step inference.
