不用额外训练,给离散扩散模型装上智能调度,并行生成质量更好、延迟更低。推理和代码生成都有效。
离散扩散语言模型通过并行生成令牌降低延迟,但独立选择易产生不兼容配置。本文提出训练免费解码框架,利用成对交互调整commit scores,通过变分松弛实现不动点更新。该方法无需辅助模型或重训练,可嵌入现有扩散解码流程。在推理和代码生成基准上,该方法在质量-延迟权衡方面取得一致改进。
Mean-Field Parallel Decoding for Discrete Diffusion Language Models
Discrete diffusion language models enable parallel token generation, offering a pathway to low-latency decoding. However, selecting tokens independently by marginal confidence limits effective parallelism: tokens that appear reliable in isolation can form incompatible configurations when several positions are updated at once. We introduce a training-free decoding framework that coordinates these parallel updates. At each forward pass, the method assigns a commit score to each masked position and refines these scores using pairwise interactions derived from the model's predictive distributions. A variational relaxation yields a simple fixed-point update that suppresses conflicting simultaneous commitments within a single forward pass. This mechanism allows the decoder to commit more tokens in parallel while maintaining competitive generation quality. The method is lightweight, requires no auxiliary model or retraining, and drops into existing diffusion decoding pipelines without modification. Experiments on reasoning and code-generation benchmarks show consistent improvements in the quality-latency trade-off.