dSpark：用调度蒸馏加速扩散语言模型生成
Learning When to Commit: Schedule Distillation for Fast Diffusion Language Models
VILA-Lab 的 dSpark 让 LLaDA-8B 和 Dream-7B 这类扩散语言模型在 GSM8K 上最多快 10 倍，方法论文和代码都放出来了。
dLLM 虽能并行生成多个 token，但往往需要多轮解码才能完成回复。dSpark 提出调度蒸馏框架，学习哪些 token 应优先揭示，让其余位置更快变得可预测，并用 KL 散度正则保持生成质量。在 GSM8K 上，dSpark 相对各基线在 LLaDA-8B-Instruct 上取得 10.12 倍加速，在 Dream-7B-Instruct 上取得 5.12 倍加速，同时任务表现更优。方法还在数学推理和代码生成任务上做了评估，代码已在 GitHub 开源。
Learning When to Commit: Schedule Distillation for Fast Diffusion Language Models
Diffusion large language models (dLLMs) can generate multiple tokens in parallel, but often require many decoding rounds to complete a response. Efficient generation depends not only on which tokens are easy to predict, but also on how committing to them helps predict the remaining positions. A highly predictable token may provide little useful context, while revealing a less predictable token can help several other positions become predictable sooner. We propose dSpark, a new schedule distillation-based framework for learning when to commit tokens for faster diffusion language generation. To achieve this, we construct unmasking schedules that prioritize tokens whose reveal makes other positions easier to predict, reducing uncertainty earlier in decoding. We then distill these schedules into the model to enable faster parallel sampling, with a KL-divergence regularizer to help preserve generation quality. We extensively evaluate our approach across mathematical reasoning and code generation tasks. On GSM8K, our method achieves remarkable 10.12$\times$ and 5.12$\times$ speedups over the various baselines on LLaDA-8B-Instruct and Dream-7B-Instruct, respectively, while maintaining superior task performance. Our code is available at https://github.com/VILA-Lab/dSpark.
