这篇论文提出PDD方法,能用更少的采样步数生成高质量视频和图像,而且训练比VSD简单多了,适合搞生成加速的研究者看看。
论文提出Parallel Decoding Distillation (PDD),一种基于轨迹的蒸馏方法,用于加速扩散模型和流匹配模型的推理。PDD兼容LTX-2.3 Text-to-Video/Audio、Wan 14B Text-to-Video和Qwen-Image Text-to-Image等预训练模型,支持4-8次函数评估。相比依赖变分分数蒸馏(VSD)和对抗损失的方法,PDD训练更简单,避免了模式坍塌,提升了生成视频的多样性。
Parallel Decoding Distillation for Fast Image and Video Generation
Generation in video diffusion or flow models is computationally expensive due to the slow and iterative sampling process. Current state-of-the-art (SOTA) acceleration methods heavily rely on variational score distillation (VSD) and adversarial losses to distill diffusion models into few-step generators. Albeit achieving high-quality video generation, these training losses are notoriously hard to optimize and suffer from mode collapse, leading to loss of video diversity and lack of motion. In this paper, we introduce Parallel Decoding Distillation (PDD), a simplified and scalable trajectory-based distillation method for fast inference of diffusion and flow matching models. Our architecture and training procedure are compatible with any pre-trained model and support sampling with a varying number of function evaluations (NFE). PDD accelerates generation by predicting multiple denoising steps per network evaluation. Conceptually, it learns a representation of the mean velocity without regressing its derivative using JVPs or finite-difference approximations. Our method achieves SOTA performance with 4-8 NFE on LTX-2.3 Text-to-Video/Audio, Wan 14B Text-to-Video, and Qwen-Image Text-to-Image. Moreover, PDD presents a significant improvement in generated video diversity.