DUET:两步视频生成的质量与多样性调和

DUET: A Diversity-Quality Duet of Distillation Experts for Two-Step Video Generation

精选理由

想看两步视频生成怎么同时保质量和多样性?DUET让sCM和DMD各管一段,在Wan2.1上做到质量接近DMD、多样性翻倍,路子很巧。

AI 摘要

扩散模型视频生成采样成本高,DUET提出将轨迹级蒸馏sCM与分布级蒸馏DMD按噪声层次分工:sCM负责高噪声步保留结构多样性,DMD负责低噪声步细化外观。基于Wan2.1-T2V-1.3B,DUET让两步生成质量接近DMD,同时多样性约为DMD两倍。DUET+通过RL引导专家适配进一步提升整体质量并维持多样性优势。

原文 · arXiv cs.AI

DUET: A Diversity-Quality Duet of Distillation Experts for Two-Step Video Generation

Diffusion models have enabled high-quality video generation in recent years, but the high cost of iterative sampling hinders their practical deployment. Few-step distillation alleviates this cost, yet exposes a quality--diversity trade-off between its two dominant paradigms: trajectory-level distillation (e.g., sCM) favors diversity, whereas distribution-level distillation (e.g., DMD) favors quality. Targeting extreme two-step video generation, we introduce DUET, which reconciles the two paradigms through a noise-level duet of experts: an sCM expert takes the high-noise step to lay out diverse structure, and a DMD expert takes the low-noise step to refine appearance detail. Since the two experts are trained independently with their native objectives, DUET sidesteps the optimization difficulties of loss-level combinations and delivers quality and diversity jointly rather than trading one for the other. We further identify the relay interface and the high-noise stage as the remaining bottlenecks, and address them with RL-guided expert adaptation, yielding DUET+. With the Wan2.1-T2V-1.3B backbone, DUET lifts the two-step quality of sCM close to the level of DMD while retaining nearly all of its structural diversity---about twice that of DMD---and DUET+ further improves overall quality while preserving this diversity advantage. Together, these results establish noise-level expert specialization as a simple, effective paradigm for reconciling diversity and quality in two-step video generation.