POGP能在保持性能的同时把去噪步数砍掉近三分之二,还比现有动态扩散方法更准,做连续控制的可以看看。
扩散策略在连续控制中性能出色,但迭代去噪带来高计算成本。POGP框架通过Bellman式递归学习每个中间去噪步骤的前缀值函数。POGP的前缀值函数提供辅助训练目标,并在测试时决定是否提前停止。在MuJoCo的4个环境中与12个基线对比,POGP将所需去噪迭代减少约2.7倍,同时保留近全部任务性能。相比最先进的动态扩散基线,前缀训练还使最终任务性能提升约3.5%。
Learning When to Stop: Prefix-Optimal Dynamic Diffusion Policies for Continuous Control
Diffusion policies are a powerful policy class for continuous control, but their iterative denoising process creates a substantial computational bottleneck. Reducing this cost requires adapting the number of denoising steps to the difficulty of each action while preserving task performance. We introduce Prefix-Optimal Generative Policies (POGP), a framework that learns a prefix value function at every intermediate denoising step through a Bellman-style recursion over the denoising chain. The prefix value function serves two purposes: it provides an auxiliary training objective that encourages intermediate outputs to become high-quality actions, and it enables a test-time stopping rule that terminates denoising when additional steps are unlikely to produce meaningful improvement. Across four MuJoCo environments and comparisons with 12 baselines, POGP reduces the required number of denoising iterations by approximately 2.7-fold while retaining near-full task performance. Compared with state-of-the-art dynamic diffusion baselines, prefix training also improves final task performance by approximately 3.5%. These results indicate that supervising intermediate denoising steps is useful not only for adaptive early stopping, but also as an auxiliary objective that improves the learned policy.