这篇论文把世界模型的训练目标改成直接预测终点,DPWM在长时程预测上比递归模型准很多,而且越长优势越明显。想搞长时程决策的可以看看。
论文提出直接预测世界模型DPWM,将任意长度的动作序列压缩为单一嵌入,单次前向预测终点观测。DPWM避免了递归卷展,使长时程端到端训练在自回归训练不稳定的尺度上可行。在连续控制和像素级基准上,DPWM相比递归世界模型大幅提升长时程终点预测,且预测时程越长优势越大。实验还显示,用同样的长时程目标重训递归基线也获得类似提升,说明训练目标而非骨干结构是主要驱动力。
Beyond Myopic World Models: Long-Horizon End-to-End Training for Direct Future Prediction
World models are expected to support imagination over extended temporal horizons, yet most are still trained through local few-step prediction objectives and deployed by recursively rolling out their own predictions. This creates a fundamental mismatch: few-step losses optimize local transition fidelity, while long-horizon prediction depends on how errors and gradients propagate through the entire trajectory. As a result, transitions with different downstream influence on the endpoint are treated uniformly during training, and small local errors are amplified through recursive inference. We argue that long-horizon accuracy is better achieved by optimizing directly, through an end-to-end endpoint prediction objective. To instantiate this paradigm, we introduce the Direct Prediction World Model (DPWM), a non-recursive architecture that compresses an action sequence of arbitrary length into a single embedding and predicts the endpoint observation in a single forward pass. This design avoids recurrent rollout in both prediction and gradient propagation, making long-horizon end-to-end training practical at horizons where unrolled autoregressive training becomes unstable. Empirically, DPWM substantially improves long-horizon endpoint prediction over recursive world-model baselines on continuous-control and pixel-based benchmarks, with larger gains as the prediction horizon increases. We further show that recurrent baselines benefit similarly when retrained with the same long-horizon endpoint objective, supporting our central claim that the training objective, rather than the particular backbone choice, is the main driver of long-horizon prediction accuracy. Our results suggest that world models can benefit from being trained and evaluated at the temporal scales where they are ultimately used, shifting the focus from local transition modeling toward long-horizon predictive accuracy.