DART 方法修复潜空间世界模型丢失运动信息的问题
Frozen Flows Forget: Diagnosing and Restoring Lost Motion in a Latent-flow World Model
一篇很扎实的世界模型诊断论文:冻结的 flow 学不会物体运动,DART 只重训 flow 加解码监督就修好了,还顺带指出了评估指标的坑。
论文指出,在冻结自监督潜空间中训练的 flow 世界模型虽然训练稳定且成本低,但预训练 flow 不会移动被操控的物体,潜空间损失重训只会产生类似瞬移的运动。作者将问题归因于训练信号:anchor 稀疏的纯潜空间监督没有说明时间轴上变化发生的位置。提出的 decode-augmented rollout training(DART)在保持表征冻结的情况下,仅用 decode 路径监督重训 flow,恢复了运动的时序结构。DART 在完整协议上超越纯潜空间基线,较大规模下接近 oracle-informed 插值参考近一半的剩余差距,论文还发现仅用像素误差评估会奖励冻结的预测。
Frozen Flows Forget: Diagnosing and Restoring Lost Motion in a Latent-flow World Model
Latent world models that integrate a flow in a frozen self supervised latent space train stably and cheaply, yet silently lose the property manipulation depends on most: motion. The pretrained flow never moves the manipulated object; retraining it with latent-only losses only trades stillness for teleport-like motion. We trace the failure to the training signal, not the representation: anchor-sparse, latent-only supervision never says where along the horizon change belongs. Decode-augmented rollout training (DART) repairs this while keeping the representation frozen, retraining only the flow with decode-path supervision. DART outperforms its latent only parent on the full protocol, restores the temporal structure of motion, and re-couples predicted motion to the scene; at larger scale it further improves prediction quality, closing nearly half the remaining gap to an oracle-informed interpolation reference. Finally, we report an unexpected finding about evaluation: pixel error alone rewards frozen predictions.