这篇论文用自验证循环替代人工标注,把世界模型的长期漂移降了44%,复合动作准确率翻近4倍,做视频生成或具身智能的可以看看。
WorldCycle提出用可逆动作循环解决视频世界模型的误差累积问题,无需标注即可验证长时程预测。该方法构造闭合动作循环,用空间闭合奖励和时间一致性奖励优化模型,使动作成为一致的状态算子。在CycleBench基准上,WorldCycle将状态返回漂移降低44%,复合动作准确率提升近4倍。
WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Models
Interactive video world models are essential for long-horizon planning and exploration, yet they suffer from compounding errors. Post-training methods such as reinforcement learning (RL) can improve these models, but they hit a verification bottleneck: for arbitrary action sequences, no ground-truth future state exists to measure long-term drift. Our key insight is that reversible action cycles make this verification possible: a sequence composed with its inverse must analytically return to the initial state, yielding annotation-free supervision on long-horizon correctness. Building on this, we introduce WorldCycle, a self-verifiable RL framework that constructs closed action cycles and their repeated executions from ordinary action sequences, and optimizes two complementary rewards: a spatial closure reward enforcing symmetry between mirrored forward and reverse segments, and a temporal consistency reward aligning states across repeated cycle executions. These rewards force the model to learn actions as consistent state operators rather than memorized temporal patterns, and extend naturally to out-of-distribution composite action cycles that the base model handles poorly. We further release CycleBench, a diagnostic benchmark for state-returning ability under complex action structures. WorldCycle reduces state returning drift by up to 44% and lifts composite-action accuracy nearly 4x over the base model, providing a vital foundation for physically grounded world models.