WorldTrace:为视频世界模型添加可寻址记忆

Addressable Memory for Video World Models

精选理由

看这个,论文解决了视频生成模型长视频记忆丢失的问题,不重新训练就能把一致性拉高15个点,做视频生成的人应该关心。

AI 摘要

交互式视频世界模型依赖键值缓存作为视觉记忆,但生成超过训练长度后,时间旋转位置编码偏移超出训练范围,模型无法通过注意力可靠寻址已存储内容。论文提出WorldTrace,一种无需训练的长期视觉持久化记忆框架,通过为每个摘要槽分配分布内的虚拟位置保持可寻址性。WorldTrace-Field压缩历史以提升时序一致性,WorldTrace-Landmark在场景转换处存储逐字轨迹用于情节回忆。在LoopBench基准上,WorldTrace-Field将时序一致性提升15.5%,WorldTrace-Landmark将情节回忆提升19.5%。

原文 · arXiv cs.LG

Addressable Memory for Video World Models

We study visual persistence in interactive video world models. These models rely on a Key-Value (KV) cache as a growing visual memory to carry forward previously generated frames. However, we find that models can no longer reliably address stored content once rollouts extend beyond the training horizon, because temporal Rotary Positional Embeddings (RoPE) offsets then fall outside the range seen during training and the model struggles to retrieve the relevant visual information through attention. Moreover, naively compressing the cache in the RoPE-rotated space corrupts memory by averaging together incompatible positional phases. To address this, we propose WorldTrace, a training-free memory framework for long-horizon visual persistence. WorldTrace keeps compressed memory addressable by assigning each summary slot a distinct, in-distribution virtual position. Within this addressable cache, we study two memory compression approaches: WorldTrace-Field compresses history for temporal coherence, while WorldTrace-Landmark stores verbatim scene traces at detected transitions for episodic recall. We further introduce LoopBench, a benchmark evaluating whether a compressed cache can reconstruct a previously visited scene after a long detour. WorldTrace-Field improves temporal consistency by +15.5%, and WorldTrace-Landmark improves episodic recall by +19.5% on LoopBench, extending visually persistent generation without retraining.