vid2vid-long：基于显式对应关系的长时序一致性视频翻译方法
Consistent Video-to-Video Translation via Explicit Correspondences
实时视频翻译一直有个老大难：为了跑得快只看固定窗口，画面一久就不一致。这篇论文的思路挺巧——直接在输入帧之间找 patch 对应关系，用 cross-attention 把历史生成结果拉回来。做实时视频生成的朋友可以看看代码思路。
arXiv 论文《Consistent Video-to-Video Translation via Explicit Correspondences》提出 vid2vid-long，针对现有实时视频到视频翻译方法只关注固定长度因果窗口、历史内容超出上下文后一致性下降的问题。
该方法的核心观察是：输入视频中相互匹配的区域，在输出视频中也应当匹配。具体实现上，模型先在输入视频上计算 patch 级别对应关系，据此检索之前生成输出的对应区域，再通过 correspondence cross-attention 将这些 patch 作为条件输入模型。
实验结果显示，vid2vid-long 在多个近期视频模型上都能持续提升长程时序一致性，同时保持生成质量和实时性能。对做交互式视频生成、实时视频翻译的开发者来说，这提供了一条不依赖加长上下文窗口的替代路径。
Consistent Video-to-Video Translation via Explicit Correspondences
Interactive video-to-video applications require real-time generation while maintaining long-range temporal consistency. However, recent methods achieve speed by restricting attention to a fixed-length causal window. This limits long-range consistency whenever relevant history falls outside the context window. To address this limitation, we introduce vid2vid-long, a correspondence-based approach grounded in a simple observation. Video-to-video translation provides an explicit cross-frame reference: the regions that match in the input video should match in the output video too. Concretely, we compute patch-level correspondences in the input, guiding retrieval of previously generated outputs in the corresponding regions, and condition the model on these patches through correspondence cross-attention. Our experiments demonstrate that vid2vid-long consistently improves long-range temporal consistency across recent video models while preserving generation quality and real-time performance.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include vid2vid-long, 视频生成, 扩散模型, 一致性.
Source attribution: curated from arXiv cs.LG (@Gaurav Parmar, Zhengqi Li, Richard Zhang, Jun-Yan Zhu, Srinivasa Narasimhan, Eli Shechtman, Yotam Nitzan).