ElasticTTT:保留先验的测试时调优视频编辑方法

ElasticTTT: Prior-Preserving Test-Time Tuning for Video Editing

精选理由

这篇论文解决了视频编辑时模型丢失先验的问题,用三种新技巧让编辑更精准,效果比以前的TTT方法好得多。

AI 摘要

测试时调优(TTT)在预训练扩散模型中用于视频编辑,但存在Prior Collapse问题,导致模型丢弃文本条件和空间潜在变量。ElasticTTT通过目标分布正则化防止过拟合记忆,对比CFG引导推理远离源偏置,以及异步噪声调度保留未编辑区域。在一次性视频编辑任务上,该方法相较现有TTT方法实现了更优的语义保真度和编辑质量。

原文 · arXiv cs.AI

ElasticTTT: Prior-Preserving Test-Time Tuning for Video Editing

Test-Time Tuning (TTT) on pretrained diffusion models has emerged as a powerful paradigm for video editing. However, there exists a foundational mismatch between the distribution-mapping nature of generative models and the single-point optimization of standard TTT. In this paper, we demonstrate that this mismatch triggers \textit{Prior Collapse}, a degenerate state where the model discards the text conditions and spatial latents, collapsing generations to the source video, or entangling the features of distinct regions. To resolve this, we propose \textbf{ElasticTTT}, a novel framework that preserves the prior generative distribution and rescues generative elasticity. Specifically, we propose \textit{Target Distribution Regularization} to prevent sharp memorization minima, \textit{Contrastive CFG} to guide inference away from source biases, and \textit{Asynchronous Noise Schedule} to preserve unedited regions. Extensive evaluations, supported by theoretical analysis, demonstrate that ElasticTTT successfully preserves the generative prior of the base model, achieving state-of-the-art performance on one-shot video editing.