自适应身份锚定(AIA):视频换脸合成配对监督的闭环关键帧放置

Adaptive Identity Anchoring: Closed-Loop Keyframe Placement for Synthetic Paired Supervision in Video Face Swapping

精选理由

这篇论文搞定了视频换脸的身份漂移问题,用闭环反馈自动找关键帧来固定身份,还顺便修复了皮肤太假的美颜感。比起只靠首尾帧的老方法,思路清晰多了。

AI 摘要

现有方法DreamID-V的SyncID-Pipe仅用首尾两帧合成身份锚点,导致长片段和极端姿态下身份漂移。提出的AIA泛化到任意锚点集,通过扩散迫使型Transformer支持帧条件化,并用闭环反馈在每帧评分最差的点插入图像换脸锚点。同时引入Reality-Referenced Texture Restoration,从真实视频的非面部区域转移微观纹理并做谱段传输,消除过平滑美颜效果。实验需验证锚点密度与漂移的关系、自适应与均匀放置的对比、AIA生成数据训练学生网络的收益。

原文 · arXiv cs.AI

Adaptive Identity Anchoring: Closed-Loop Keyframe Placement for Synthetic Paired Supervision in Video Face Swapping

Video face swapping has no natural paired supervision: no real footage exists of one person's face performing another person's video. The strongest current answer, DreamID-V's SyncID-Pipe, mints pairs by replacing the identity in exactly two frames of a real clip -- the first and the last -- and regenerating the rest from a pose sequence alone. Pose carries no appearance evidence of the swapped-in identity, so over long clips, occlusions, and extreme pose excursions the synthesized identity has a long unanchored span on which to drift; no published ablation examines anchor count or placement. We propose Adaptive Identity Anchoring (AIA): (i) generalize the synthesizer to arbitrary anchor sets, architecturally natural for diffusion-forcing-style transformers where conditioning on a frame is clamping its tokens to zero noise; (ii) place anchors by a closed feedback loop that scores every generated frame against the real reference identity and inserts an image-face-swapped anchor at the worst-scoring frame until the pair passes a threshold or exhausts a budget; (iii) reuse the loop's verdict as an automatic data filter. A second pathology, the beauty-filter look of over-smoothed skin, has the same root cause: micro-texture, like identity, is priced by none of the pipeline's objectives. We therefore pair AIA with Reality-Referenced Texture Restoration: matched re-graining from each real frame's non-face regions, band-split transfer of sub-identity micro-texture from the real footage, and a second, spectral acceptance channel refereed by the footage's own spectrum. Identity-anchor density, we argue, is a controllable quality dial, and we specify falsifiable experiments -- drift-versus-gap curves, uniform-versus-adaptive placement at matched budgets, student training on AIA-minted data, and texture ablations with a human beauty-filter study -- that would validate or refute the proposal.