MirrorWorld 把镜像反射生成做成视频修复任务,用 SRD 和 GTA 分别管反射内容和位置,效果比图像方法和视频修复基线都好。
MirrorWorld 提出一种反射感知的视频修复框架,用于解决视频扩散模型生成镜像反射时内容不一致的问题。其引入语义关系蒸馏(SRD),借助冻结的视觉基础模型将可见场景与镜像区域的语义关联传递给生成过程。同时设计几何变换对齐(GTA)模块,学习反射内容在镜面内的空间排列变换。在由四个现有视频镜像数据集统一构建的基准上,MirrorWorld 的反射重建质量优于基于图像的反射生成方法和强视频修复基线。
MirrorWorld: Taming Video Diffusion Models for Mirror Reflection Generation
Recent advances in video diffusion models (VDMs) have enabled high-fidelity video synthesis. However, generating mirror reflections remains challenging because the content within a mirror must remain consistent with the surrounding scene. Existing VDMs are not specifically designed to model scene-to-mirror relationships, which can lead to reflections with incorrect content or inconsistent spatial arrangements. We observe that mirror reflection generation involves two complementary challenges: determining what scene content should be reflected and how the reflected content should be spatially arranged within the mirror region. Motivated by this observation, we propose MirrorWorld, a reflection-aware video inpainting framework that models scene-to-mirror relationships during generation. Specifically, we introduce Semantic Relation Distillation (SRD), which transfers relational information from a frozen visual foundation model to encourage semantic associations between visible scene content and mirror regions. We further propose Geometric Transformation Alignment (GTA), which learns a transformation that guides the spatial arrangement of reflected content. The two components play complementary roles, with SRD modeling what should be reflected and GTA modeling how it should be arranged. To facilitate research on this problem, we construct a benchmark for video mirror reflection generation by repurposing four existing video mirror datasets into a unified reflection reconstruction task. Experimental results show that MirrorWorld achieves improved reflection reconstruction quality over representative image-based reflection generation methods and strong video inpainting baselines.