UGG：统一生成式模型同时搞定 4D 场景的 RGB 视频与几何
UGG: Unified Generative Geometry Model
一个模型同时出视频、深度和相机位姿，靠分模态噪声设计把生成和预测合成一件事。做 3D、视频生成或具身的开发者值得看看，稀疏观测下的几何预测成绩比拆开来的基线都强。
研究者发布 UGG，一个面向 4D 场景理解的统一生成式框架，在单一模型中联合建模 RGB 视频、深度和相机位姿。核心机制是对不同模态和不同时间步分配不同的噪声水平，把视觉生成和几何预测统一表述为条件补全任务，从而能从几何、视觉标注子集各不相同的异构数据集中学习。
推理阶段，UGG 利用从观测 RGB 帧中提取的几何证据，在多个可能的场景假设之间做搜索，在不依赖 ground truth 几何的情况下提升几何预测精度。实验覆盖相机控制视频生成、深度估计和相机位姿预测三类任务，在稀疏观测等困难场景下均超过非统一基线。
此前该领域被切分为前馈式几何预测模型和生成式视觉合成两条路线，UGG 用一个框架打通两者，代码和可视化已在项目页开源。
UGG: Unified Generative Geometry Model
Understanding 4D scenes is a core challenge in visual intelligence, requiring models to jointly capture scene geometry, appearance, and dynamics from partial observations. However, existing approaches are subdivided into feedforward models for direct geometric prediction, and generative models for visual synthesis. We introduce UGG, a unified framework for 4D scene understanding that jointly models RGB video and geometric representations such as depth and camera pose. By assigning different noise levels across modalities and time, UGG formulates visual generation and geometric prediction as conditional completion within a single generative model. This flexible formulation enables the model to learn from diverse, heterogeneous datasets with different subsets of geometric and visual annotations. Furthermore, our generative formulation enables inference-time search over multiple plausible scene hypotheses, using geometric evidence extracted from observed RGB frames to improve geometric prediction without ground truth geometry. We demonstrate UGG's effectiveness over non-unified baselines across camera-controlled video generation, depth estimation, and camera pose prediction, including in challenging sparse observation settings. Code and visualizations are available at https://ugg-model.github.io/
This report is categorized under AITOP's "Models" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include UGG, 视频生成, 深度估计, 多模态.
Source attribution: curated from arXiv cs.LG (@Amani Kiruga, Zhiyi Li, Ruojin Cai, Hansen Lillemark, Xiaoshan Wu, Ayush Tewari, Yilun Du).
