做3D重建和神经渲染的团队终于有了一个无需相机位姿就能生成连续几何的通用方案——IVGT直接解决了显式点图冗余和几何不连续的老问题,做多视图重建的开发者值得一试。
IVGT提出了一种隐式视觉几何Transformer,能从无位姿的多视图图像中学习连续的神经场景表示。与现有方法预测显式点图不同,IVGT在规范坐标系中隐式建模连续几何,支持任意3D位置的连续空间查询。通过轻量解码器预测符号距离函数值和颜色,可直接提取连续表面几何,并渲染任意视角的RGB图、深度图和法线图。模型经多数据集联合训练,在网格/点云重建、新视角合成、深度/法线估计和相机位姿估计等任务上表现优异,展现了跨场景的泛化能力。
IVGT: Implicit Visual Geometry Transformer for Neural Scene Representation
Reconstructing coherent 3D geometry and appearance from unposed multi-view images is a fundamental yet challenging problem in computer vision. Most existing visual geometry foundation models predict explicit geometry by regressing pixel-aligned pointmaps, often suffering from redundancy and limited geometric continuity. We propose IVGT, an Implicit Visual Geometry Transformer that implicitly models continuous and coherent geometry from pose-free multi-view images. This formulation learns a continuous neural scene representation in a canonical coordinate system and supports continuous spatial queries at any 3D positions, retrieving local features to predict signed distance (SDF) values and colors using lightweight decoders. It allows direct extraction of continuous and coherent surface geometry, enabling rendering of RGB images, depth maps, and surface normal maps from arbitrary viewpoints. We train IVGT via multi-dataset joint optimization with 2D supervision and 3D geometric regularization. IVGT demonstrates generalization across scenes and achieves strong performance on various tasks, including mesh and point cloud reconstruction, novel view synthesis, depth and surface normal estimation, and camera pose estimation.