论文精选

可微多平面图像实现快速轻量级新视角合成

Fast and Lightweight Novel View Synthesis with Differentiable Multiplane Image

精选理由

对于需要在新视角合成中兼顾速度、模型大小和稀疏视图性能的开发者,该方法提供了一个轻量级替代方案,比3DGS快30%且模型小85%,值得在移动端或资源受限场景下尝试。

AI 摘要

本文提出一种基于可微多平面图像(MPI)的新视角合成方法,旨在平衡渲染速度、模型大小和稀疏视图下的性能。该方法利用视觉基础模型预测的点图进行几何初始化,并通过可微优化调整MPI层。为解决稀疏初始化导致的空洞和伪影问题,引入了一步扩散模型,同时参与MPI优化和渲染后处理。与基于3D高斯泼溅的代表性方法相比,该方法在正面视图场景下速度提升30.7%,模型大小仅为其14.8%,同时保持竞争性的合成质量。

原文 · arXiv cs.AI

Fast and Lightweight Novel View Synthesis with Differentiable Multiplane Image

Recently, novel view synthesis has witnessed remarkable progress, with mainstream methods such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) delivering impressive results. However, these approaches often struggle to balance rendering speed and model size, and their optimization-based training can be highly time-consuming. Furthermore, they typically rely on dense observations, often failing to produce satisfactory results under sparse-view conditions. Although feed-forward reconstruction significantly reduces the optimization time of 3DGS, its pixel-aligned formulation generates millions of Gaussians from a single image, severely limiting its practical deployment on mobile devices. To address these limitations, we revisit the Multiplane Image(MPI) representation, which represents scenes using a compact set of planar layers for efficient novel view synthesis. Leveraging recent advances in visual foundation models, we utilize predicted point maps for reliable geometric initialization, followed by differentiable optimization. To address the issues of holes and artifacts in sparsely initialized MPI, we introduce one-step diffusion, which participates in both the differentiable optimization of MPI and the postprocessing of rendering results. Compared with a representative GS-based method, our approach is 30.7% faster and uses only 14.8% of its model size, while achieving competitive synthesis quality on front-view scenarios