MDA:混合密度表示法消除深度估计中的飞点伪影

Modeling Depth Ambiguity: A Mixture-Density Representation for Flying-Point-Free Depth Estimation

精选理由

做深度估计或3D视觉的团队,边界飞点问题终于有了一个轻量且有效的解决方案——MDA几乎不增加计算成本就能大幅提升边界质量,值得在现有模型上试试。

AI 摘要

深度估计中的“飞点”问题一直难以解决:在物体边界处,模型常会在前景与背景之间的空白区域预测出虚假的3D点。研究团队发现,根本原因在于传统模型为每个像素只分配一个深度假设,而边界像素实际对应两个表面,单一假设被迫取中间值,导致飞点。他们提出MDA(混合密度表示法),让模型为每个像素预测多个深度假设及其概率,边界处不同假设可对齐不同表面,解码时从中选择而非取中间值。该方法在不同骨干网络上均显著改善边界重建,几乎消除飞点伪影,且计算开销极小。此外,MDA还能自然扩展到透明物体(预测多个深度层)和天空区域(分离无限远天空与有限深度区域),生成无飞点的天际线。

原文 · arXiv cs.AI

Modeling Depth Ambiguity: A Mixture-Density Representation for Flying-Point-Free Depth Estimation

Despite advances in depth estimation, flying points remain a persistent failure mode: near object boundaries, depth estimators often predict spurious 3D points in the empty space between foreground and background surfaces. We trace this artifact to a standard modeling choice: assigning each pixel a single depth hypothesis. At boundaries, a pixel can straddle a foreground and a background surface, so its true depth is ambiguous between the two. A model that predicts a single depth cannot keep both possibilities, so training instead pulls the prediction toward an intermediate depth that lies on neither surface. We address this with MDA, a mixture-density representation that lets the model predict multiple depth hypotheses and their associated probabilities for each pixel. Near boundaries, different hypotheses can align with different surfaces, and the decoded depth is selected from one of these hypotheses rather than placed in the empty space between them. Across different backbones, MDA substantially improves boundary reconstruction and largely removes flying-point artifacts even under severe input blur, while adding negligible runtime overhead. The same mixture-density framework naturally extends to transparent objects, where it predicts multiple depth layers at transparent pixels, and to sky regions, where a dedicated component separates the unbounded sky from finite-depth regions, producing flying-point-free skylines. Project Page: https://biansy000.github.io/mda-site/.