Apple 发布 Depth Pro 2:单目深度估计兼顾精度与视频时序一致性
Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth
Apple 把自家的 Depth Pro 升级到第二代,一个网络同时吃图像和视频,深度图又准又稳还不抖。之前这类工具要么清晰要么时序一致,很难两全。做 AR、3D 重建或者视频特效的可以下载权重试试。
Apple 研究团队在 arXiv 发布 Depth Pro 2,用于图像和视频的全分辨率密集度量深度估计。此前单目深度估计方法普遍存在权衡:多帧方法时序一致性好但边界偏糊,单帧方法边界锐利但视频里深度会抖动。
Depth Pro 2 的做法是训练单个可扩展网络,同时处理图像和视频输入,并以原生分辨率输出深度图,从而在度量精度、边界锐度和时序一致性三个维度上同时超越此前工作。团队发布了一整族模型,构成新的 Pareto 前沿,即在不同运行时间与精度组合下均不劣于现有方法。
代码与模型权重已在 GitHub 的 ml-depth-pro-2 仓库开源,开发者可直接下载使用。该工作对 AR、3D 重建、视频后期等依赖单目深度估计的应用有直接参考价值。
Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth
We present Depth Pro 2, an approach to estimating dense metric depth for images and video at full resolution. By analyzing prior methods for monocular depth estimation, we find that they commonly trade off temporal consistency and boundary sharpness: multi-frame methods favor consistency, single-frame methods favor sharpness. To overcome this trade-off, we train a single, scalable network that processes both images and video and predicts depth maps at native resolution. Depth Pro 2 produces depth maps that are metrically accurate, sharp, and temporally consistent all at once, outperforming prior work in all of these dimensions. We release a family of models that define a new Pareto frontier in terms of metric accuracy, sharpness, temporal consistency, and runtime. We release code and weights at https://apple-aiml-research.github.io/ml-depth-pro-2/
This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Depth Pro 2, depth estimation, Apple, open-source models.
Source attribution: curated from arXiv cs.LG (@Peiyun Hu, Marko Mihajlovic, Lyndon Duong, Lahav Lipson, Wei Dong, Amaël Delaunoy, Stephan R. Richter, Vladlen Koltun).