做室内机器人感知与导航的团队,终于可以摆脱深度传感器的硬件依赖了——仅用RGB相机就能构建3D场景图,且主动探索效率翻倍。建议做移动机器人或固定监控系统的开发者点开看看,直接复用这个硬件无关的框架。
现有3D场景图生成方法依赖深度传感器(如LiDAR或RGB-D相机),限制了在仅有RGB相机场景下的部署。本文提出一种仅基于RGB输入的主动增量式3D场景图构建框架,统一感知与规划,利用共享的结构化表示捕捉物体语义、3D几何、关系上下文及多视角信息。实验表明,该RGB-only方法在Replica数据集上达到与使用真实深度基线相当的F1分数;在ReplicaCAD上的主动探索实验中,语义驱动的视点选择在相同探索预算下检测到的物体数量是几何前沿基线的两倍以上。此外,外部相机设置证明互补的RGB视图能有效引导场景图构建并提升上下文理解,且无需额外探索成本。
Current approaches to 3D scene graph generation rely on dedicated depth sensors, such as LiDAR or RGB-D cameras, for metric 3D reconstruction. This limits deployment to specialized robotic platforms and excludes settings where only RGB cameras are available, such as fixed external infrastructure. Existing pipelines also typically operate on passively collected observation trajectories, rather than selecting viewpoints based on the partially built scene representation, and therefore fail to effectively exploit the semantic and spatial information encoded within the graph during exploration. This paper presents a fully visual framework for the active, incremental construction of 3D scene graphs from RGB input only, addressing both limitations. The proposed approach unifies perception and planning around a shared structured representation that captures object semantics, 3D geometry, relational context, and information from multiple viewpoints. Because the framework is hardware-agnostic and relies only on RGB observations, it can incorporate inputs from both onboard robot cameras and fixed external cameras within the same representation. Experiments on the Replica dataset show that the RGB-only pipeline achieves F1-score parity with baselines using ground-truth depth. Active exploration experiments on ReplicaCAD further show that semantic-driven viewpoint selection detects more than twice as many objects as a geometric frontier-based baseline under the same exploration budget. Finally, the external-camera setting demonstrates that complementary RGB views can effectively bootstrap the scene graph and improve contextual understanding at no additional exploration cost.