面向嵌入式设备RGB-D相机的可供性分割Pareto最优前沿填充

Filling the Pareto-Optimal Front for Affordance Segmentation on Embedded Devices Using RGB-D Cameras

精选理由

论文给嵌入式设备上的可供性分割提供了新思路,用Jetson Nano和RealSense做出实时方案,能耗跟手机电池差不多,比同类tiny方法更能处理复杂场景。

AI 摘要

论文提出两种方法,利用RGB-D数据在小型深度网络中整合深度信息,包括重构的硬件感知神经架构搜索和专用微调流程。两个真实世界数据集上的实验显示,多数生成的解决方案能识别Pareto最优前沿,平衡泛化性能和硬件要求。原型采用Jetson Nano和RealSense RGB-D相机,整体能耗与智能手机电池兼容,可实现实时性能。

原文 · arXiv cs.LG

Filling the Pareto-Optimal Front for Affordance Segmentation on Embedded Devices Using RGB-D Cameras

While depth sensors have the potential to complement RGB data for affordance segmentation in wearable robots, their usage seems to remain underexplored. The paper proposes two approaches: a reformulated version of hardware-aware neural architecture search, endowed with a newly designed search space to integrate depth (D) information into small-sized deep networks, and a dedicated fine-tuning approach, including a preprocessing layer to merge depth information with RGB data and make it compatible with conventional architectures. In both cases, those methods aim to generate solutions that benefit from modern (portable) hardware accelerators and overcome existing tiny-like approaches, which often fail to tackle critical scenarios due to the severe constraints set by the supporting hardware. Extensive experiments on a pair of real-world datasets demonstrate the effectiveness of the proposed method as compared with existing solutions. The approach presented in the paper generates, in most cases, solutions that identify the Pareto optimal front to balance generalization performance and hardware requirements. The paper also describes the supporting prototype, including a Jetson Nano board and a RealSense RGB-D camera. When considering the energy profile of the device, the overall system can attain real-time performances within an energy budget that is compatible with standard batteries, such as those used in smartphones.