论文精选73°

N0-Foundation:触觉智能新范式

$\mathcal{N}_0$-Foundation: Towards the Age of Tactile Intelligence

精选理由

N0-Foundation发布触觉智能新范式,含3万小时多模态数据和NeoForce模型,解决可变形物体操作难题。

AI 摘要

研究人员推出N0-Foundation,一种触觉驱动的具身操作范式。该系统包含基于视觉的触觉传感器、通用操作接口(UMI)和同步视觉触觉数据采集系统。团队构建了NeoData数据集,包含超过30000小时的同步视觉和触觉演示数据,涵盖6种具身形式、450项任务和数十亿对RGB与触觉帧。同时发布开源子集OpenNeoData(5000小时)和触觉表示模型NeoForce。

原文 · arXiv cs.LG

$\mathcal{N}_0$-Foundation: Towards the Age of Tactile Intelligence

We present $\mathcal{N}_0$-Foundation, a paradigm for tactile-enabled embodied manipulation, which integrates tactile sensing hardware, large-scale multimodal data, tactile representation learning, and standardized evaluation. First, we engineer the infrastructure for scalable data collection, including a vision-based tactile sensor, a tactile Universal Manipulation Interface (UMI), and a synchronized visuo-tactile data collection system supporting both robot embodiments and UMI-based demonstrations. Leveraging this infrastructure, we construct NeoData, which contains more than 30000 hours of synchronized visual and tactile demonstrations, spanning six embodiments, 450 tasks, and billions of paired RGB and tactile frames collected through a mixture of real-robot teleoperation and UMI-based demonstrations. To facilitate open research, we further release OpenNeoData, a 5000-hour open-source subset of NeoData. The dataset addresses a central limitation of existing manipulation corpora, critical for deformable-object manipulation, precise assembly, delicate force control, and sustained surface interaction. Capitalizing on the large-scale, heterogeneous tactile measurements, we propose NeoForce, a visuo-tactile representation model that learn transferable tactile representations across different sensor designs. To enable systematic evaluation of tactile embodied models built upon our infrastructure, datasets and tactile representations, we further propose a comprehensive benchmark, which combines the real-world NeoReal suite and the simulated NeoSim suite for standardized evaluation. Experiments across both suites show that policies benefit from the physical contact state rather than from the device-specific appearance of the tactile signal. We release the dataset, the representation, and the benchmark, aiming at supporting future work on tactile-enabled embodied manipulation.