论文精选

提出一种轻量级机器人记忆方法,通过训练时使用VLM学习可部署的隐式记忆

Workspace Models: Lightweight Robotic Memory via Saliency-Driven Supervision

精选理由

这个研究很实用,它教我们如何用VLM来训练一个轻量级的记忆模块,让机器人不用每次都重新分析所有历史数据,效率更高。

本文提出了一种名为workspace token的轻量级隐式记忆方法,通过训练时使用VLM识别任务相关的当前和历史信息,然后将其压缩成可高效查询的表示。在模拟和硬件实验中,该方法作为观察的替代品,使机器人政策能够解决需要长期记忆的任务,同时避免了部署时的VLM推理。

原文 · arXiv cs.AI

Workspace Models: Lightweight Robotic Memory via Saliency-Driven Supervision

Complex robotic manipulation tasks frequently require a long-term memory of past events and actions. As conditioning on full histories renders policies prone to spurious correlations and degrades performance, many approaches to policy memory involve compressing historical information through expensive VLM queries in-the-loop to process only task-salient information. In this paper, we propose an alternative approach in which computationally intensive VLM queries are made during train-time to learn a lightweight latent memory that can be efficiently queried at deployment time. Our representation, which we call the \textbf{workspace token}, is trained by (1) using a VLM to identify current and historical information necessary for completing a task, then (2) distilling these into the workspace token using a set-reconstruction decoder loss. In both simulation and hardware, we show that the workspace token can be used as a drop-in replacement for observations during deployment, enabling policies to solve memory-intensive tasks without the need for VLM reasoning in-the-loop. Interestingly, we found that workspace tokens are not only more lightweight but also lead to better policy performance.