AI模型精选73°

RoboTTT:将机器人策略视觉上下文扩展到8000时间步

RoboTTT: Context Scaling for Robot Policies

精选理由

NVIDIA发布了RoboTTT,能把机器人策略的上下文窗口做到8000步,看懂人类演示后直接模仿,长任务完成率比单步模型高87%。想了解机器人怎么学更长历史可以看这个。

AI 摘要

RoboTTT将机器人模型的视觉运动上下文从单步或短历史扩展到8000个时间步,比最先进策略提升三个数量级且不增加推理延迟。它实现了从人类视频演示的一次性上下文模仿、在线策略改进、对扰动的鲁棒性以及在多阶段长程任务上更强的性能,并首次观察到预训练上下文长度与闭环性能的稳定提升。在真实机器人操作任务上,RoboTTT比单步上下文基线整体性能提升87%,可完全完成五分钟十阶段装配任务。使用8000时间步上下文训练的模型比1000时间步的相同模型性能高出62%,表明上下文长度是机器人基础模型的新缩放轴。

原文 · arXiv cs.AI

RoboTTT: Context Scaling for Robot Policies

Recent robot foundation models operate with single-step or short-history visuomotor context. We introduce Test-Time-Training Robot Policies (RoboTTT), a robot model and training recipe that scale visuomotor context to 8K timesteps, three orders of magnitude beyond state-of-the-art policies, without growing inference latency. At this context length, we unlock new robot capabilities: one-shot in-context imitation from human video demonstrations, on-the-fly policy improvement, robustness to perturbations, and stronger performance on multi-stage, long-horizon tasks. We also observe, for the first time, steady gains in closed-loop performance as pretraining context length scales. At its core, RoboTTT integrates Test-Time Training into robot foundation models such as Vision-Language-Action policies, yielding a sequence model whose recurrent state consists of fast weights, parameters updated by gradient descent during both training and inference, compressing histories into weight space and retrieving contextual information for long-context conditioning. To scale training context length, the recipe combines sequence action forcing with truncated backpropagation through time. On challenging real-robot manipulation tasks, RoboTTT improves overall performance by 87% over the single-step context baseline and fully completes a five-minute, ten-stage assembly task, which no baseline ever does. RoboTTT trained with 8K-timestep context outperforms the same model pretrained with 1K timesteps by 62%, suggesting context length as a new scaling axis for robot foundation models. Videos are available at https://research.nvidia.com/labs/gear/robottt/

RoboTTT:将机器人策略视觉上下文扩展到8000时间步 · AI 热点