斯坦福和英伟达搞了个新方法RoboTTT，能让机器人记住5分钟内的动作历史，一次看人演示就能模仿，还能自己纠错。上下文越长效果越好，8K比1K强62%。
Stanford SVL与NVIDIA Robotics联合提出RoboTTT，基于测试时训练（TTT）方法，将机器人模型原生扩展到8000时间步的上下文（约5分钟肌肉记忆），推理成本恒定。相比以往机器人策略仅能处理数帧（<0.1秒），该方法将上下文长度提升3个数量级。在电路板组装任务中，RoboTTT通过一次人类视频演示即可实现零样本模仿学习。实验显示，8K上下文预训练性能比1K高出62%，且从128到8K时间步的闭合环路性能持续提升无饱和迹象。
I’m very excited by this test time training work for robotic learning! It’s an awesome collaboration...
I’m very excited by this test time training work for robotic learning! It’s an awesome collaboration between @StanfordSVL and @NVIDIARobotics ! Jim Fan @DrJimFan We scaled a robot model natively to 8,000 timesteps of context, 5 minutes worth of muscle memory, with constant inference cost. Robot policies used to live their lives a few frames at a time (< 0.1 sec), instantly forgetting what just happened. We pushed to 3 orders of magnitude beyond SOTA. Introducing RoboTTT. Test-Time Training (“TTT”) carries a tiny model *inside* the model. Every incoming sensor reading triggers one gradient step on that tiny core, so the history keeps getting compressed into its weights. The hidden state has a fixed size (literally a small neural net), so the robot can “grok” arbitrarily long experience with little overhead. Learning continues indefinitely after deployment. We can then put an entire video in context as prompt! RoboTTT enables one-shot in-context learning from human video: in circuit board assembly, a human demonstrates a never-seen configuration once, and the robot imitates it faithfully. Humans drop things all the time, but we pick them up so fast that we don’t even notice. That reflex to fix is half of our physical competence. RoboTTT shows self-improvement on the fly: the robot is skilled at recovering from its own errors mid-episode, and each fix enters its context to inform the next move. The TTT core distills a general-purpose, failure-to-correction mapping from the training data. One more thing. What excites me the most is a new Context Scaling Curve: from 128 to 8K timesteps, closed-loop performance hill-climbs steadily with no sign of saturation. 8K-context pretraining beats 1K by 62%. What LLM enjoys, robotics should too. Soon, even 1M context is not a fantasy. Deep dive in thread: Your browser does not support the video tag. 🔗 View on Twitter 🔗 View Quoted Tweet 💬 1 🔄 3 ❤️ 49 👀 7939 📊 6 ⚡